[HOWTO][WIP] Run Xorg on your Kobo + full-blown Firefox

[NiMa]

This is meant to be a continuation of the previous thread, "Create your own native development environment for your Kobo". I decided to separate Xorg from it because the title was not reflective of the contents of the post. I'll post more tips on here, and all my new Xorg-on-Kobo updates will be there instead.

Hi,
So after a little bit of tinkering with Alpine Linux on my Kobo (https://www.mobileread.com/forums/sh...d.php?t=336175), I really wanted to get Xorg up and running. So I got down the rabbit hole of Xorg config and video drivers, and came up with a working solution. I ended up with a working Xorg display on the eInk screen of the Kobo, with Openbox or Fluxbox and a full-blown Firefox (which worked really well) + an x11vnc session that you can access with common VNC software on your main computer. Touch input is supported! If you don't want to use that, you could opt for these methods instead:
1. Use xdotool to control mouse/keyboard from Linux CLI
2. Control the system from a VNC client.
I also tried to watch YouTube from a VNC client and it was kind of... well ugly (0.5 fps in fact).
Here's a video with the new speed improvement (instructions below): https://youtu.be/qOQt6Fxew2s
Tested devices : Kobo Glo HD, Kobo Libra H2O
*If this works on your device, please let me know, and I'll add it on the list!*
Kobo Libra H2O Overall experience very good, Firefox is able to (slowly) play mobile YouTube videos! Tested Reddit, Old Reddit, Mobileread, all pretty good! Touch experience fairly reliable, albeit very limited.
Kobo Glo HD Overall experience pretty good, Firefox is slower comparatively to the Libra H2O. You may find useful installing Midori or Falkon instead. Touch tested, reliable and working.

How do I do this?
First, head over there and make your steps through, but don't install GCC/G++/Clang if you don't want to.
TL;DR: stop at step 13, then come back here again.
Note: You may want to enable Landscape mode system-wide for a better experience:

.kobo/Kobo/Kobo eReader.conf
----
[DeveloperSettings]
ForceAllowLandscape=true

Then, download the .py file at the end of this post (attachments).

Dumping the script (xorg-kobo-touch.py) to root storage via SSH/SCP
Do this on your host machine:

scp <path of script> root@<kobo ip>:/mnt/user/home

In my case, this would be:

scp xorg-kobo-touch.py root@192.168.2.2:/mnt/user/home

Main setup
0. Kill nickel, fickel, etc. with this (ensuring you're still connected to Wi-Fi):

export DBUS_SESSION_BUS_ADDRESS NICKEL_HOME WIFI_MODULE LANG WIFI_MODULE_PATH INTERFACE
sync
killall -TERM nickel hindenburg sickel fickel fmon 2>/dev/null

1. You have likely already opened an SSH/Telnet session in the chroot. Add a repository to /etc/apk/repositories:

echo "http://dl-cdn.alpinelinux.org/alpine/edge/testing/" >> /etc/apk/repositories

Update the packages list:

apk update

Install the necessary packages:

apk add alpine-sdk xorg-server-dev libx11-dev libxdamage-dev libxfixes-dev xorg-server git xf86-video-dummy xvfb font-noto openbox fluxbox xterm firefox-esr matchbox-keyboard onboard scrot imagemagick xdotool geany tint2 python3 evtest bash pcmanfm x11vnc xf86-video-fbdev

Firefox is not mandatory, but very useful for browsing the Web. Geany & PCManFM are not required if you don't want to install them. Do a little thing for convenience :

mv /usr/share/fonts/noto/NotoSansMono*.ttf /usr/share/fonts/noto/NotoSerif*.ttf /home

2. Here we'll start by compiling a program which will allow us to see the Xorg display on the eInk screen. Start by creating a directory:

mkdir /home/root && cd /home/root

and clone the repository:

git clone --recurse-submodules https://github.com/schuhumi/fbink-xdamage

then, change directory to the cloned repo:

cd fbink-xdamage

Important: We have to modify the Makefile to allow building for Kobos, because this program was originally designed for Kindles. I'll probably fork it soon to avoid this. Open 'nano' and you'll see this somewhere :

KINDLE=1

Change it to

KOBO=1

5. Compile the program with

make

It may take a while depending on your Kobo model.

6. When all is finished and it did not exit with an error code:

X -dpi 150 &

This will start an X server with xf86-video-fbdev with a DPI set to 150. You'll *not* see anything on your Kobo's screen yet.

7. Launch fbink-xdamage (this displays X's screen updates on your eInk display with FBInk:

cd /home/root/fbink-xdamage && export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/root/fbink-xdamage/FBInk/Release/; env DISPLAY=:0 ./fbink_xdamage &

Note: if you have a Kobo Nia, currently it is not supported due to FBInk version being too old (Sep. 2019). I'm going to be testing new FBInk releases soon and will (probably) update this note.

7. Export the DISPLAY environment variable:

export DISPLAY=:0

7.1 Export a temporary HOME environment variable:

export HOME=/home/root

It will allow us to store persistent settings accross reboots.

8. Launch openbox:

openbox &

tint2 (dock)

tint2 &

9. Create a file which will scale windows to a proper "resolution":

cat > /home/root/dpi.sh << EOF
#!/bin/bash

XFT_SETTINGS="
Xft.antialias:  1
Xft.autohint:   0
Xft.dpi:        175
Xft.hinting:    1
Xft.hintstyle:  hintfull
Xft.lcdfilter:  lcddefault
Xft.rgba:       rgb
"

echo "$XFT_SETTINGS" | xrdb -merge > /dev/null 2>&1
EOF

You can change the "Xft.dpi" value to another number, but I think that 175 is ideal.

10. Launch xterm and run a command:

xterm -e "/home/root/dpi.sh;sh" &

11. Launch onboard:

onboard &

Note: you can append a -s argument to the command line (before the ampersand) to choose the size of the keyboard, e.g. "onboard -s <x>x<y> &"

12. Enable touch support:

cd /home

cd /home && evtest --grab /dev/input/event1 > evtest-log &

while true; do python3 xorg-kobo-touch.py ; done &

Note: it may throw a "ValueError: could not convert string to float: '(ABS_MT_POSITION_X)'" error, don't pay attention to it, it should disappear when you'll tap on the screen.

13. You're done! You may want to first fire up x11vnc (see below) to set up some things like the position & theme of onboard keyboard with a computer where you can properly drag things. Those settings will stay accross reboots.

Want to start this automatically with NickelMenu? Here are my launch scripts. They *may* not work. Just look at them and adjust the scripts depending on your needs.

Spoiler:

xorg-boot.sh (before chroot, this is the one NickelMenu launches):

#!/bin/sh

mount /mnt/onboard/alpine.img /mnt/user
mount -v --bind /dev /mnt/user/dev
mount -v --bind /dev/pts /mnt/user/dev/pts
mount -vt proc proc /mnt/user/proc
mount -vt sysfs sysfs /mnt/user/sys
mount -vt tmpfs tmpfs /mnt/user/run
mount -vt tmpfs tmpfs /mnt/user/tmp
mkdir /mnt/user/dev/shm
mount -vt tmpfs tmpfs /mnt/user/dev/shm
cp /etc/resolv.conf /mnt/user/etc
export DBUS_SESSION_BUS_ADDRESS NICKEL_HOME WIFI_MODULE LANG WIFI_MODULE_PATH INTERFACE
sync
killall -TERM nickel hindenburg sickel fickel fmon 2>/dev/null
chroot /mnt/user /bin/sh /xorg-chroot.sh

xorg-chroot.sh (located in the root of the Alpine chroot):

#!/bin/sh
export HOME=/home/root
mkdir /tmp/root
X -dpi 175 &
cd /home/root/fbink-xdamage
sleep 3
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/root/fbink-xdamage/FBInk/Release/; env DISPLAY=:0 ./fbink_xdamage &
sleep 3
cd /home
evtest --grab /dev/input/event1 > evtest-log &
sleep 1
while true; do python3 /home/xorg-kobo-touch.py ; done &
sleep 1
echo 20 > /sys/class/backlight/mxc_msp430.0/brightness
DISPLAY=:0 openbox &
DISPLAY=:0 tint2 &
DISPLAY=:0 onboard -s 960x307 &
DISPLAY=:0 xterm -e "/home/root/dpi.sh;sh"
DISPLAY=:0 xterm &

x11vnc
To start the VNC server, open a new prompt/window, SSH your Kobo again, *don't* close the other SSH sessions, then type:

x11vnc -display :0

After that, you'll want to install a VNC client on your main PC; I use https://tigervnc.org/ , cross-platform Windows/Mac/Linux, and it works very well.
Then ... experiment

Here are some screenshots...
Cheers!

Searching the old method? I archived it below. Note that I'll probably no longer provide support for it.

Spoiler:

This is meant to be a continuation of the previous thread, "Create your own native development environment for your Kobo". I decided to separate Xorg from it because the title was not reflective of the contents of the post. I'll post more tips on here, and all my new Xorg-on-Kobo updates will be there instead.

Hi,
So after a little bit of tinkering with Alpine Linux on my Kobo (https://www.mobileread.com/forums/sh...d.php?t=336175), I really wanted to get Xorg up and running. So I got down the rabbit hole of Xorg config and video drivers, and came up with a *kind of* working solution (warning: it is slow). I ended up with a working Xorg "display" (you'll see why below) on the eInk screen of the Kobo, with Openbox or Fluxbox and a full-blown Firefox (which worked really well) + an x11vnc session that you can access with common VNC software on your main computer. Touch input is supported! If you don't want to use that, you could opt for these methods instead:
1. Use xdotool to control mouse/keyboard from Linux CLI
2. Control the system from a VNC client.
I also tried to watch YouTube from a VNC client and it was kind of... well ugly (0.5 fps in fact).
Here's a video: https://www.youtube.com/watch?v=w_eOp1c8G98
Here's a video with the new speed improvement (instructions coming soon): https://youtu.be/qOQt6Fxew2s
Tested devices : Kobo Glo HD, Kobo Libra H2O
*If this works on your device, please let me know, and I'll add it on the list!*
Kobo Libra H2O Overall experience very good, Firefox is able to (slowly) play mobile YouTube videos! Tested Reddit, Old Reddit, Mobileread, I'd qualify it as usable. Touch experience fairly reliable, albeit very limited.
Kobo Glo HD Overall experience pretty good, Firefox is slower comparatively to the Libra H2O. You may find useful installing Midori or Falkon instead. Touch tested, reliable and working.

How do I do this?
First, head over there and make your steps through, but don't install GCC/G++/Clang if you don't want to.
TL;DR: stop at step 13, then come back here again.
Note: You may want to enable Landscape mode system-wide for a better experience:

.kobo/Kobo/Kobo eReader.conf
----
[DeveloperSettings]
ForceAllowLandscape=true

Then, download the .py file at the end of this post (attachments).

Configuring the xorg-kobo-touch.py file
You'll have to replace things in there if your Kobo is not a Libra and you're not running at 840x632 resolution (half of 1680x1264).
Lines 34-35: Remove the "/ 2" at the end of each line if you want to run at full resolution (if your Kobo has been made >2015, beware! It will be ridiculously small!)

Dumping the script to root storage via SSH/SCP
Do this on your host machine:

scp <path of script> root@<kobo ip>:/mnt/user/home

In my case, this would be:

scp xorg-kobo-touch.py root@192.168.2.2:/home

Main setup
1. You have likely already opened an SSH/Telnet session. Add a repository to /etc/apk/repositories:

echo "http://dl-cdn.alpinelinux.org/alpine/edge/testing/" >> /etc/apk/repositories

Update the packages list:

apk update

Install the necessary packages:

apk add xorg-server xf86-video-dummy xvfb font-noto openbox fluxbox xterm firefox-esr matchbox-keyboard onboard scrot imagemagick xdotool geany tint2 python3 evtest bash pcmanfm x11vnc

Firefox is not mandatory, but very useful for browsing the Web. Geany & PCManFM are not required if you don't want to install them. Do a little thing for convenience :

mv /usr/share/fonts/noto/NotoSansMono*.ttf /usr/share/fonts/noto/NotoSerif*.ttf /home

2. Initiate an Xvfb session by typing this:

Xvfb :1 -screen 0 800x600x16 &

Note: to use touch input, you have to have your resolution set at native (ex.: 1680x1264) or proportionnaly scaled (ex.: 840x632). You could also replace 800x600 by your Kobo's native resolution, but it is not properly scaled and the text will be very, very small. Then, export the DISPLAY environment variable:

export DISPLAY=:1

This step here creates a "fake" display with the Xvfb dummy video driver. It allows us to run Xorg without using a proper display, because I didn't figure out how yet.

3. Launch Openbox by typing this:

openbox &

After, type in this to run tint2 (dock)

tint2 &

and onboard (virtual keyboard for touch input | alternative: matchbox-keyboard):

onboard &

You could also opt for Fluxbox, but it is not quite what you'd want for a touchscreen device (focus on windows only when you tap the top bar... ouch)

4. (optional) Launch xterm or Firefox:

xterm &

5. Now, the fun part. Open a new shell in your host machine then SSH/Telnet to the Kobo. *Don't* close the one you were using. Enter the chroot again by typing this:

chroot /mnt/user /bin/sh

Kill Nickel, Fickel, etc. with this (totally stealed from inkvt.sh, sorry)

export DBUS_SESSION_BUS_ADDRESS NICKEL_HOME WIFI_MODULE LANG WIFI_MODULE_PATH INTERFACE
sync
killall -TERM nickel hindenburg sickel fickel fmon 2>/dev/null

Then, type in this (scrot is faster):

DISPLAY=:1 scrot --overwrite /run/screen.png

or

DISPLAY=:1 import -window root /run/screen.png

This will take a screenshot of the dummy X screen. If this finishes with a blank output and seems to create a non-empty file, then continue:

while true; do DISPLAY=:1 scrot --overwrite /run/screen.png ; done

or

while true; do DISPLAY=:1 import -window root /run/screen.png ; done

It stores it in /run which is a tmpfs so no write damage will be done to the eMMC/SD.

6. Open a fourth shell in SSH and, as always, don't close the two others. Chroot again:

chroot /mnt/user /bin/sh

then do this:

cd /home

this

export DISPLAY=:1

and to conclude, this:

evtest --grab /dev/input/event1 > evtest-log

Important: Touch input will *not* work if event1 is not the right input device. It may be as well event0 or event2. To find out which it is, do

evtest --grab /dev/input/event0

and try tapping on your Kobo's touchscreen. If it prints a bunch of info each time you tap your finger on the screen, you're good. If not, replace event0 by event1 and retry, and etc.

7. Open a third shell in SSH to your Kobo. *Don't* close the three others. Chroot again by entering this:

chroot /mnt/user /bin/sh

Then do this:

cd /home

to change the working directory to /mnt/user/home (/home in the chroot jail), this:

export DISPLAY=:1

and finally, this:

while true; do python3 xorg-kobo-touch.py ; done

Don't pay attention to the "Could not convert string to float: '(ABS_MT_POSITION_X)'" warnings. They will disappear after evtest receives touch input data.

8. Open a fourth shell in SSH to your Kobo. *Don't* close the four others. Then, *don't* chroot, and do this instead:

while true; do fbink -g file=/mnt/user/run/screen.png,w="1680",h="1264" ; done

or

watch -n 0.5 "fbink -g file=/mnt/user/run/screen.png,w="1680",h="1264""

Replace the w/h values with the ones your Kobo uses. 1680x1264 is the native resolution of my Libra.

9. Something should happen. You should see a delightful thing on the screen of your Kobo; an X desktop! And look; if you properly configured xorg-kobo-touch.py, you should have touch! Isn't that awesome?
Complaints: I know it's slow. This is a WIP, and I'm currently trying to really run a real Xorg server in my Kobo with FBInk, which would drastically increase the speed.
Other options:
What you can do is use xdotool:

xdotool type "test"

The above command types "test".

xdotool key Return

This hits the Enter key. Useful, isn't it?

x11vnc
To start the VNC server, open a new prompt/window, SSH your Kobo again, *don't* close the other SSH sessions, then type:

x11vnc -display :1

After that, you'll want to install a VNC client on your main PC; I use https://tigervnc.org/ , cross-platform Windows/Mac/Linux, and it works very well.
Then ... experiment

Here are some screenshots and I'll post a video soon.
Cheers!

Disclaimer: I am not responsible if you brick your Kobo with this technique, or if you lose files.

Special thanks to @NiLuJe and schuhumi (GitHub) for providing tips and tools during the making of this project. Now all that's left is to develop a multi-touch driver for capacitive touchscreens!
Disclaimer: I am not responsible if you brick your Kobo with this technique, or if you lose files.

[Elektron]

Woah! Awesome!

[Elektron]

Originally Posted by NiMa

I also tried to watch YouTube from a VNC client and it was kind of... well ugly (0.5 fps in fact).

That's to be expected, due to the eInk screen.

[NiMa]

Originally Posted by Elektron

That's to be expected, due to the eInk screen.

No, from a real VNC client on my Linux computer. Full-color!

[JSWolf]

Is there any way to run a full browser on a Kobo without the need for a VNC?

[Elektron]

Possibly the old Debian thread made by MarekGibek could help?

[NiMa]

Originally Posted by JSWolf

Is there any way to run a full browser on a Kobo without the need for a VNC?

That's what I'm working on currently. I'm trying to register touch inputs and pass them to xdotool as mouse movements. I found /dev/input/event1 but not that useful...

In fact, you can run Firefox without VNC (I'll post a video soon); combine that with InkVT (small terminal + tactile keyboard), and resize the on-screen image a little bit by changing the resolution parameters passed to FBInk, and you'll have a (primitive) touch experience that you can pass touch/mouse inputs with xdotool via InkVT.

But I hope that I'm going to be able to register touch inputs; it would be so awesome! At this point, you'd just have to add a virtual keyboard program for X and you'd be good to go... Just an entire operating system on your Kobo, wouldn't that be absolutely amazing?
I'm working on it...

[NiMa]

Progress!
I've successfully found a way to obtain touchscreen touch coordinates : evtest. It gives me the X/Y position of my finger on the screen and then I can pass those coordinates to xdotool, (mousemove), and then click.
More coming soon !!!!!!

[PeterT]

Might |I suggest that this (and your other HowTo) would be well suited for the MobileRead wiki....

[NiMa]

Originally Posted by PeterT

Might |I suggest that this (and your other HowTo) would be well suited for the MobileRead wiki....

That's very kind, thanks. Maybe... but here with Xorg I'm not finished with development. The other HowTo would be ready though.

[Elektron]

How do you add a page to the wiki?

[NiMa]

Originally Posted by Elektron

How do you add a page to the wiki?

This is quite difficult, I took several minutes just searching this.
You have to type an URL!!!
https://wiki.mobileread.com/wiki/Kobo_Glo_H
This will redirect you to an empty page. You can then choose to edit it. Choose the right category though

BTW touch is working

[NiMa]

Well... this was quite faster than what I would have thought this would take me time to do! But it is there! Check it out in the main post!
Sure, it's not multitouch, you can't do scrolling, but it's better than nothing I think. And with PgUp/PgDn in the virtual keyboard, scrolling webpages is not that bad I think.
If you try it, please report if it worked or not, I'd like to know!

[Elektron]

Yay!!!

[NiMa]

Originally Posted by JSWolf

Is there any way to run a full browser on a Kobo without the need for a VNC?

Now yes!