Debian on modern kindles with pre-built qemu and debian image, scripts, desktop

[theol0403]

Hello, this is my first big post. I have been lurking for awhile, and have gone and tried everything on this forum. When I started, I didn't know anything about linux, coding, or debian. Now, after a year, I know a lot more. I will release a couple tutorials in the future about stuff I learned, giving back to this forum.

One of the many things that I couldn't do at first is get debian on my kindle. However, finally I figured it out. Now I have a fully functional desktop on my kindle =).
All this information you can find on this forum, but here it is consolidated. These are all the tips that if I had In the beginning, when I didn’t know everything, I could have done a lot earlier. The only tutorial you will need for setting up debian is https://wiki.mobileread.com/wiki/Deb...n_Kindle_Touch
I will copy much from there, but add a couple steps. I have attached the completed product (debian image) of this whole post, but I will explain the process.
You will need a jailbroken kindle with usbnet.

1. Setup qemu with an armel system

Spoiler:

Ok. Here is my setup. My computer is windows, with a lubuntu virtual machine. Within that I have a qemu virtual machine emulating the same processor of the kindle (armv7; armel).

You will need an ubuntu or lubuntu (or debian) installation (probably in vmware) on your computer.

The first thing I did was try to use only ubuntu to debootstrap, but I was having errors with when I chrooted in the kindle. I got

FATAL: KERNEL TOO OLD.

Now I know how to fix that, but when I tried recently with bootstrapping in ubuntu, I was having problems with apt on my kindle.

What works for me is bootstrapping natively in the qemu, where it sets everything up without error. I also can compile and build in that qemu, so it is worth setting up.
Here is a VERY useful tutorial on how to set up qemu for armel, but I was having problems with the EXTREME wait time in installing debian onto it (internet failing and having to completely restart the process).
https://gist.github.com/Liryna/10710751#install-qemu


So what I ended up doing is using a pre-built image made from some random site, which I think was done with the exact process as the tutorial. Here are the links for the prebuilt image.
https://people.debian.org/~aurel32/qemu/armel/
We will want the debian_wheezy_armel_standard.qcow2 with the initrd.img-3.2.0-4-versatile and the vmlinuz-3.2.0-4-versatile.

STEP 1:
In your ubuntu system, create a folder for qemu. Then navigate to it in a terminal and wget these three files. (command is wget <file>).
Type “sudo apt-get install qemu” to install qemu
Then, launch the following command.

qemu-system-arm -M versatilepb -kernel vmlinuz-3.2.0-4-versatile -initrd initrd.img-3.2.0-4-versatile -hda debian_wheezy_armel_standard.qcow2 -append "root=/dev/sda1" -redir tcp:2222::22

You can put it in a script for easy access. (to do this type “nano <script name>.sh”. Paste the command. Exit (ctrl-x). Then enter “chmod +x <script name.sh>” to let you run it in the terminal).

Logins:
user = root
password = root

Now you have a full debian wheezy system with the same processor as the kindle! Files you compile with gcc will work on the kindle.


NOTE: It may come that your kernel will be updated. Notably this will give you errors with mounting, with the system not knowing what ext3 is. To fix this close qemu and run "extractkernel.txt" (sh extractkernel.txt). This will update the kernels. To see that this works look in your qemu folder and you will see much more files.

Because you typed “-redir tcp:2222::22”, you can ssh into the qemu system, so that you can do stuff such as copy and paste.
To do this type

ssh -p 2222 root@localhost

in a new terminal window.

Also, you can navigate the qemu filesystem by typing these commands:

apt-get install sshfs
mkdir qemumount
sshfs -p 2222 root@localhost:/ ./qemumount/

Now you can browse the filesystem inside qemumount

2. debootstrapping an debian image in qemu

First, you want to find out how much space you have on the kindle. I often find that I run out of space in debian, so always have the largest debian installation possible.

Type “df” on the kindle
Scroll up, and look for /mnt/us
Under available, look at the amount of space available. Divide by 1000
Now in the following command under the count, type in a bit less than that number. 
A good size is 1700 (1.7 GB).

DEBOOTSTRAP IN QEMU:

Spoiler:

This is not too different from the wiki debian tutorial.


In the qemu system, type:

dd if=/dev/zero of=/tmp/debian.ext3 bs=1M count=<SIZE WANTED THAT WILL FIT ON KINDLE>
mkfs.ext3 /tmp/debian.ext3
tune2fs -i 0 -c 0 /tmp/debian.ext3
mkdir /mnt/debian/
mount -o loop -t ext3 /tmp/debian.ext3 /mnt/debian

Next, you will fill that container. Type

apt-get install debootstrap
debootstrap --variant=minbase --arch=armel jessie /mnt/debian http://ftp.us.debian.org/debian

You will have to wait awhile.
NOTE: there are two variations from the wiki tutorial.
One: --variant=minbase will really reduce the size of the installation.
Two: instead of “testing” I put “jessie”. This is because now, testing is the sid (stretch) release, which says FATAL: KERNEL TOO OLD when you chroot on the kindle.
Jessie is the last release that supports the kindle kernel.

Unmount the image by typing:

umount /mnt/debian/

Now copy the image in /tmp/debian.ext3 to your computer (you can find it in qemumount)

NOTE: Attached is the image up to this step!!! It is smalldebian.ext3
This file is the bare minimum container size, being only 200 MB. In order to increase the size of your debian installation, follow these steps:

in ubuntu, type 
dd if=/dev/zero of=debian.ext3 bs=1M count=<SIZE WANTED THAT WILL FIT ON KINDLE>
mkfs.ext3 debian.ext3
tune2fs -i 0 -c 0 debian.ext3
sudo mkdir /mnt/debian/
sudo mkdir /mnt/smalldebian/
sudo mount -o loop -t ext3 debian.ext3 /mnt/debian/
sudo mount -o loop -t ext3 smalldebian.ext3 /mnt/smalldebian/
sudo cp -r /mnt/smalldebian/* /mnt/debian/
sudo umount /mnt/debian/
sudo umount /mnt/smalldebian/

now debian.ext3 has the contents of smalldebian.ext3 but with a larger container (potential space)



STEP 3: copy the image to kindle.


IMPORTANT: Copy the image to /mnt/base-us/ . this is to stop hangups and bricks.

STEP 4: Chroot into debian

Attached is the “chrootdebian.txt” file. Copy this to the kindle’s /mnt/us/
Turn on wifi
Now on the kindle type

 sh /mnt/us/chrootdebian.txt

You now have a working debian environment!!!

SECTION 2:--------------------------------------------

Setting up debian and installing a desktop environment.

I have made a script that installs the system described here:
https://www.mobileread.com/forums/sh...52&postcount=5


Explaining install script - you can skip:

Spoiler:

PART 1:
I will tell you how to fix some issues in debian, and install the basics, if you only want debian for building, coding, and compiling.

One error that keeps coming up is a LANG error. To fix this type the following in a chroot:

apt-get -y install locales; sed -i '/en_US.UTF-8/s/^#//' /etc/locale.gen; locale-gen en_US.UTF-8

Also to stop apt from installing unwanted software and using space, type this:

echo APT::Install-Recommends "0" ; APT::Install-Suggests "0"; >> /etc/apt/apt.conf

Also some basics to install is:

apt-get install sudo psmisc wget nano

PART 2: Installing the matchbox-desktop interface.
I will provide a run-through of the steps included, but then provide a script that installs everything.

Download everything in http://dosowisko.net/kindle/debian/ to a folder in the chroot. This folder is attached (matchbox-dos.zip)
In that folder type in the following to install dos’s hacks:

dpkg -i *.deb;apt-get -y install -f
cp gtkrc.Moko /usr/share/themes/Moko/gtk-2.0/
rm /usr/share/themes/Moko/gtk-2.0/gtkrc
mv /usr/share/themes/Moko/gtk-2.0/gtkrc.Moko /usr/share/themes/Moko/gtk-2.0/gtkrc
tar xzf ffportraitdos.tar.gz -C /usr/share/themes/
cp gtkrc /etc/gtk-2.0/
cp start-x /usr/bin/
chmod +x /usr/bin/start-x
cp startx-flipped /usr/bin/
chmod +x /usr/bin/startx-flipped
cp stop-x /usr/bin/
chmod +x /usr/bin/stop-x
mkdir /usr/share/applications/
cp stopx.desktop /usr/share/applications/
cp session /etc/matchbox/
chmod +x /etc/matchbox/session

type start-x
A desktop environment should launch

NOTE: In start-x there is the script to start the desktop environment. It starts a xephyr server over top on the current ui. Change the resolution to fit you kindle.

PART 3:

I have provided a script that does all the above steps smoothly (including fixing the locales and installing stuff)
To run copy matchbox-dos.zip and installdebian.txt to kindle /mnt/us/
NOTE: this is NOT done inside a chroot, debian needs to be unmounted
Then execute installdebian.txt ( sh /mnt/us/installdebian.txt)
It will do everything for you.

After installing you will find some files on /mnt/us/
chroot.sh is for opening a shell into an already mounted debian system.
The launch-debian files mount debian and directly starts the desktop environment.
Use the attached chrootdebian.txt (sh) file to mount and chroot debian.


Summary:

If you encounter any problems please report and if you need help with anything please ask.
Any feedback welcome.
Good Luck.

smalldebian.ext3 links:
https://mega.nz/#!4mwkzZza!V7H4EwYMTJHkNxYk8s7HdXybkQyMj1ula69nBsm CtKo

http://mir.cr/1V0UD2WN

[encol]

Nice tutorial, thanks for sharing
On what firmware version you tried this?

[theol0403]

I have a paperwhite G090 7th Generation with the latest firmware (5.8.9.1). =)

[encol]

NOTE: large libraries such as gtk or qt will not work on the kindle, because they are not installed on it.

Kindles have gtk on it, there are many gtk apps around this forum compiled for it

[theol0403]

Yes, I know that, but you have to statically build them (include many gtk files inside the program) for it to work. So compiling gtk the normal way won't work. I am currently working on building from the kindle source. Do you think I should remove that line? Thanks for the feedback

[encol]

You can easily compile gtk apps using pkg-config, e.g.:

gcc test.c -o test.out `pkg-config gtk+-2.0 --cflags --libs`

[theol0403]

Yes, but will it work on the kindle? Actually, it might. I will try right now. I thought I did but maybe not.

EDIT: You were right It works!!! I will remove that line. Thanks

[knc1]

Originally Posted by theol0403

Yes, I know that, but you have to statically build them (include many gtk files inside the program) for it to work. So compiling gtk the normal way won't work. I am currently working on building from the kindle source. Do you think I should remove that line? Thanks for the feedback

If you are writing of the touchscreen Kindle models, GTK+ is pre-installed.
Most likely only v-2, and not any v-3, but I admit I haven't checked recently.

We have a respectable number of GTK+ applications posted -
Install one and see if it works.

[theol0403]

Originally Posted by knc1

If you are writing of the touchscreen Kindle models, GTK+ is pre-installed.
Most likely only v-2, and not any v-3, but I admit I haven't checked recently.

We have a respectable number of GTK+ applications posted -
Install one and see if it works.

Alright now I know that. I have downloaded pretty much every application on this fourm, so I "knew" that it was possible. However, I thought that the applications were statistically compiled with the gtk library. All the gtk on kindle tutorials have a long gcc command that links everything. However, just right now I natively compiled in qemu a gtk app, and put it on the kindle. It worked. So now I know. Thanks.

[knc1]

The ones that where compiled statically may have been:
targeting GTK+ V-3
targeting some not so current firmware version
targeting some not so current touchscreen model
or the poster just made the same mistake you almost did.

If you hit a post by twobob -
those are most certainly correct (for the device/firmware version he was targeting).
He ported a lot of things to the Kindle while he was here.

Edit:
The mention of twobob and very long lists of libraries just sparked a memory.

Since (at the time) Kindle firmware did not ship with the build-time version control symbolic links, he would copy the firmware libraries to where he was building and then include them in the build commands, so that the linker could find them, not so the build would be static.

Edit-2:
I haven't checked recently, but I don't think QT is pre-installed.
Amazon would not want to be paying the license fees for commercial use.
QT is only free for non-commercial use.

The folks behind the Gimp ToolKit did not impose fees on the users, commercial or otherwise.

[nasser]

theol0403, Thanks a lot! I just used your method to compile a Kindle application! See: https://www.mobileread.com/forums/sh...4&postcount=48

[zaoqi]

Debian 9 ("stretch") — current stable release

[knc1]

Originally Posted by zaoqi

Debian 9 ("stretch") — current stable release

That is too new, you will have compatibility problems with the system libraries.

Even the newest firmware build, for the newest generation of Kindles, only uses GLIBC-2.20:
Ref:
https://www.mobileread.com/forums/sh...4&postcount=24

The system library used for Debian 9 is GLIBC-2.24:
Ref:
https://packages.debian.org/stretch/libc6

Translation:
Symbols are versioned.
If you create a program that requires any symbols more recent than GLIBC-20, it will not run on a Kindle.
Unless you provide those newer symbols in a library that you create yourself and use it to 'extend' the older version of the system library that is pre-installed.

If what you built runs on a Kindle, consider yourself lucky, because that is what happened, you got lucky.

Note:
It is real hard to create a 'C' language program without a reference to memcpy.
And memcpy changed at GLIBC-2.14.
Any system with an older version of the system library will be unable to provide that symbol.
Now take another look at my list of what devices / what firmware versions provide which GLIBC versions.

[theol0403]

I have the latest paperwhite with the latest firmware bought last year. Jessie is the newest version that can run on the kindle. If you try debootstrapping debian sid or later it will give you "FATAL: KERNEL TOO OLD" error. Same with compiling software on an emulator above jessie and running it on the kindle.
However, jessie is not perfect either. If I try to install and run Firefox ESR or Gnash from the jessie repositories it gives me the same error. To get them to work I have to explicitly install them from wheezy repositories. So wheezy is much better for working with the kindles except that things are older and you may have trouble compiling recent versions of software (note that jessie and jessie-backports are the current stable release, so most packages and libraies are up to date, unlike wheezy which is oldstable).

[zaoqi]

KT3 supports Debian 9!