Step by Step instructions for kindle development environment

[coplate]

I put together a few pieces, and just wanted to share the step-by-steps, so people had them.

This is specific to the i.mx6 models, basically PW2 and up.

I'll be replying with individual posts, that have 3 different methods.

For all the instructions, if not specified, just run everything as root 100% a lot of tools didnt work without being run as root.

Well, since neither of #2 or #3 have the tools needed to install more tools, tha't probably as far as my instructions will get this week.

If anyone want to help me fill out the rest of the steps, that would be good, but for now, I think Method #1 will be my gold standard - and I should be able to chroot into that, instead of the other two filesystems, to make the emulator on emulator pain easier.

Method #1 ends with me chrooting into an image, instead of using qemu to run it fully seperate.

Here is a page that covers topics related to that, including making the disk image without running the installer in qeume like I did.
https://wiki.debian.org/EmDebian/CrossDebootstrap



UPDATE 2017-07-07

"Method #2 - using the how to inside 5.8.1 source" is the method I am actually using for the kernel modules and executabled. the 5.8.9 source has a whole new method that I am going to investigate, and replace my method #3 with.

To test the programs on my PC, I can compile with regular GCC to test it, and then compile with cross-compiler to put it on the kindle.

[coplate]

This method has the fewest shortcuts.

This part gets you the qemu installed, including hte 'static' package to allow chroot much later.

1. Download a debian VM image for VMware player. I got osboxes 8.5
2. su root
3. apt-get install qemu binfmt-support qemu-user-static
4. dpkg --add-architecture armel
5. apt-get update
6. apt-get install libc6:armel

Now starting with Part 2 of this post>> http://blog.troyastle.com/2010/07/bu...n-vm-with.html

This will install a armel VM onto your vmware VM

1. wget http://http.us.debian.org/debian/dis....0-4-versatile
2. wget http://http.us.debian.org/debian/dis...boot/initrd.gz
3. qemu-img create -f raw armdisk.img 2G
4. qemu-system-arm -m 256 -M versatilepb -kernel vmlinuz-3.16.0-4-versatile -initrd initrd.gz -hda armdisk.img -append "root=/dev/ram"

That last command actually runs the installer - I chose not to install X and Gnome etc, but I wish I had -- This took many hours, because its running an emulator on an emulator.

After your install is done, you have to go back to the command line, and Kill the process with ctrl+C.

I was planning on using the chroot method, so I set up some paths, but you can pick you own paths.

This next step gets the boot loader out of the installation, for qemu to use.
the fdisk command wil have ouptut like this, s o I did 2048*512 for my start offset.

$ sudo fdisk -l armdisk.img
...
Units: sectors of 1 * 512 = 512 bytes
...
Device Boot Start End Sectors Size Id Type
armdisk.img1 2048 3913727 3911680 1.9G 83 Linux
...

1. mkdir -p /chroots/armel-jessie/
2. fdisk -l armdisk.img; # To find the sector offset of where it was installed in the .img
3. mount -o loop,offset=1048576 armdisk.img /chroots/armel-jessie/
4. cp /chroots/armel-jessie/boot/initrd.img-3.16.0-4-versatile .
5. umount /chroots/armel-jessie/

At this point above, while the image is mounted, you can copy your kindle ssh key into the directory somewhere, so that one you run ssh on the instance, you'll be able to put it in place.


Now to finally boot into that, and use it to compile for the kindle.

1. qemu-system-arm -m 256 -M versatilepb -kernel vmlinuz-3.16.0-4-versatile -initrd initrd.img-3.16.0-4-versatile -hda armdisk.img -append "root=/dev/sda1" -redir tcp:2222::22

The -redir flag lets me ssh from my Windows 10 install into the emulated emulator, without fighting with ip addresses, since my qemu running doesnt have a well resizable window.

Now that you are booted into the qemu instance - or SSH into it, you can install the libraries needed.

• apt-get install libtool build-essential ccache automake
• apt-get install xorg
• apt-get install libxtst-dev

In there wherever you find convenietn (prob
From there I grabbed something that needed to lik to libX11 and libXtst, and pushed it to my PW3 and it worked.


From here - now that everything is installed, it is time to start using the combination of qemu-arm-static and chroot.

qemu-arm-static lets you run crosscompiled code on your own computer, that's why we installed it in the first aprt commands.

• sudo mount -o loop,offset=1048576 armdisk.img /chroots/armel-jessie/
• sudo cp /usr/bin/qemu-arm-static /chroots/armel-jessie/usr/bin/
• sudo chroot /chroots/armel-jessie/
• gcc mouseturn.c -lX11 -lXtst
• scp a.out 192.168.2.57:/mnt/us
• ssh 192.168.2.57

# [root@kindle us]# cd /mnt/us
# [root@kindle us]# ./a.out /dev/input/event3 1:1,1
button: 272, point -1 -1
button: 273, point -1 -1

it looks like this works too - now I don't need to use qemu.

[Cinisajoy]

Does this work on any e-ink kindle?
Does it have to be jailbroke first?

[coplate]

UPDATE 2017-07-07

Here are my new instructions I actually follow

1. Download the Paperwhite 5.8.1 source from amazon
2. grab the tar.gz build_linaro-gcc_4.8.3.tar.gz
3. grab the tar.gz linux-3.0.35.tar.gz
4. Extract those to your computer
5. Build the linaro compiler using the 'BUILD_HOW-TO.txt' in the package
   • The howto will explain a lot of dependencies you need to install
   • I change the PREFIX in the makefile to '$(HOME)/kindle/opt/cross-gcc-linaro'
   • In the howto, the last step you need to follow for this part is 'Install the Kernel Headers'
     1. apt install < the list of dependencies>
     2. make
     3. mkdir -p $HOME/kindle/opt
     4. tar -C $HOME/kindle/opt -xjf cross-arm-linux-gnueabi-gcc-linaro-4.8-2014.04.tar.gz
     5. tar -C $HOME/kindle/opt/cross-gcc-linaro/arm-linux-gnueabi -xzf khdrs.tar.gz
6. Build the linux kernel using the 'BUILD_HOW-TO.txt'
   • make ARCH=arm CROSS_COMPILE=$HOME/kindle/opt/cross-gcc-linaro/bin/arm-linux-gnueabi- imx60_wario_defconfig
   • make ARCH=arm CROSS_COMPILE=$HOME/kindle/opt/cross-gcc-linaro/bin/arm-linux-gnueabi-

Amazon source download page: https://www.amazon.com/gp/help/custo...deId=200203720


I used the 5.8.1 source, just because I was trying to get an older one that would work on any version of my kindle. I had originally tried the 5.7.4 source, but the instructions didn't seem to work right, or there were some missing files, so I moved up 1 step.

Making the linux kernel is only necessary for the kernel modules, the programs can be built just by installing that linaro cross compiler.

This is my makefile for Kernel modules

export ARCH:=arm
export CROSS_COMPILE:=/home/osboxes/kindle/opt/cross-gcc-linaro/bin/arm-linux-gnueabi-

#obj-m := hello-2.o
#obj-m := evdev_mouse.o
#obj-m := ev_mdc.o
obj-m := tkbd.o

KDIR := /home/osboxes/kindle/gplrelease/linux-3.0.35

PWD := $(shell pwd)


EXTRA_CFLAGS := -I$(src)/drivers/


default:
	$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
clean:
	$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) clean

This is my makefile for programs

# i used amazon's own toolchain, from kindle fw sources
CROSS_COMPILE = /home/osboxes/kindle/opt/cross-gcc-linaro/bin/arm-linux-gnueabi-
CFLAGS = 
LDFLAGS =
SRC = tpoint.c
OUT = tpoint.lix

# -print-search-dirs

kindle: $(SRC)
	$(CROSS_COMPILE)gcc -DBUILD_KINDLE $(SRC) -o $(OUT) $(CFLAGS) $(LDFLAGS)

host: $(SRC)
	gcc -DBUILD_KINDLE $(SRC) -o $(OUT) $(CFLAGS) $(LDFLAGS)

clean:
	rm $(OUT)

Old copy of method 2 ( twobobs pre-set qemu image ) is included below, but that file is no longer on the internet.

Spoiler:

Method #2 - twobobs pre-set qemu image
Starting with this image twobob linked:
https://www.mobileread.com/forums/sh...46&postcount=6

1. mkdir /chroots/armel-twobob
2. wget https://dl.dropboxusercontent.com/u/...b.5gb.ext3.zip
3. unzip deb.5gb.ext3.zip
4. FILENAME=/mnt/base-us/deb.5gb.ext3
5. dd if=/dev/zero of=$FILENAME bs=1M count=1500 oflag=append conv=notrunc
6. e2fsck -f $FILENAME
7. resize2fs $FILENAME
   

This sets up that build, but it doesn't have apt-get installed when I ran it with qemu
I haven't finished doing it yet, but the instruction here indicate that this was used in chroot, and to compile you set them up from inside the chroot, using apt-get from outside it, but its at this point I stopped.

These instructions are also from a thread for getting an updated debian onto the kindle itself, so I'm not 100% sure I'm on the right path with this.


https://www.mobileread.com/forums/sh...34#post2633334

1. mount -o loop deb.5gb.ext3 /chroots/armel-twobob/
2. chroot /chroots/armel-twobob/
   

Originally Posted by twobob

deb.5gb.ext3 is a general purpose image that WILL need the additional steps outlined above doing to make it a "build system", when you are bored of it or you screw it up just replace it again. (EDIT: I did this just last night)

Enjoy


The script to mount this partition is available on the DEBIAN ON A PAPERWHITE thread (and ill post it again below)

Originally Posted by twobob

root@kindle:/# apt-get install libtool build-essential ccache automake
I would guess that this would, with the debian grip guts, weigh in around 320 MB)

(then inside debian)
root@kindle:/# mkdir -p /mnt/us/extensions
(on my system that becomes /mnt/us/debian/mnt/us/extensions)

build stuff like:
root@kindle:/# ./configure --prefix=/mnt/us/extensions/{thing}/usr <other options>

root@kindle:/# make clean; PATH=/usr/lib/ccache:$PATH make ; make install

then MOVE(/COPY) the /mnt/us/debian/mnt/us/extensions stuff to /mnt/us/extensions from outside the chroot.

[coplate]

I think this will end up haveing several of the same bits as #2, with twobobs premade image, but it starts with this command, instead of downloading his image.

First clone your kindle - you must have already put your SSH key onto it.

1. mkdir -p /chroots/kindle
2. rsync -avP --numeric-ids --exclude='/dev' --exclude='/proc' --exclude='/sys' root@192.168.2.57:/ /chroots/kindle

Then working off these instructions to run apps from within it, download the qemu static, and a test file

1. apt-get install qemu binfmt-support qemu-user-static
2. dpkg --add-architecture armel
3. apt-get update
4. apt-get install libc6:armel
5. wget http://http.us.debian.org/debian/poo...10-1_armel.deb
6. dpkg -x hello_version_armel.deb hello_armel

Then install it into your kindl folders, and try to run things

1. cp /usr/bin/qemu-arm-static /chroots/kindle/usr/bin
2. cp ~/hello_armel/usr/bin/hello /chroots/kindle/usr/bin/
3. chroot /chroots/kindle

[LIST]

osboxes@osboxes:~$ sudo chroot /chroots/kindle/
/ # cd usr/bin/
/usr/bin # ./hello
Hello, world!
/usr/bin #

So far, all I have done with this is to run the test script there.

[knc1]

Originally Posted by coplate

- - -

mkdir /chroots/armel-twobob
wget https://dl.dropboxusercontent.com/u/...b.5gb.ext3.zip
unzip deb.5gb.ext3.zip
FILENAME=/mnt/base-us/deb.5gb.ext3
dd if=/dev/zero of=$FILENAME bs=1M count=1500 oflag=append conv=notrunc
e2fsck -f $FILENAME
resize2fs $FILENAME


- - - -

Formatting the file system would help a lot (5.5).
A file full of zeros isn't going to pass fschk, I can tell you that without even trying it.

[coplate]

Originally Posted by Cinisajoy

Does this work on any e-ink kindle?
Does it have to be jailbroke first?

All of this is to develop apps for jailbroken kindles.

I have only ever tried on my Paperwhite 3, I'll let some of the smarter guys come on and tell me if the 'armel debian' should work on most versions of the kindle.

Originally Posted by knc1

PW2 and up is i.MX6
Older devices need an older setup.

I expect it would work on "Sixth Generation" and up, starting on PW2.

Thanks knc1!

[coplate]

Originally Posted by knc1

Formatting the file system would help a lot (5.5).
A file full of zeros isn't going to pass fschk, I can tell you that without even trying it.

That's an existing ext3 filesystem, based on the twobos posts, i'm assuming some kind of debian on it, but I cant find anything.

All the dd does it expand it from 500MB to 2G

[knc1]

PW2 and up is i.MX6
Older devices need an older setup.

[geekmaster]

Originally Posted by knc1

Formatting the file system would help a lot (5.5).
A file full of zeros isn't going to pass fschk, I can tell you that without even trying it.

The following 'resize2fs' also does nothing in this case. It extends a filesystem to fill all the trailing zeroes after appending /dev/zero to an the end of an existing filesystem. It is faster to format a small fs then append zeoes and resize it, than format a large fs, though.

[Yourcat]

Does anyone know whether Qemu for Windows works good enough for compilation? One may avoid running a VM in a VM.
Currently I cross compile on Ubuntu x64 ignoring all existing Kindle libs - for simple executables this works.

[knc1]

Originally Posted by Yourcat

Does anyone know whether Qemu for Windows works good enough for compilation? One may avoid running a VM in a VM.
Currently I cross compile on Ubuntu x64 ignoring all existing Kindle libs - for simple executables this works.

A better way to avoid a VM in a VM is to avoid Windows as a development environment for anything other than Windows.

Doing otherwise is only for those people with the mindset to rearrange the deck chairs on the Titanic.

[knc1]

Originally Posted by coplate

That's an existing ext3 filesystem, based on the twobos posts, i'm assuming some kind of debian on it, but I cant find anything.

All the dd does it expand it from 500MB to 2G

If all you want is bigger, then seek to the last block and write one block of zeros there.
Leave the area from existing end to new end as a sparse file.

Then resize the file system.

ext2/3/4 will create actual blocks in the sparse area as required without any penalties.

Sparse files compress much better that one filled with zeros.
At least if your on a file system that supports sparse files and your compression program can effectively detect them.

But your example (and twobob's) is a nice example of rearranging the deck chairs.
It gives the developer something harmless to do for which they can bill the time.

[coplate]

Originally Posted by geekmaster

The following 'resize2fs' also does nothing in this case. It extends a filesystem to fill all the trailing zeroes after appending /dev/zero to an the end of an existing filesystem. It is faster to format a small fs then append zeoes and resize it, than format a large fs, though.

The resize2fs doesn't just fill the filesystem with zeroes, it also tells the partition map that the partition is now 2 GB, and that's what we want from it in this case.

Otherwise when we boot from it, the OS will see only 512MB

Originally Posted by knc1

... sparse file[s] ...

It looks like qemu and debian can use the sparse files, so that would be a nice shortcut for people working on this, if they start with that image.

[coplate]

Originally Posted by Yourcat

Does anyone know whether Qemu for Windows works good enough for compilation? One may avoid running a VM in a VM.
Currently I cross compile on Ubuntu x64 ignoring all existing Kindle libs - for simple executables this works.

I downloaded qemu for windows here:
https://qemu.weilnetz.de/w64/

And copied all the files from my vmware to windows, and ran this:

"c:\Program Files\qemu\qemu-system-arm.exe" -m 256 -M versatilepb -kernel vmlinuz-3.16.0-4-versatile -initrd initrd.img-3.16.0-4-versatile -hda armdisk.img -append "root=/dev/sda1"

It works, but it honestly is slower here then running it inside of vmware.

Compiling a simple c file takes 1.5s when inside VMware, vs 11 seconds in qemu for windows.