Geekmaster's adventures in K1 spelunking

[geekmaster]

My next thing to do is make my kindle video player do correct 2-bit pixel packing for the K1 framebuffer, then after that, I want to build tinycc (tcc) so I can update my tcc download package to support the K1 (and I will also test it on my K2 and PW2 and PW3 devices now that I have them). For testing app compatibility, hardware version are not the only concern -- firmware update 5.1 for the K5 broke eink compatibility that was working on 5.0, because lab126 F-ed up and (contrary to standard procedure) redefined a critical eink structure without renaming it, and without renaming the header file that contained it. My code therefore cannot use that header file, and instead copied (and renamed) those structures. One more reason why amazon (and google android) somewhat discourages native code development and pushes developers to build java apps, I guess...

[geekmaster]

I added 2 bit-per-pixel (4 pixels-per-byte) rendering to my kindle video player, to support the K1. It is mostly working, but the packed bit order inside the byte is incorrect. Unlike the K4 with most significant bits of a packed byte on the left, on the K1 they must be in reverse order. I also compensated for the negative video.

I also noticed that I only did a screen-centering offset adjustment for 4-bit video (to support large-screen DX and DXG, when this was developed). When the PW1 came out, gmplay worked fine, but it used 8-bit video and gmplay did not bother centering 8-bit video. I added 8-bit centering, so the video should not be pushed into a corner on higher resolution kindles like my new PW3 (on which the ORIGINAL gmplay worked just fine, but just not centered). BTW, I am considering pixel doubling for the PW3, because the 800x600 video only occupies about a quarter of the screen.

My launcher script now runs xzcat and gmplay in the background (with a '&' postfix), and then waits for the K1 'R' key to be pressed (because I found the 'R' keycode in an install script), and then does "killall -9 gmplay", which allows early termination instead of needing to wait for the video to end (which was getting boring during development and testing).

As it turns out, even -4 compression level gets "corruption" errors, and I am running at only -4 compression. However, "xz -z1" is WAY more compressed than "gz -9" compression like I was using in the past (and which also fails on the K1).

Bed time for Bonzo. I will sort out the internal pixel ordering inside K1 framebuffer bytes tomorrow. G'night y'all...

[geekmaster]

I tried a number of different bitmasks to get the pixels inside the frambuffer bytes sorted out, so I can draw individual pixels instead of individual bytes as I do in my eink algorithmic art scripts (giving fat pixels on K3 and earlier).

My gmplay video player handles pixel packing correctly on DX/DXG/K3 (which all use Epson Broadsheet eink controllers with two pixels per byte, bitmasks 0xF0 and 0x0F) and correctly for FreeScale SoC embedded eink controllers with one pixel per byte, bitmask 0xFF).

The K1 uses an Apollo eink controller with four pixels per byte. If I use bitmasks 0xC0, 0x30, 0x0C, and 0x03, black areas display as stripes. Likewise, bitmasts 0x03, 0x0C, 0x30, and 0xC0 are also striped. Both sets of bitmasks updated the eink display without flicker at multiple frames per second. I also tried bitmasks 0x05, 0x0A, 0x50, and 0xA0 to see it there was any bit interleaving, but the display updates became full-flash updates, with a much slower framerate (i.e. grayscale pixels). This is mysterious because I got solid black areas when running my "tangle" eink demo script that uses 0xFF and 0x00 for the pixel values (i.e. quad-width pixels). Solid black dithered areas should use the same value. The only difference I can see is that the scripts use the /proc to update the display, which the gmplay C program is using ioctl calls. Perhaps an eink structure misalignment is causing this weirdness (just like the lab126 FUBAR between FW 5.0 and 5.1). Perhaps I just need more coffee to sort this out?

I loaded up the Apollo driver gpl source code (part of the firmware source code from amazon) into my "programmer's editor", and I will grab another coffee and see what I can learn from that code... Sometimes ya just gotta UTSL.

EDIT: I just changed the bitmasks to 0xFF, to force the entire byte to 0x00 or 0xFF depending on the value of every fourth pixel. Now I have solid areas, but only eight shades of dithered gray because of the fat pixels. This shows it is not the eink structures at fault, nor is my code writing to the wrong bytes. There seems to be something funky about pixel pairs. More testing to ensue...

EDIT2: I just tried bitmasks 0xF0 and 0x0F (same as for 4-bit pixels). Solid black areas are striped again, though that could be because of the dithering checkboard pattern being stretched, in this case. However, that does not explain the striping when I use 2-bit pixel bitmasks. I still have Apollo eink secrets to learn, obviously...

EDIT3: I code up a test pattern that overwrites the left 32 bytes (128 pixels). That pattern shows that the pixels are packed exactly as I had first presumed would make the most sense). However, that is what I tried to do in my dithered-bit packing (which is obviously not working). I have examined the code carefully multiple times. This is where another pair of eyes comes in handy. I dislike publishing broken code, but MY eyes are sure not (yet) seeing where I went wrong... I just brewed some more coffee, so I will grab a cup and RE-re-examine my code.

[geekmaster]

Coffee is amazing, isn't it. Not good to let your blood caffeine levels drop too low (well, at least not while debugging code, that is).

Anyway, I found the problem in gmplay bit packing. When I duplicated a block of code from 4-bit pixel packing, and modified the copies suitably, I did not logically OR them together, so only the second code block was having any effect on the display. Adding in that missing logic OR made it all work properly.

When the video playback competes, gmplay reports (to standard output, which is redirected to /mnt/us/documents/RUNME.txt, where it can be viewed on the Kindle):
"1153 frames in 620.6 secs = 1.9 FPS"

That is about 27% faster than when running on a kindle DX, but I think we can do better than this (by reducing the eink update area, though we can provide a nice static dithered grayscale border surrounding the smaller video window, and even though full grayscale, it only does a full-flash update once when it is first drawn).

The next step is to port my 2-bit eink changes back into my gmplay-2.0 (with sound). But I have yet to test sound support on the K1...

[geekmaster]

Every time I use the thumbwheel roller on the K1, I think about how nice of an input device it would be to control a paddle in a Breakout-style game (with the paddle moving up and down the right side of screen). Breakout came with the Apple-II, and it was kinda cool (and still is, for that matter).

Yup, for sure. Just gotta do that! It sounds like an hour or two of coding, depending on amount of "unknown unknowns" (or late night coding goofs) decide to jump up in my face. Ya never know, eh?

Or we could do a "Bubble Shooter" game? Things with paddles, thumbwheel, yeah!

[geekmaster]

Oh, and I just discovered that on a K1, if you slide the power switch OFF while it is waiting for a keypress in the "waitforkey" routine, well, time to find a paperclip. RESET time...

[NiLuJe]

FWIW, a breakout-like game was part of the initial Kindlet that came out in the US Kindle Store .

I don't remember if they ever ported it to KDK2 (i.e., touch-based input) though...

I can only approve of such an endeavour (I have very fond memories of the Arkanoid series on my Amiga....) .

[geekmaster]

I really want my kindle binaries to run on all the eink kindles. That means sticking with standard interfaces, such as standard linux system calls, /proc files, framebuffer access, and so forth.

And apparently a "minimum common denominator" ARM instruction set too. I feel damned lucky just to have found a toolchain that works on the K1 (especially considering that all the other aboriginal toolchains that SHOULD be K1-compatible have been converted to musl and DO NOT WORK on the K1. Thankfully the armv4l was not yet converted, and is still ucllibc based, and still works on a K1.

The problem is that code statically-compiled with this armv4l toolchain gets "Illegal instruction" errors when run on NEWER kindles (recently tested on a PW3). Same problem going the other direction -- code compiled for newer kindles blows chuncks on the K1. What's up with that?

Must I resort to pure embedded assembly language (no C runtime, no libraries, no "Illegal instructions" because I write each and every instruction myself)? I hope not -- though in reality I REALLY LOVED coding in assembler back when that was the main programming language, before companies started migrating to COBOL (or FORTRAN in some shops). At least on a machine like an IBM 360 with its orthogonal instruction set (all instructions can use all registers). And NO STACKS -- everything was passed (and kept as long as possible) in REGISTERS (in the pre-cache era). Nowadays, cache is much like a very large register set that is automagically managed for you.

And regarding cache, I am fascinated a lot with this particular lecture:


And while I am recommending stuff that fascinates me, this link (once again for the newbies):
http://aggregate.org/MAGIC/

But yeah, what magical incantations must I conjure (or mutter) to build binaries that run on all the eink kindles?

[geekmaster]

I just got a notice that another K1 just shipped (which I forgot about). This one comes in its original box, with manual. That will be 3 of them in my hands.

[geekmaster]

PHP Code:

# file Update_kindle_1.2/update-016-KU3-029987/root_fs.img 
Update_kindle_1.2/update-016-KU3-029987/root_fs.img: Squashfs filesystem, little endian, version 3.0, 11586773 bytes, 348 inodes, blocksize: 65536 bytes, created: Wed Jan 28 21:12:12 2009
mi 


I see that the rootfs amazon packed in their FW 1.2 update is type Squashfs, which means that it is read-only. No "mntroot rw" on the K1, apparently. Besides ramdisks, only /opt and /mnt/us appear to be writable (as mentioned previously, no /var/local on the K1).

The dmesg errors talk about jffs2? I tried mounting it as jffs2 (instructions posted previously to this thread) and that also failed.

Trying to mount that K1 rootfs on my recent linux mint gives this:

[307062.882781] jffs2: Old JFFS2 bitmask found at 0x00a45e24
[307062.882782] jffs2: You cannot use older JFFS2 filesystems with newer kernels
[307062.883654] jffs2: Cowardly refusing to erase blocks on filesystem with no valid JFFS2 nodes
[307062.883658] jffs2: empty_blocks 0, bad_blocks 0, c->nr_blocks 83
[308265.533226] squashfs: SQUASHFS error: Major/Minor mismatch, older Squashfs 3.0 filesystems are unsupported

I get a similar error trying to mount it inside my armv4l build environment too:

(armv4l:1) /home $ mount -t squashfs K1-1.2-rootfs.img rootfs
squashfs: SQUASHFS error: Major/Minor mismatch, older Squashfs 3.0 filesystems are unsupported
mount: '/dev/loop0'->'rootfs': Invalid argument

[NiLuJe]

@geekmaster: SquashFS appears to drop compatibility at each major release.

So mounting it is going to take a tremendously old kernel.

On the other hand, you should be able to unpack it w/ unsquashfs from squashfs-tools 3.2.
(I'm basing that on the version slots available on Gentoo).

* sys-fs/squashfs-tools
     Available versions:  
     (3.0)  3.2_p2
     (3.1)  3.4
     (0)    4.3 (~)4.3-r1
       {lz4 lzma lzo xattr +xz}
     Homepage:            http://squashfs.sourceforge.net
     Description:         Tool for creating compressed filesystem type squashfs

[knc1]

Version 3.0 is when squashfs was accepted into the mainline kernel.

You should still be able to get the earlier versions from the squashfs web site.

[geekmaster]

Srrangely, 7-zip on win7 has no problem extracting the files, but then they are sitting on NTFS instead of a linux FS. I want to migrate the K1 files into my K1 (aboriginal) build environment, with hopes of building sound apps.

I wonder how well a linux version of 7-zip would work...

Meanwhile, squashfs website...

[donB006]

I tried mount -t squashfs also and that failed for some reason. I finally just used this instead of mount:

  unsquashfs [OPTIONS] FILESYSTEM [directories or files to extract]

[geekmaster]

I built a static tinycc (tcc) compiler for the K1. As mentioned previously, I have not been able to build any dynamic apps that run on the K1. In my armv4l build environment, tcc can compile dynamic apps (which run in the build environment, but not on the K1). And tcc can also run on the K1, and even build dynamic apps. However, tcc CANNOT build any static apps (either on the K1 or in the build environment) -- it complains about various missing floating point functions. Did I configure it wrong?

Also, I am having problems compiling alsa sound apps -- tons of errors in the header files (with both gcc and tcc)... I recall having similar issues last time, years ago. I was hoping to test sound playback on the K1, but no success so far... Perhaps less work compiling a sound app for the K5 first (trying to re-create what I did back then).