View Single Post
Old 08-11-2015, 05:46 AM   #1
newman
homo viator
newman ought to be getting tired of karma fortunes by now.newman ought to be getting tired of karma fortunes by now.newman ought to be getting tired of karma fortunes by now.newman ought to be getting tired of karma fortunes by now.newman ought to be getting tired of karma fortunes by now.newman ought to be getting tired of karma fortunes by now.newman ought to be getting tired of karma fortunes by now.newman ought to be getting tired of karma fortunes by now.newman ought to be getting tired of karma fortunes by now.newman ought to be getting tired of karma fortunes by now.newman ought to be getting tired of karma fortunes by now.
 
newman's Avatar
 
Posts: 78
Karma: 252594
Join Date: May 2010
Device: Kindle DXi with 3.4.2 firmware, Kobo Glo HD
How to update Kindle 2/DX/DXG with YifanLu's firmware from 3.2.1 to 3.4.2

How to update Kindle 2/DX/DXG with YifanLu's firmware from 3.2.1 to 3.4.2

This guide comes with no warranty, you may brick your device. Here I describe how it worked for me, I don't know if this guide can be directly applied to any other device than the one I used - be cautious.

Why Would I Do That?
The only sound reason I know of is TLS (Transport Layer Security) support in experimental web browser. After 2014 and 2015 revelations of fatal shortcomings of SSL 3.0 (a secure protocol used in HTTPS communication) like POODLE, many sites disabled it's support in web servers resulting in Kindle's experimental web browser being unable to connect to many secured web sites. The protocol to which many web sites migrates is TLS. However, that protocol (set of protocols) was not enabled in Kindle until 3.4.1 firmware. In 3.4.2 this is still enabled.

As a bonus you'll get tons of general fixes all around Kindle, though I don't know of many particular bugfixes, or enhancements, I know of two:
  • fonts are even more contrast
  • Amazon's Kindle Format 8 (KF8/AZW3) works including right-to-left scripts like Arabic, Farsi, ...
  • accented characters (diacritics) in "Notes and Marks" is fixed (see attachment)
  • text style is preserved in "Notes and Marks" (see attachment displaying headline "Hospodářův kalendář..."
  • selection of text which "overflows" from one page to the next won't skip couple of first words on the second page if the selection was too quick

Any Shortcomings?
  • You may brick your Kindle.
  • It takes some time to figure the right approach to upgrade. Every Kindle may be a bit different.
  • With additional code to system tight on RAM you may get more sluggish system.

Prerequisites
  • Strong familiarity with UNIX shell.
  • Kindle 2/DX/DXG (all flavors -- international or US, 3G-only or whatever) with 3.2.1 or higher version of the firmware.
  • Jailbroken Kindle (since you have Kindle with 3.2.1 firmware, your Kindle had to be jailbroken in the past, just make sure the JB is still in place and effective, e.g. by installing USBnet hack).
  • Recent version of kindle_update_tool (I used this one)
  • Patience.

Optional Prerequisites
  • USBnet hack in place and running SSH may be useful should anything went horribly wrong.
  • Remove other hacks not deemed helpful in case of emergency rescue (e.g. ScreenSaver hack, Fonts hack, ...), you may also be tight on space on root partition as well as on operating memory (RAM), every megabyte counts.
  • Backup your data on Kindle, and leave there some 100 MB of free space.
  • Using Kindle's "Reset to factory Defaults" (erases all your documents!) may be of some help too in making sure Kindle is as close to what Amazon expects it to be.

Get The Right Upgrade Files
Once prerequisites are met, start by downloading Kindle 3 firmwares. There are three flavors for three similar versions of Kindle. Basically, I believe - though I am not entirely sure about that - they are all the same, so, feel free to download firmwares from any flavor, I used "B006" on DXi.

Since I expect you to have 3.2.1 firmware, we will follow the official upgrade path: 3.2.1→3.3→3.4→3.4.2 (note: the first step is the same if you start from 3.1 or from 3.2, you just have to use different BIN file and may face different challenges; but if you start from 3.0 or 3.0.1, you have to upgrade to 3.1 first -- for such an upgrade you can alter the upgrade method presented here to your needs but I can't be specific since I started from 3.2.1). Download upgrade files for 3.3, 3.4, and 3.4.2.

3.2.1→3.3 Upgrade
This is by far the easier upgrade. Now I'll mostly paraphrase YifanLu's comment in this regard.

Use kindle_update_tool to extract Update_kindle_3.3_B006.bin:

Code:
$ kindle_update_tool.py e Update_kindle_3.3_B006.bin 
Signature: FC02 (OTA update)
min version: 525120101
max version: 611680021
device code: 06
optional: no
md5 of tgz: f90023068f5556befd51c9522dab4c60
Decompress newly created file Update_kindle_3.3_B006.bin.tgz:

Code:
$ tar xvvf Update_kindle_3.3_B006.bin.tgz 
-rw-r--r-- signdaemon/signdaemon      192 2011-09-09 20:44 update-Update_525120101-611680021.dat
-rw-r--r-- signdaemon/signdaemon     2469 2011-09-09 20:44 111111111-k3.ffs
-rw-rw-rw- signdaemon/signdaemon      128 2011-09-09 20:45 111111111-k3.ffs.sig
-rw-r--r-- signdaemon/signdaemon 24760248 2011-09-09 20:44 Update_525120101-611680021.bin
-rw-rw-rw- signdaemon/signdaemon      128 2011-09-09 20:45 Update_525120101-611680021.bin.sig
-rw-rw-rw- signdaemon/signdaemon      128 2011-09-09 20:45 update-Update_525120101-611680021.dat.sig
Find "Update_525120101-611680021.bin" and use the kindle_update_tool to extract it:

Code:
$ kindle_update_tool.py e Update_525120101-611680021.bin
Signature: FC02 (OTA update)
min version: 525120101
max version: 611680021
device code: 06
optional: no
md5 of tgz: c14a5f16c6dfbf8e606fb039daafdf89
And decompress the TAR:

Code:
$ tar xvvf Update_525120101-611680021.bin.tgz 
-rw-r--r-- signdaemon/signdaemon    1008 2011-09-09 20:41 update-06-525120101-611680021.dat
-rwxr-xr-x signdaemon/signdaemon    6205 2011-09-09 20:40 525120101-611680021.ffs
-rw-r--r-- signdaemon/signdaemon 6031949 2011-09-09 20:41 06/update-06-525120101-611680021.bin
drwxrwxr-x                            Creating directory: 06
-rw-rw-rw- signdaemon/signdaemon     128 2011-09-09 20:41 06/update-06-525120101-611680021.bin.sig
-rw-r--r-- signdaemon/signdaemon 5878741 2011-09-09 20:41 06/update-06-529960105-611680021.bin
-rw-rw-rw- signdaemon/signdaemon     128 2011-09-09 20:41 06/update-06-529960105-611680021.bin.sig
-rw-r--r-- signdaemon/signdaemon 5863321 2011-09-09 20:40 06/update-06-553210206-611680021.bin
-rw-rw-rw- signdaemon/signdaemon     128 2011-09-09 20:41 06/update-06-553210206-611680021.bin.sig
-rw-r--r-- signdaemon/signdaemon 1926999 2011-09-09 20:40 06/update-06-558700031-611680021.bin
-rw-rw-rw- signdaemon/signdaemon     128 2011-09-09 20:41 06/update-06-558700031-611680021.bin.sig
-rw-r--r-- signdaemon/signdaemon 1369833 2011-09-09 20:40 06/update-06-572340009-611680021.bin
-rw-rw-rw- signdaemon/signdaemon     128 2011-09-09 20:41 06/update-06-572340009-611680021.bin.sig
-rw-r--r-- signdaemon/signdaemon 1519255 2011-09-09 20:41 06/update-06-576290015-611680021.bin
-rw-rw-rw- signdaemon/signdaemon     128 2011-09-09 20:41 06/update-06-576290015-611680021.bin.sig
-rw-rw-rw- signdaemon/signdaemon     128 2011-09-09 20:41 525120101-611680021.ffs.sig
-rw-r--r-- signdaemon/signdaemon 2153983 2011-09-09 08:48 kernels/611680021-kernel.tar.gz
drwxrwxr-x                            Creating directory: kernels
-rw-rw-rw- signdaemon/signdaemon     128 2011-09-09 20:41 kernels/611680021-kernel.tar.gz.sig
-rw-r--r-- signdaemon/signdaemon    3880 2011-09-09 20:40 support.tar.gz
-rw-rw-rw- signdaemon/signdaemon     128 2011-09-09 20:41 support.tar.gz.sig
-rw-rw-rw- signdaemon/signdaemon     128 2011-09-09 20:41 update-06-525120101-611680021.dat.sig


Now you’ll have a bunch of files. Go into the "06" folder and find "update-06-576290015-611680021.bin" (if you don't upgrade from 3.2.1, find the right BIN file numbers here). Open that in a hex editor of your choice and go to offset 0xC. Change the byte to 0x02 for K2, 0x03 for K2i, 0x04 for DX, 0x05 for DXi, and 0x09 for DXG. Save it.

Code:
$ hexedit 06/update-06-576290015-611680021.bin
Copy that file to Kindle's USB root and run it from Kindle's interface. YifanLu adds: "You might get an error and it's OK because some of the modem files may be different, but your Kindle will be updated regardless." I did not get any error, so I can't verify that.

After restart your Kindle should be upgraded to "3.3 (611680021)", verify that in Settings.

3.3→3.4 Upgrade
In theory you could hexedit Update_kindle_3.4_B006.bin on byte 0xC to contain your specific Kindle code (see paragraph above), and it may work -- feel free to try it -- and you could fast-forward to 3.4→3.4.2 upgrade. However, in my case (1) file /etc/fb.modes was different from what the update file expected and when Kindle tried to apply a patch on it, it failed; (2) my DXi always hanged on patching libwebkit and libwebkitgtk -- two libraries of size about 14-22 MB which, to be patched by bspatch tool, require at least 20 MB of free RAM, which my Kindle did not have.
  • Extract "Update_kindle_3.4_B006.bin" with kindle_update_tool.
  • Decompress resulted "Update_kindle_3.4_B006.bin.tgz".
  • Remove all *.sig and *.dat files, we will create our own signatures afterwards.
  • Rename all .ffs files to .sh:
    Code:
    $ rename ffs sh 611680021-1725970040.ffs 999999999-reboot.ffs
  • Open "611680021-1725970040.sh" in text editor of your choice and add following code snippet on new line after #!/bin/sh. They will provide logging useful for debugging:

    Code:
    set -x
    exec > /mnt/us/my-upgrade.out
    exec 2> /mnt/us/my-upgrade.err

Verify you did not mess anything in the script:

Code:
$ bash -n 611680021-1725970040.sh
Create you own signed upgrade BIN, replace "--dxi" with appropriate flag which suits your Kindle type:

Code:
$ kindle_update_tool.py m --dxi --sign kindle_3.4_B006-my-log 611680021-1725970040.sh 999999999-reboot.sh rootfs_md5_list.tar.gz update-adds.tar.gz update-kernel.tar.gz update-patches.tar.gz
calculating signature for 611680021-1725970040.sh
cmd = openssl dgst -sha256 -sign /tmp/tmplDABFe -out /tmp/tmp9hCOlU 611680021-1725970040.sh
calculating signature for 999999999-reboot.sh
cmd = openssl dgst -sha256 -sign /tmp/tmplDABFe -out /tmp/tmpfJzg2A 999999999-reboot.sh
calculating signature for rootfs_md5_list.tar.gz
cmd = openssl dgst -sha256 -sign /tmp/tmplDABFe -out /tmp/tmpZ_tyBa rootfs_md5_list.tar.gz
calculating signature for update-adds.tar.gz
cmd = openssl dgst -sha256 -sign /tmp/tmplDABFe -out /tmp/tmpGXsunh update-adds.tar.gz
calculating signature for update-kernel.tar.gz
cmd = openssl dgst -sha256 -sign /tmp/tmplDABFe -out /tmp/tmpj7ERcV update-kernel.tar.gz
calculating signature for update-patches.tar.gz
cmd = openssl dgst -sha256 -sign /tmp/tmplDABFe -out /tmp/tmpWtd2nD update-patches.tar.gz
adding 611680021-1725970040.sh
adding 999999999-reboot.sh
adding rootfs_md5_list.tar.gz
adding update-adds.tar.gz
adding update-kernel.tar.gz
adding update-patches.tar.gz
calculating signature for bundle file
cmd = openssl dgst -sha256 -sign /tmp/tmplDABFe -out /tmp/tmp_VkSLM /tmp/tmp4HgOUe
making bin file
output written to update_kindle_3.4_B006-my-log.bin


Upload "update_kindle_3.4_B006-my-log.bin" to Kindle and run the upgrade. It will fail, probably. You'll see a couple of restarts (or resets to factory defaults?!) but at least in my case Kindle always survived. Now, in Kindle USB root you'll find two logs my-upgrade.out and my-upgrade.err. Inspect them to see what was the problem which prevented the upgrade. Now, you're bit on your own...

File /etc/fb.modes is different on DX - not sure about Kindle 2 - to what was was expected because it contains display resolutions which are present on DX but not on Kindle 3 (DX has 824×1200, K 2 and 3 just 600x800) and a patched result did not match the expected MD5 hash (see appropriate parts of 611680021-1725970040.sh for the verification process):

Code:
...
local4.info   ota_install I def:patch:patchinfo="/etc/fb.modes":patching file
ota_install: I def:patch:patchinfo="/etc/fb.modes":patching file
local4.crit   ota_install C def:patch:patchinfo="/etc/fb.modes":checksum failure
ota_install: C def:patch:patchinfo="/etc/fb.modes":checksum failure
local4.info   ota_install I def:patch:current_md5=f0845d7fe22da159acbf1fef65302abd,expected_md5=645ec280a0ac85ac6c4672d614d7388d,patch_md5=e8f0742b63cfa44f2cc77582557ae9d6:checksum verify


To see other files which potentially might fail in MD5 hash check use Busybox attached to this article:

Code:
# busybox md5sum -c rootfs_md5_list | grep -v OK
/opt/amazon/ebook/lib/KindleDeviceServicesShasta-1.0.jar: FAILED
/opt/amazon/ebook/lib/json_simple-1.1.jar: FAILED
md5sum: can't open '/opt/config/not_shipped': No such file or directory
/opt/config/not_shipped: FAILED
md5sum: can't read '/etc/kdb.src/luigi/': Is a directory
/etc/kdb.src/luigi/: FAILED
/etc/shadow: FAILED
md5sum: can't open '/etc/udev/rules.d/XX-luigi.rules': No such file or directory
/etc/udev/rules.d/XX-luigi.rules: FAILED
/etc/init.d/video: FAILED
md5sum: can't open '/usr/sbin/usbnetwork': No such file or directory
/usr/sbin/usbnetwork: FAILED
md5sum: WARNING: 8 of 2403 computed checksums did NOT match
Look for lines with "FAILED" because if those files are being updated, then you'll have to fix them manually (see below with what happens with /etc/fb.modes).

I had to copy out system /etc/fb.modes file, patch it with appropriate file from "update-adds.tar.gz", edit the result a bit to fit my Kindle's display resolution, create a proper patch (see man bsdiff), replace the old patch from "update-adds.tar.gz" by the new one, and finally change the expected MD5 in "611680021-1725970040.sh". Also MD5 hash of /etc/fb.modes should be changed in "rootfs_md5_list" file from "rootfs_md5_list.tar.gz". There are other ways to fix this problem (e.g. local patching and transfer of the file, see how libwebkit{,gtk} was handled, but I did not know it was possible back then).

Once fixed, upgrade process hanged later on patching libwebkit. The line where the upgrade process stopped before being killed by Kindle was, according to the logs:

Code:
bspatch /usr/lib/libwebkit-1.0.so.2.5.0 patch/usr/lib/libwebkit-1.0.so.2.5.0 147.libwebkit-1.0.so.2.5.0.patch
There are two ways to overcome it: (1) local patching (tested and works), or (2) add a swap file (untested).

(1) Local patching
I copied out /usr/lib/libwebkit-1.0.so.2.5.0 from Kindle (use USBnet or copy it by a suitable command at suitable place in the script) to my computer, patched it by bspatch, verified that MD5 hash of the patched file is the one expected in "611680021-1725970040.sh", put it to Kindle's USB root and added this line instead of the broken bspatch line:

Code:
cp /mnt/us/libwebkit-1.0.so.2.5.0.NEW patch/usr/lib/libwebkit-1.0.so.2.5.0
(2) Add a swap file
I did not tried this approach in upgrade scenario. [UPDATE: Although it looks like a more sane way to fix the problem of not enough free memory for bspatch-ing on Kindle, I added swap file to my general Kindle environment as a permanent way how to enlarge virtual memory but I got frequent filesystem corruptions!] Let's extend virtual memory by 64 MB! Download and unpack NiLuJe's stripped Busybox (also in files attached to this article), which does only operations related to swapping, to /mnt/us/ (that is the topmost directory on Kindle when attached as USB storage). Add following three lines right after those two exec lines in 611680021-1725970040.sh:

Code:
dd if=/dev/zero of=/mnt/base-us/swapfile bs=1M count=64
chmod +x /mnt/base-us/busybox
/mnt/us/busybox mkswap /mnt/base-us/swapfile
/mnt/us/busybox swapon /mnt/base-us/swapfile
For more info see https://www.mobileread.com/forums/sho...1&postcount=63.

Repack with kindle_update_tool, upgrade, see if it got fixed. If successful, this upgrade should take some 20 minutes.

3.4→3.4.2 Upgrade
In 3.4.1 (sic) upgrade Amazon added TLS and disabled SSL 3.0 support, so another big library got updated: libwebkitgtk (about 22 MB). Do the same you did when upgrading to 3.4 (extract TGZ from downloaded BIN, unpack TGZ, rename FFS to SH). Add the logging shell code snippet to "1725970040-2687240004.sh". Also MD5 hash of /etc/fb.modes should be changed in "rootfs_md5_list" file from "rootfs_md5_list.tar.gz", if you did that change in 3.3→3.4 upgrade before. Repack, upgrade and see what went wrong. Again, upgrade process will probably fail on bspatch-ing of libwebkit and then on libwebkitgtk, do the same fixing you have done in upgrade to 3.4 (i.e. local patching, or add a swap file).

If successful, this upgrade should take about 5 minutes and you'll end up with 3.4.2 on your Kindle, verify you are able to see TLS support in "Version" paragraph at howsmyssl.com and some site which runs TLS-only, e.g. instapaper.com. Enjoy!
Attached Thumbnails
Click image for larger version

Name:	screen_shot-46796.gif
Views:	1713
Size:	30.3 KB
ID:	140985   Click image for larger version

Name:	screen_shot-46797.gif
Views:	1562
Size:	56.0 KB
ID:	140986   Click image for larger version

Name:	screen_shot-46798.gif
Views:	1506
Size:	68.7 KB
ID:	140987   Click image for larger version

Name:	screen_shot-27337.gif
Views:	1444
Size:	49.3 KB
ID:	141278  
Attached Files
File Type: gz bb-swap-k3.tar.gz (14.5 KB, 1141 views)
File Type: bz2 busybox.bz2 (339.7 KB, 912 views)

Last edited by newman; 10-27-2015 at 03:20 AM.
newman is offline   Reply With Quote