kobopatch users & non-Kobo-supported languages

[jackie_w]

Originally Posted by NiLuJe

There's (currently) ~70MB of free space in the rootfs on a stock device, you have quite a bit of leeway .

Thanks. I was trying to decide whether I needed to add some kind of warning to the updated notes (when I get round to it). As there's not much danger it would probably confuse rather than illuminate.

ETA: Posts #2 and #63 updated with the new info about overwriting system fonts.

[NiLuJe]

Nothing should actually *write* to the rootfs at runtime, so the most plausible source of potential issue would be running out of free space *during* an update.

[DNSB]

Originally Posted by NiLuJe

Nothing should actually *write* to the rootfs at runtime, so the most plausible source of potential issue would be running out of free space *during* an update.

AFAIR, the update is unpacked on the FAT32 partition and then copied to the root partition. Keep less than 100MB free on the FAT32 partition and updates will happily fail with an error message.

[NiLuJe]

@DNSB: That's... not actually clear to me.

That there'd be a free space check during the update's *download* OTA, sure, why not.
But I'm concerned with storage issues in the rootfs, not the USB partition .

And there, the actual installation is basically an unchecked tar call. There's an integrity check *before* the unpack, but that would have no bearing on a storage space issue arising *during* the unpack.

The only thing that would flag such a failure is the lack of an updated revinfo entry in the install log (/usr/local/Kobo/install.log).

It'll happily keep on trucking and try to install the kernel, bootloader & co, even if the unpack screwed up first. (Which, granted, would probably not be *that* problematic, because those don't live in the rootfs).

----

That said, we currently have quite a bit of leeway in terms of free space there, so I wouldn't worry too much about it, especially given the fact that the only people actually putting stuff in the rootfs ought to know what they're doing in the first place .

[Semwize]

@NiLuJe

Delete these fonts

  8472 -rw-r--r--    1 root     root        8.3M May 14 14:15 KBJ-TsukuMinPr6N-RB.ttf
  3811 -rw-r--r--    1 root     root        3.7M May 14 14:15 KBJ-UDKakugoPr6N-M.ttf
  4523 -rw-r--r--    1 root     root        4.4M May 14 14:15 ub_arudjingxihei.ttf

those who do not use them? Free space.

[davidfor]

Originally Posted by Semwize

@NiLuJe

Delete these fonts

  8472 -rw-r--r--    1 root     root        8.3M May 14 14:15 KBJ-TsukuMinPr6N-RB.ttf
  3811 -rw-r--r--    1 root     root        3.7M May 14 14:15 KBJ-UDKakugoPr6N-M.ttf
  4523 -rw-r--r--    1 root     root        4.4M May 14 14:15 ub_arudjingxihei.ttf

those who do not use them? Free space.

I'm fairly sure the kepub renderer uses the first two for fallback to fill in missing characters.

[DNSB]

Originally Posted by NiLuJe

@DNSB: That's... not actually clear to me.

That there'd be a free space check during the update's *download* OTA, sure, why not.
But I'm concerned with storage issues in the rootfs, not the USB partition .

It'll happily keep on trucking and try to install the kernel, bootloader & co, even if the unpack screwed up first. (Which, granted, would probably not be *that* problematic, because those don't live in the rootfs).

If the unpack fails due to a lack of space in the FAT32 partition, it will not try to copy files to the root partition. Instead you'll get an error message. Now if the root partition runs out of space during the file copy, that I've never tested. An interesting experiment to try some day. Just make sure I'm ready to do a factory reset.

[NiLuJe]

@DNSB: There's *never* an unpack to the FAT32 partition? There's no "copy" to the rootfs either.

There's a "copy" (i.e., a download by Nickel, OTA; or a manual copy by the user, via USBMS) of the KoboRoot tarball + upgrade folder to the FAT32 partition, then there's an unpack from there to the rootfs during the update process.

(Semantics, true, but that needed clarification not to confuse my point, which was solely about the actual unpack process to the rootfs, during which the state of the FAT32 partition is irrelevant, as nothing gets written to it ).

EDIT: Okay, I was indeed glossing over a step in case of an OTA, assuming Nickel downloads the kobo-update ZIP first, actually stores it on disk, and then unpacks it, there's indeed an extra unpack to the FAT32 partition there, my bad .
I'm not familiar with the actual implementation, but it could just as well be "streaming" the unpack straight from memory, without needing the ZIP data to ever make it to disk.
Worth mentioning, but not the step I was concerned with .

[DNSB]

Originally Posted by NiLuJe

@DNSB: There's *never* an unpack to the FAT32 partition? There's no "copy" to the rootfs either.

There's a "copy" (i.e., a download by Nickel, OTA; or a manual copy by the user, via USBMS) of the KoboRoot tarball + upgrade folder to the FAT32 partition, then there's an unpack from there to the rootfs during the update process.

(Semantics, true, but that needed clarification not to confuse my point, which was solely about the actual unpack process to the rootfs, during which the state of the FAT32 partition is irrelevant, as nothing gets written to it ).

EDIT: Okay, I was indeed glossing over a step in case of an OTA, assuming Nickel downloads the kobo-update ZIP first, actually stores it on disk, and then unpacks it, there's indeed an extra unpack to the FAT32 partition there, my bad .
I'm not familiar with the actual implementation, but it could just as well be "streaming" the unpack straight from memory, without needing the ZIP data to ever make it to disk.
Worth mentioning, but not the step I was concerned with .

The pattern I've seen suggests the update.zip files is downloaded and unpacked (the two files and the upgrade directory) on disk -- if you tap update later, the .zip file is missing but the KoboRoot.tgz, manifest.md5sum and the upgrade directory are present in the .kobo directory. If the file does not unzip correctly, the update does not attempt to continue and an error message is written to the log file. Once that is done, limited testing suggests the files internal to the KoboRoot.tgz are extracted in memory and copied to the root partition. This suggests that as long as the root partition has enough free space for the largest file, the file copy from KoboRoot.tgz should not cause any issues.

nickel: (   334.153 @ 0x179fa08 / ui.warning) void Unzipper::startWork() "/mnt/onboard/.kobo/updatefilename.zip" failed verify

[NiLuJe]

@DNSB: Well, that's not terribly efficient, but, oh, well. .

In any case, that's definitely not the step I'm concerned with (which is partly why it completely skipped my mind ^^), but thanks for the details .

AFAIK, nickel itself never deals with the KoboRoot (by which I mean unpacking it), that's left entirely to the init script, so I'm mightily confused by you two final sentences.

The naively sane approach to the zip unpack failing would be to clean up and forget about it (or retry a fresh and/or delta download), not continue to attempt an update from half-broken assets?

[DNSB]

Originally Posted by NiLuJe

@DNSB: Well, that's not terribly efficient, but, oh, well. .

In any case, that's definitely not the step I'm concerned with (which is partly why it completely skipped my mind ^^), but thanks for the details .

AFAIK, nickel itself never deals with the KoboRoot (by which I mean unpacking it), that's left entirely to the init script, so I'm mightily confused by you two final sentences.

The naively sane approach to the zip unpack failing would be to clean up and forget about it (or retry a fresh and/or delta download), not continue to attempt an update from half-broken assets?

I'm not sure what is confusing you with my last two sentences. There is unzipping the firmware .zip file and there are the further actions of unpacking (untgzing? untarring?) the contents of the KoboRoot.tgz file into the root file system. If the zip unpack fails to verify, as mentioned, the upgrade does not attempt to continue. If the zip unpack succeeds, once the presence of the RootKobo.tgz file is noticed, the Kobo reboots and the rcS script continues the upgrade.

Looking at the rcS, upgrade-wifi.sh and upgrade-generic.sh, it would appear that if /mnt/onboard/.kobo/KoboRoot.tgz exists, the first step is to set the boot to the recovery partition.

dd if=$UBOOT_RECOVERY of=/dev/mmcblk0 bs=128k count=1 seek=6

At this point, KoboRoot.tgz is verified and if good, unpacked into /. After the unpack, if the upgrade directory is present in /mnt/onboard/.kobo, upgrade-wifi.sh is executed. If all goes well, the boot is set back to the standard boot partition and reboot happens.

A few oddities. If a Kobo.tgz file is found, it seems to be treated the same as KoboRoot.tgz but is unpacked into /usr/local/Kobo. No check for the upgrade directory is made and the two upgrade-*.sh scripts are not run and no reboot is done.

I will admit I haven't played much with Kobo's startup scripts since I stopped trying to use advboot for playing with KOReader 5 or 6 years ago.

[NiLuJe]

@DNSB: Yep, that's it. I got the impression you were saying that *nickel* was handling part of that after the download, which is what was confusing me .
(That and the use of "file copy" to describe what tar's doing when unpacking. But I get what you were trying to say now ^^ (i.e., unlink() -> sendfile() [well, a pair of read() + write() because the (un)compressor filter writes to a pipe]).

The Kobo vs. KoboRoot thing might be related to how a factory reset reinstall is handled (or not, I'm possibly getting my wires crossed with Kindles, there ).

[DNSB]

Originally Posted by NiLuJe

@DNSB: Yep, that's it. I got the impression you were saying that *nickel* was handling part of that after the download, which is what was confusing me .
(That and the use of "file copy" to describe what tar's doing when unpacking. But I get what you were trying to say now ^^).

The Kobo vs. KoboRoot thing might be related to how a factory reset reinstall is handled (or not, I'm possibly getting my wires crossed with Kindles, there ).

My apologies for not making the steps clear enough.

I don't think Kobo vs. KoboRoot is related to a factory reset. In theory, nothing should write to the recovery partition so Kobo would not make it that easy. When I was playing with updating the recovery image for a Glo, I had the uSD card mounted on my laptop and still have rather copious notes on how many ways I found to not create a valid recovery image.