Where can i report bugs ?

[Frenzie]

No explicit reason was given in https://github.com/koreader/koreader-base/pull/212 but our current tar command is a far cry from anything supported by busybox in any case.

busybox tar should suffice for simpler stuff like dictionary/OCR download, but we need a way to handle tar on Android regardless.

[BloodRagg]

Originally Posted by NiLuJe

busybox's tar doesn't support checkpoints, and/or isn't actually there on some target platforms, and/or doesn't support something else that I can't remember OTOH.

If its only the checkpoints this could suffice with some modification.

untar_progress(){
  local total=$(tar -ztf "$1" |wc -l)
  local step=$((total/100));[ $step -gt 1 ] || step=1
  tar -zxvf $@ | awk -v total=$total -v step=$step \
    'BEGIN {ORS="\n"} {if(NR%step==0) system("fbink -qe -y5 -P" int(NR*100/total))}'
}

untar_progress file.tar

I always forget its multiplatform

[NiLuJe]

IIRC, there are also some Kindle-specific concerns with the busybox tar version they ship.

Plus, I'd rather not touch the checkpoints code, it was already annoying enough to get right, and has ties to the OTA manager, which we really want to make sure behaves consistently across all supported platforms.

Also, compared to DIY approaches, it has the advantage of being actually linear, as it's based on data length, and not file amounts .

[BloodRagg]

Originally Posted by NiLuJe

IIRC, there are also some Kindle-specific concerns with the busybox tar version they ship.

Plus, I'd rather not touch the checkpoints code, it was already annoying enough to get right, and has ties to the OTA manager, which we really want to make sure behaves consistently across all supported platforms.

Also, compared to DIY approaches, it has the advantage of being actually linear, as it's based on data length, and not file amounts .

I'll bet, aren't you mixing compressed blocks with uncompressed ones here ?
I couldn't get that code working on something else I was working on.

To code or not to code , that is the question

[Frenzie]

Originally Posted by NiLuJe

Also, compared to DIY approaches, it has the advantage of being actually linear, as it's based on data length, and not file amounts .

Which is much appreciated.

[BloodRagg]

New version, yeee
So I was testing the update function today, but somehow it breaks down, any idea on how I can debug this ?

[Frenzie]

"Breaks down" how?

[BloodRagg]

You know my setup is custom, so I thought I'd do it myself before bothering anyone with it.

1. Select update from hamburger menu.
2. Popup do you want to update (installed 201904, avail 201905), cancel,>>update<<.
3. Downloading file, computing zsync delta.
4. Popup: Failed to update KOReader
Cancel, keeping temporary files.
>>Retry<< the update process with a full download.
Abort and cleanup all temporary files.
5. Popup: Error updating KORreader. Would you like to delete temporary files?
Cancel,
>>Ok<<

Is there something I can turn on, or do I have to monitor the scripts manually ?

root@kobo$ ls -la (during download)

-rwxr-xr-x    1 root     root           158 May  9 21:58 obtain-ip.sh
drwxr-xr-x    2 root     root          4096 May  9 22:07 ota
drwxr-xr-x   26 root     root          1024 May  7 09:41 plugins
>> -rw-------    1 root     root      63475712 May  9 22:07 rcksum-EuvN0D <<
-rwxr-xr-x    1 root     root          8743 May  9 21:58 reader.lua
-rwxr-xr-x    1 root     root           231 May  9 21:58 release-ip.sh
drwxr-xr-x    3 root     root          1024 May  7 09:41 resources


root@kobo$ ls -la (after download)
total 66369
drwxr-xr-x    2 root     root          4096 May  9 22:07 .
drwxr-xr-x   23 root     root          1024 May  9 22:07 ..
-rw-r--r--    1 root     root        410387 May  9 22:06 koreader-kobo-latest-stable.zsync
-rw-r--r--    1 root     root      67502080 May  9 22:07 koreader.installed.tar
-rw-r--r--    1 root     root         40545 May  9 21:58 package.index


[crash.log]
rename: Bad file descriptor

[crash.log] (after retry)
rename: Bad file descriptor
rm: can't remove './ota/koreader.updated.tar*': No such file or directory              
rename: No such file or directory

[Frenzie]

You can run the program with the -d flag or turn on debug logging in the developer options, but I doubt that'd do much of anything for your "Bad file descriptor" issue. My guess would be you're doing something that breaks tar.

[NiLuJe]

Possibly something similar to how tar will SIGBUS when it tries to overwrite itself during the unpacking on Kindles because of the fuse layer?

Check the Kindle koreader.sh script, it has some extra ugly band-aid to make it behave (basically, copy tar somewhere safe like /tmp, and use that copy to do the unpacking).

Note that this only papers over the issue, if the startup script itself has been updated, it might also cause the restart to fail, and cause the usual post exit cleanup to be skipped.

[BloodRagg]

Originally Posted by Frenzie

You can run the program with the -d flag or turn on debug logging in the developer options, but I doubt that'd do much of anything for your "Bad file descriptor" issue. My guess would be you're doing something that breaks tar.

Originally Posted by NiLuJe

Possibly something similar to how tar will SIGBUS when it tries to overwrite itself during the unpacking on Kindles because of the fuse layer?

Check the Kindle koreader.sh script, it has some extra ugly band-aid to make it behave (basically, copy tar somewhere safe like /tmp, and use that copy to do the unpacking).

Note that this only papers over the issue, if the startup script itself has been updated, it might also cause the restart to fail, and cause the usual post exit cleanup to be skipped.

I got it to update, by renaming the tar file from installed to koreader.updated.tar and restarting koreader. All went just fine. Does the above still apply ?

(I assume the fbink version in koreader will be updated in the future, my log is swamped by the message:
[FBInk] Couldn't find a Kobo version tag (onboard unmounted or not running on a Kobo?)!

[Frenzie]

If you overwrite the installed files with the installed files the end result is that you've done nothing.

Whatever you're doing is unrelated to koreader.sh because you're entirely in Lua. If it's not tar because you're getting a koreader.installed.tar, then logically it's either the download of the zsync file (which will show in debug output) or zsync itself instead.

Either way, you run the commands yourself.

./tar -whatever etc
./zsync -o ota/some-filename http://ota.koreader.rocks etc etc

You can just print the relevant string:

https://github.com/koreader/koreader....lua#L368-L387

Whichever fails, fails. But you already know it's because you mount ota to /opt/ram/koreader/ota.

[BloodRagg]

Should have been more clear I renamed the koreader.installed.tar to koreader.updated.tar which downloaded fine but did nothing in the update. Then restarted koreader, then the update proceeds

Unpacking files over a bind mount should not be a problem at all
This is substantiated by the fact that it works fine if I rename the tar file and restart koreader., the update process proceeds.

So all I have left is open files, but I will do some more testing when I have the time

[Frenzie]

Originally Posted by BloodRagg

Should have been more clear I renamed the koreader.installed.tar to koreader.updated.tar which downloaded fine but did nothing in the update.

But it didn't. That's what the "Failed to update KOReader" message is telling you. It means zsync failed.

[BloodRagg]

Night mode isnt working when returning back from sleep, but sleep has to be at least 5 or 10 minutes.
Its in daymode, then after a couple of touches is reverts back to nightmode.