Aura H2O Koreader and malformed nickel database

[Ken Maltby]

I don't believe there is anything of Koreader left running when you exit Koreader. (Unlike with Nickel.) The only program that provides me a means of checking the database would be calibre, at the moment. How are you finding that the database is corrupted?

Luck;
Ken

[johnnyb]

Originally Posted by Ken Maltby

But you can easily do that, the make the default folder for File Manager. You just long press on the link to that folder and a window will appear with the option to "Set as HOME directory". I assume you have set "Show hidden files" as default in the advanced options, or at least have it checked in the File Manager menu.

You could do the "then hide all hidden files again to unclutter the folder a bit)..." by unchecking the Show hidden files option, but the hidden folder is the one that has a name that starts with a dot/period, it is that - and all the folders and files inside that folder, that are hidden and shown. I have my library in ~50 folders inside a hidden folder.

Luck;
Ken

Last time I checked, when unchecking the "show hidden files" option, Koreader didnt default to my previous choice anymore, a hidden folder named .PDF
Has that behavior changed?

[Ken Maltby]

Originally Posted by johnnyb

Last time I checked, when unchecking the "show hidden files" option, Koreader didnt default to my previous choice anymore, a hidden folder named .PDF
Has that behavior changed?

Hmm... Right, without the "show hidden files" option checked, your .PDF hidden folder is not shown.

Luck;
Ken

[Patricius]

Originally Posted by had

Patricius maybe that issue could be completely avoided if in
/mnt/onboard/.adds/koreader/koreader.sh you cause a shutdown when koreader exits?

Instead of that, I prefer to make koreader run in foreground and reboot after it exists in rcS. This could make the patch koreader-update-independent. The original patch is dirty enough (wait nickel to launch and kill it), so I don't care to add up some more dirtiness in exchange of the ease of update-handling.

The new version: http://pastebin.com/iXMztd2F

[Patricius]

How are you finding that the database is corrupted?

There are three ways to know it:
1. Calibre pops communication error dialog in file transfer.
2. Like johnnyb has said, nickel home tiles don't responsive to modification.
3. Random books in nickel home/shelves/library having blank cover and a "buy" drop down menu on the right-hand side.

Is it possible for Koreader to broke some presumptions needed by nickel and other background processes?

[tshering]

Originally Posted by Patricius

There are three ways to know it:
1. Calibre pops communication error dialog in file transfer.
2. Like johnnyb has said, nickel home tiles don't responsive to modification.
3. Random books in nickel home/shelves/library having blank cover and a "buy" drop down menu on the right-hand side.

KSM users can check the integrity by selecting: tools > kobo diverse.msh > db chk integrity.sh

Originally Posted by Patricius

Is it possible for Koreader to broke some presumptions needed by nickel and other background processes?

I am rather sure that KSM is the culprit.

[Patricius]

Originally Posted by tshering

I am rather sure that KSM is the culprit.

But it appears there are two kinds of scenarios that causes the database corruption:

1. Calibre with kobo plugins (touch, touch_extend) and any running fmon process, after file transferred.
I've done the isolation test for this (without Koreader). Since KSM uses fmon, I consider the corruption with KSM is due to the same reason. Without the source, I don't know what exactly fmon does besides file-watching and process-spawning.

2. Kill the nickel after it have killed the on-animator, lauch koreader.sh and some transfers with Calibre after it brings back the nickel.
This is simply what my first modification does. I'm still a layman about nickel, maybe frostschutz would be kind enough to explain the reason of corruption in this case? I have no time to test upon this scenario in detail (without calibre/koreader, or even kill and relaunch nickel then do some transfers).

[tshering]

Originally Posted by Patricius

But it appears there are two kinds of scenarios that causes the database corruption:

1. Calibre with kobo plugins (touch, touch_extend) and any running fmon process, after file transferred.
I've done the isolation test for this (without Koreader). Since KSM uses fmon, I consider the corruption with KSM is due to the same reason. Without the source, I don't know what exactly fmon does besides file-watching and process-spawning.

2. Kill the nickel after it have killed the on-animator, lauch koreader.sh and some transfers with Calibre after it brings back the nickel.
This is simply what my first modification does. I'm still a layman about nickel, maybe frostschutz would be kind enough to explain the reason of corruption in this case? I have no time to test upon this scenario in detail (without calibre/koreader, or even kill and relaunch nickel then do some transfers).

When nickel is started from KSM, there are still some scripts running that are located on the user partition (and in many cases instances of fmon). That being the case, the user partition cannot be unmounted when nickel connects via usb. This is not pretty, and becomes dangerous if (and I think, but am not sure, only if) files are modified that are currently in use (which seems to be the case with the database when one adds books with calibre.)

[Patricius]

Originally Posted by tshering

When nickel is started from KSM, there are still some scripts running that are located on the user partition (and in many cases instances of fmon). That being the case, the user partition cannot be unmounted when nickel connects via usb. This is not pretty, and becomes dangerous if files are modified that are currently in use (the database)

So you are talking about the first case?
But I would whether think the user partition is umount successfully (maybe forced, regardless of opened database) because the device entered the "slave mode" after all.
This explains the behaviour I've mentioned in #19:

1. install fmon (or uncomment the on_start.sh & reboot) ->
2. connect and transfer with calibre (success or communication error, database malformed) ->
3. disconnected, content processed (or not), books and shelves disappear ->
4. comment out on_start.sh and reboot ->
5. broken items still broken, but newly transferred items shows normally.

Which means the database has been damaged sometimes during the `umount -f` process.

If this is true, maybe adding a script that kills/re-spawns all fmon processes then make it triggered by an udev rule will solve the problem.

But what about the second case? Someone with shell access of the device can try my setup and lsof the user partition after koreader has brought nickel back. :/
Or, I'll do it in /etc/init.d/rcS if my database get ruined again. :P