Aura H2O Koreader and malformed nickel database

[Patricius]

After playing with KSM and fmon a little bit, I can be sure that the culprit must be something between Calibre and fmon. My previous statement of the usability of KSM might due to strong luck.

As the remuneration of my water proof function, I got this conclusion:
The database corruption occur if and only if any script in $mount_root/.kobo/fmon/ is executed And using calibre for file moving.

[davidfor]

So, you have done the isolation tests? That is, sideloading with calibre and not using fmon and sideloading without calibre and then using fmon?

[Patricius]

Yes, all of them (koreader, fmon, KSM, calibre and its plugins), which takes me great among of time:P.
I've even tested on an older version of calibre on windows, the results are the same.
I've also try to switch different versions of kobo plugins(and feature that do the kepub convertion), but seems that whatever plugin I'm using, the database will be corrupted.

[Patricius]

But for the sake of simplicity, I just comment out all lines in on_start.sh instead of making a fresh installation or manual update.

without calibre, (k)epub transfers are done without any problem. A typical process causing corruption and reverse it would be:
1. install fmon (or uncomment the on_start.sh & reboot) ->
2. connect and transfer with calibre (success or communication error, database malformed) ->
3. disconnected, content processed (or not), books and shelves disappear ->
4. comment out on_start.sh and reboot ->
5. broken items still broken, but newly transferred items shows normally.

[Ken Maltby]

I take it this is all a recent development. It makes me wonder if the Kobo software engineers would spend their valuable time thinking of ways to mess with the ability of 3rd party software to work with their firmware/software? This would be odd, though, given the original open and supportive approach that the Kobo devices started out with.

As some of you may have noticed, I have always found the database approach to file management a flaky idea to start with and the database easily corrupted. It should not matter to me as I make no use of it for my reading, thanks to KSM & Koreader. I don't think my experimental 600+ caliber generated library that is exposed to the database, has resulted in any corruption of the Kobo database, but I haven't checked it lately.

Luck;
Ken

[Patricius]

The Koreader would be my first choice if I only read in English...

However, Koreader is still lack of vertical text and ruby support, which is used by lots of Chinese and Japanese books.
Also, the way Calibre handles book titles in Chinese and Japanese is a nightmare because it mixes pronunciations of two languages into one title without accents, making it hard as cross-word puzzle to guess the real title in one glance. The best analogy I can come up with is to read a English sentence cadenceless with all non-prepositions in French pronunciations....

Maybe I should look into advboot and see if it will suit my needs.

[jackie_w]

Originally Posted by Ken Maltby

I take it this is all a recent development.

I don't think so. I've just looked at my notes. My database corruptions after installing koreader happened in November 2013.

Just for info I had similar database corruption problems after I tried one of the NightMode hacks on my H2O. That was March 2015.

[davidfor]

Originally Posted by Patricius

Also, the way Calibre handles book titles in Chinese and Japanese is a nightmare because it mixes pronunciations of two languages into one title without accents, making it hard as cross-word puzzle to guess the real title in one glance. The best analogy I can come up with is to read a English sentence cadenceless with all non-prepositions in French pronunciations....

I don't understand that statement. Calibre doesn't "handle" titles. What you see is what you typed in, or calibre pulled from either the book or downloaded from a site. If the title is displayed incorrectly, it is quite likely that the source that calibre used had it wrong. It does create a "sort title" based on the title using rules of the language. These can be tweaked.

And of course, if there is a bug, if you can report it and explain what is wrong, it will probably be fixed quickly.

[Patricius]

Originally Posted by davidfor

I don't understand that statement. Calibre doesn't "handle" titles. What you see is what you typed in, or calibre pulled from either the book or downloaded from a site. If the title is displayed incorrectly, it is quite likely that the source that calibre used had it wrong. It does create a "sort title" based on the title using rules of the language. These can be tweaked.

And of course, if there is a bug, if you can report it and explain what is wrong, it will probably be fixed quickly.

My fault, I should be more specific. For the word 'handle' I mean import and transfer, and the 'title' should be changed to 'file names'. I mean it will change the file names and makes them hard to recognize in the Koreader file manager. According to the project manager, this is done for the compatibility. I know there is a 'save to disk' function to preserve the file names but I'll lost the book shelf and kepub function if I use it.

For the fmon/calibre/kobo-plugin issue, if it success after using the device plugin (treat the device as a folder), is it right to make a conclusion that it is the plugin that should be blamed? And if I test it on an older firmware with the same setup and succeed, then it shouldn't be the fmon/calibre/kobo-plugin, right?

I've only read the rcS and fmon scripts (not fmon source, which I failed to find) briefly, and I don't understand the operation detail of nickel and calibre. Is it okay to throw this problem to the developers at this stage without knowing which one is the culprit?

[Patricius]

I just have managed to 'dual boot' into nickel or koreader by adding few lines to /etc/init.d/rcS.
Now if I want to boot into koreader, I only need to touch when it loops with blinking squares, for nickel, just do nothing. Mis-touches can be solved using koreader's home button.

Now without fmon (and KSM, which uses fmon), both readers work like a charm.
I've pasted the modified rcS here in case anyone has the same problem http://pastebin.com/4VnWgN4R
The modified parts are line 131, and lines 167+.

p.s. the 6 sec in line 168 is a magic number.

[frostschutz]

Careful with that, it could give you a malformed database.

[johnnyb]

I've had some success with hidden folders for Koreader books (it's a bit of a nuisance that you cannot point Koreader to that hidden folder, make it the default folder, and then hide all hidden files again to unclutter the folder a bit)...
I would recommend restarting the device after you leave Nickel and before you enter Nickel again. Otherwise it messes up some settings and you have to log in to some things again and re-apply some settings (like the custom "export highlights").
In addition, always check if Nickel is able to write to the database. To confirm, try deleting one of the tiles on the start screen. If it doesn’t delete and the screen refreshes, definitely restart your device.
(I know this is shitty behavior and should have been fixed ages ago by Kobo and the third party app developers but, welcome to Kobo, where many things are possible if you are able to identify the script in order to stick to it )

[Ken Maltby]

Originally Posted by johnnyb

I've had some success with hidden folders for Koreader books (it's a bit of a nuisance that you cannot point Koreader to that hidden folder, make it the default folder, and then hide all hidden files again to unclutter the folder a bit)...

But you can easily do that, the make the default folder for File Manager. You just long press on the link to that folder and a window will appear with the option to "Set as HOME directory". I assume you have set "Show hidden files" as default in the advanced options, or at least have it checked in the File Manager menu.

You could do the "then hide all hidden files again to unclutter the folder a bit)..." by unchecking the Show hidden files option, but the hidden folder is the one that has a name that starts with a dot/period, it is that - and all the folders and files inside that folder, that are hidden and shown. I have my library in ~50 folders inside a hidden folder.

Luck;
Ken

[Patricius]

Originally Posted by frostschutz

Careful with that, it could give you a malformed database.

If I understand it correctly, the database will be threatened only if I boot into Koreader and then go back to nickel and do some transfer. The patch has no difference with the original version if not tapped when booting.

Great kobo mods by the way! (・∀・)

[had]

Originally Posted by Patricius

If I understand it correctly, the database will be threatened only if I boot into Koreader and then go back to nickel and do some transfer. The patch has no difference with the original version if not tapped when booting.

Great kobo mods by the way! (・∀・)

Patricius maybe that issue could be completely avoided if in
/mnt/onboard/.adds/koreader/koreader.sh you cause a shutdown when koreader exits?