[ichrispa]

Hello Davidfor,

first of all I apologize for not realizing that updating the shelves and series info is a two phase process. Given how volatile the database is across different firmware versions, I fully understand and support you design choice. It's quite reasonable to do it like that, I just didn't expect it.

I don't think you mentioned that some books didn't cause problems. I had was think it was all. But, someone reported something similar to today. I'm waiting to get a copy of the book to look at.

How would you like it send to you? I have isolated a wonderful speciment: Clean and freshly created DB, upload the book and it appears on the home screen. Upload it again and the database is broken (becomes noticable after a reboot).

It is imperative that if no "processing contents" screen appears, a reboot of the KT is done. Otherwise DB corruption or even valid changes will not become visible. That should be kind of obvious and I suppose it is old news to you, but it took me some time to figure out the sqlite caching problem before I came up with that one.

That is interesting. Can you send me a copy of the database?

Certainly. Sending it to you via PM right away.

The shelves should have been set when the book was sent. This is info in different tables and is simple to maintain. I will have to recheck this in case something changed.

The automatic bits of this happen if "Metadata management" is set to automatic. If it is either manual or on send, it only happens when the book is sent. That would need a resend when the device is connected again.

Yes, shelves are created. However the shelves only appear empty after a reboot. And they are only filled with books after the second pass sync

My database management is set to manual right now, because fully automatic updating usually screws up my kobo as soon as it is connected. However updating on send is also a viable alternative. Now that I found out how the driver works, the option becomes much more understandable.

With your "experiment" it makes me thing that when connected to the PC, the Kobo application is still in memory including the contents of the database. When the disconnect happens, if for some reason it can't read the database, it just continues with the in-memory data. That is a theory at least.

I believe that it drops the in memory content/sqlite journal when filesystem changes are detected (processing contents) after the device is disconnected. If it cannot detect changes, it appears indeed that it continues using a cached version of the database. Maybe if one could "induce" a trivial filesystem change, such as altering attributes (access/create time?) of an ebook, this behavior could be inhibited.

In any case I will send you the ebook and the database right away. I have a genuine interest in getting the driver to work with my KT, however if you believe this is an isolated problem (as in error 40: "the error is setting 40cm from the screen") I won't take up any more of your time



EDIT:
While describing the procedure for how to "break" the KT using that specific epub in my PM to you, the solution to the problem dawned to me... here's what I believe is happening: The problem is neither the database version nor some specific epub, it is the in cache database instance.

Roughly described, the KT is always using a cached DB instance. Processing contents only updates relevant information, not the entire database.

To demonstrate, here's what I wrote in the PM:

1. Remove all ebooks and database from KT.
2. Copy clean/empty/registered database to KT (registered as foo, otherwise as created by reset KT).
3. Disconnect & reset KT.
4. Send epub to Main memory.
5. Eject KT.

At this point, you can see the epub on the home screen without any series information. The shelve is also not visible, although the table that defines shelves is unrelated to the contents processed by the KT and the database entry is there.

As I understand it, there is currently a cached DB version in the KT. The cache itself and parts of the file where updated durch the "processing contents" phase - other parts of the database, such as the shelves, where not updated and are still served from the cache. That explains why the new shelf is not visible yet present in the DB. However the cache and the database are not strictly speaking contradictory, they are just out of sync.

There are now two ways to proceed:
1) Reconnect to calibre straight away again and attempt to update the database again.
2) If you reboot the KT, the shelve appears - ie the cache is flushed to the file and the file reloaded. Cache and DB are now consistent again.

If you choose (1):
When reuploading the book or updating the database, the discrepancy between the cache and the file becomes contradictory. The database is thus corrupted.

If you choose (2):
When reuploading the book or updating the database, Calibre updates the "correct" database. Again: some information is cached and does not become visible until the KT rebooted again.

I did manage to confirm this uploading a single ebook using method 2 to a fresh and clean database. Uploading the same ebook with (1) fails. The procedure still always fails when attempting (2) with multiple ebooks at once, resulting in a corrupted DB after the first reboot. The manifestation of this problem is that the books uploaded before the reboot are all no longer visible (thought the shelves are).

Does that make any sense?