[davidfor]

Originally Posted by ichrispa

I can confirm that the third method, copy-pasting to the KT, works. That's how I manage my ebooks.

I have run a couple more tests. As a premise, I took as granted that my database is somehow corrupted or the firmware garbled, so I performed a hard reset (restored firmware to 2.1.0) and then reupdated to 2.4.0. I then deleted the database and had the firmware create a blank one.

First of, the database version after that was 71, confirming that a manual updating in the order 2.1.0 to 2.3.1 to 2.3.2 to 2.4.0 somehow ends up having the database version getting stuck at 69.

That's strange that it didn't happen. I have been through this with the beta firmware and it always updates. The differences between the two database versions are a new table, "SyncQueue" which is only used for the new archiving function, and in "user" renaming column "DeviceID" to "___DeviceID". I can see the latter causing problems, but I would have expected errors like not opening books at all.

Starting with a blank database, I uploaded a series of books with series information and shelves set.

In no case was the KT ejected via calibre during these tests. It was always ejected from the OS using HAL.

Here's what I found:

Database closing

After starting calibre and uploading the books, the database was always closed after the job was completed. That applies to both a running instance of calibre as well as having calibre closed. In all cases, the following was returned when checking for open instances:

The database is always closed correctly.

That makes me happy

Database corruption

The database corruption scenario is the same as described above. It might however be ebook specific, as it appears to only happen with certain books and not with others. While I was testing around, I used the Harry Potter series this time. No corruption occured. When using A Song of Ice and Fire, the database got busted. I will later try to find out what attribute of an epub triggers this behavior.

I don't think you mentioned that some books didn't cause problems. I had was think it was all. But, someone reported something similar to today. I'm waiting to get a copy of the book to look at.

I did try to restore my old database by reinserting all values from the backup into the fresh database:

Although the "pragma integrity_check;" returns "ok" for the backup, when I tried to "dump" the table contents of the backup into the fresh database, numerous errors occured. That includes mutliple instances of primary key violations and non unique values where required errors. However this might be attributed to the different database version, since the backup was version 69 and the fresh instance was version 71.

That is interesting. Can you send me a copy of the database?

Setting series/shelve information

Here's the strange part. The behavior of this is very similar to what I described in my previous test: The books transfered show up, but the series/shelves information appears not to be set. Shelves appear empty if newly created.

Since I did not eject using calibre I inspected the changes the driver made to the database.

After adding the books using calibre, the contents table does not include the series and bookshelve information. In fact, the books are not present in the contents table at all!

After unmounting/disconnecting the device the KT goes into the "Processing contents" mode. No shelves or series information show up. When reconnecting the KT and reinspecting the database, the books are in the contents database, but have (obviously) no series/shelves information set.

Restarting calibre and running "Tranfer to device" again (on the books already on the device), then disconnecting the KT, results in skipping the "Processing contents" dialog, since the books are already in the database.

After that, both shelve information and series are shown properly.

That is how it works. Adding the book to the devices database is left up to the device. It adds a set of rows to the contents and the volume_shortcovers tables. The driver doesn't create these rows as I am not 100% sure what to put in them. Plus, Kobo has changed them a few times over the year I have been looking at this. Because of this, setting the series info is done on the next connect to calibre. You don't need to resend the book, just connect.

The shelves should have been set when the book was sent. This is info in different tables and is simple to maintain. I will have to recheck this in case something changed.

The automatic bits of this happen if "Metadata management" is set to automatic. If it is either manual or on send, it only happens when the book is sent. That would need a resend when the device is connected again.

(My) conclusion

I cannot conclude to what the database corruption is attributed. But it appears to be related to specific epubs.

When the KT processes new content, all information is derived from the metadata included in the epub. Any manual changes to the database are effectively overwritten by "Processing contents". Only if an ebook is already in the database, the series and shelves information are kept.

I would conclude that the driver does not add entries to the database when transferring books. It only runs UPDATE, not INSERT. Only if a book is already in the database by having run "Processing Contents" the driver is capable to set the proper information.

Yep.

ReEDIT:

While finishing up the tests I made a little mistake that proved to be very conclusive: I deleted all file/folders on the kobo, then unplugged it.

Since there was no database present and no more contents to be scanned, the KT should not show any books. However, the contents show by the KT where identical to what was there before I accidentally deleted everything. Books, shelves, metainfo - everything was accessable.

I would deduce that the KT does not close its handle of the database when connecting to a PC.

I haven't done that, but I have seen the same effect. I have seen something like that when the database was corrupted. The device worked OK, I connected to the PC and calibre reported the corrupt database. I ejected and the device kept working OK until I powered off. On restart, tried to read the corrupt database. Sometimes, it showed old data, sometimes it reset the database and prompted to do the set-up.

With your "experiment" it makes me thing that when connected to the PC, the Kobo application is still in memory including the contents of the database. When the disconnect happens, if for some reason it can't read the database, it just continues with the in-memory data. That is a theory at least.

In any case, the important thing is your Touch is working. And it should continue working.