Firmware 4.37.21586

[Aleron Ives]

Originally Posted by enuddleyarbl

In my case, this firmware (Firmware 4.37.21586) caused it to happen all the time.

If it's that much of a problem, why don't you just roll back to 4.35? That would seem to be the most reliable solution.

[enuddleyarbl]

I should have. But, since rebooting the Kobo before connecting fixed things, I figured "it'll be fine." I neglected to consider the human factor.

I guess I'll be sideloading my books yet again.

[John F]

Originally Posted by DNSB

...

And yes, my suggestion about using a NickelMenu entry to reboot before connecting to the computer is the only consistent solution I've found to connect without seeing .sqlite-wal and .sqlite-shm files indicating a database that was not closed before the connection.

...

I don't know what those files are, but are you saying that a manual power down and power on is different then NickelMenu's reboot?

[JSWolf]

Next time I connect my Libra 2 to the computer, I'll check for those extra sqlite files.

[DNSB]

Originally Posted by Aleron Ives

Which firmware introduced the constant filesystem dirty bit problem?

It's been an issue on and off for quite a while. The last 3 releases have had more of an issue on my Sage than on my Clara HD where I rarely see the issue. Even more fun, it's not a constant issue in my testing. On my Sage, it happens about 1 in 5 connects. The only constant that I see is that either the KoboReader.sqlite or the BookReader.sqlite database file will be accompanied by the <filename>.sqlite-wal (write-ahead log) and <filename>.sqlite-shm (shared memory). When the database is properly closed, the sqlite-wal and sqlite-shm files will be merged into the database and removed.

For now, my workaround is rebooting before the USB connection which forces a database shutdown.

As previously mentioned, Kobo is aware of the issue and it will be fixed—hopefully sooner than later.

2.2. Write-Ahead Log (WAL) Files

A write-ahead log or WAL file is used in place of a rollback journal when SQLite is operating in WAL mode. As with the rollback journal, the purpose of the WAL file is to implement atomic commit and rollback. The WAL file is always located in the same directory as the database file and has the same name as the database file except with the 4 characters "-wal" appended. The WAL file is created when the first connection to the database is opened and is normally removed when the last connection to the database closes. However, if the last connection does not shutdown cleanly, the WAL file will remain in the filesystem and will be automatically cleaned up the next time the database is opened.

2.3. Shared-Memory Files

When operating in WAL mode, all SQLite database connections associated with the same database file need to share some memory that is used as an index for the WAL file. In most implementations, this shared memory is implemented by calling mmap() on a file created for this sole purpose: the shared-memory file. The shared-memory file, if it exists, is located in the same directory as the database file and has the same name as the database file except with the 4 characters "-shm" appended. Shared memory files only exist while running in WAL mode.

The shared-memory file contains no persistent content. The only purpose of the shared-memory file is to provide a block of shared memory for use by multiple processes all accessing the same database in WAL mode. If the VFS is able to provide an alternative method for accessing shared memory, then that alternative method might be used rather than the shared-memory file. For example, if PRAGMA locking_mode is set to EXCLUSIVE (meaning that only one process is able to access the database file) then the shared memory will be allocated from heap rather than out of the shared-memory file, and the shared-memory file will never be created.

The shared-memory file has the same lifetime as its associated WAL file. The shared-memory file is created when the WAL file is created and is deleted when the WAL file is deleted. During WAL file recovery, the shared memory file is recreated from scratch based on the contents of the WAL file being recovered.

[DNSB]

Originally Posted by John F

I don't know what those files are, but are you saying that a manual power down and power on is different then NickelMenu's reboot?

They are effectively the same. Just a lot faster to open NickelMenu's menu and tap on 'Reboot' since the manual power cycle requires me to remove my Sage from it's case to access the power button, holding the power button to force the shutdown and then hold it again to power the Sage back on. A simple tap on Nickelmenu and then reboot is a lot easier. Replace <location> in the line below with main, library, reader, browser or selection depending on where you want to see the reboot option.

menu_item : <location> : Reboot             : power          : reboot

[compurandom]

If you have a corrupt database, sometimes you can export it and import it to a new database and copy the new one over top of the old one and _THEN_ reboot and it fixes it. But damage to the database form this can accumulate until it isn't corrupt but isn't right either.

A better solution when you have a corrupt database is to copy an older backup in and then reboot.

And if that doesn't work, just *delete* the database and and the kobo will do a soft factory reset that doesn't delete your side loaded books. Log back into your account and it will reimport them. However screen savers and dictionaries still get deleted.

[enuddleyarbl]

I checked the database backup Kobo Utilities made when I connected this morning and there was no indication of that copy of KoboReader.sqlite being corrupt (my understanding is that KU checks the database copy after making it and if corrupt, changes the name to reflect that). So, I unzipped the backup, copied that file over to my Sage's .kobo directory (replacing the file there), disconnected and rebooted. Everything seems to be good now.

[Aleron Ives]

What's the difference between BookReader.sqlite and KoboReader.sqlite? Does only one of them tend to corrupt?

[DNSB]

Originally Posted by Aleron Ives

What's the difference between BookReader.sqlite and KoboReader.sqlite? Does only one of them tend to corrupt?

KoboReader.sqlite is the main database file that calibre interacts with. The BookReader.sqlite file is an encrypted database and I am not able to say anything about what it is for. There is also a 3rd sqlite file called fonts.sqlite which, oddly enough, lists the fonts in the user fonts directory.

KoboReader.sqlite is the one that corrupts more often in my experience but then it is the larger and more active database. On my Sage, it is 69MB compared to 19KB for BookReader and 9KB for fonts. My suspicion is that this likely due to calibre attempting to use KoboReader.sqlite when you connect your Kobo ereader to calibre.

[Aleron Ives]

According to davidfor, BookReader.sqlite is for books borrowed from Kobo Plus and libraries.

https://www.mobileread.com/forums/sh...d.php?t=341536

I would assume that unless you're doing one of those things that the database will never be modified, and thus it's not likely to corrupt, since it's never open.

[John F]

Originally Posted by DNSB

They are effectively the same. Just a lot faster to open NickelMenu's menu and tap on 'Reboot' since the manual power cycle requires me to remove my Sage from it's case to access the power button, holding the power button to force the shutdown and then hold it again to power the Sage back on. A simple tap on Nickelmenu and then reboot is a lot easier. Replace <location> in the line below with main, library, reader, browser or selection depending on where you want to see the reboot option.

menu_item : <location> : Reboot             : power          : reboot

Thanks. Just curious.

[compurandom]

Originally Posted by Aleron Ives

According to davidfor, BookReader.sqlite is for books borrowed from Kobo Plus and libraries.

I suspect it has reading statics and reading life info in it too.

[Aleron Ives]

That's doubtful. Looking at my last backup, my BookReader.sqlite file hasn't been modified since I initialised my device, whereas the KoboReader.sqlite file was last modified on the day I made the backup. BookReader.sqlite doesn't seem to be used if you don't use Kobo Plus or OverDrive on the device.

[compurandom]

Originally Posted by Aleron Ives

That's doubtful. Looking at my last backup, my BookReader.sqlite file hasn't been modified since I initialised my device, whereas the KoboReader.sqlite file was last modified on the day I made the backup. BookReader.sqlite doesn't seem to be used if you don't use Kobo Plus or OverDrive on the device.

good to know!