Calibre seems to reset metadata.calibre everytime I connect

[DNSB]

Originally Posted by JeffElkins

128gb is insane overkill, but what the hell ...

It was the only SDXC µSD card that was not in use I had on hand. But having 33+ years at 8 hours per day at ~250WPM books to read is, possibly, a bit of overkill. The years came from adding up the word count for the books in calibre, dividing by 250 to get minutes, dividing the minutes by the minutes in a 365.25 day year and multiplying by 3 for the 8 hours/day).

((1,485,512,307/250)/525,960)*3=33.9

Edit: to generalize this, replace *3 by *(24/hours read per day), i.e. 6 hours per day would give 4.

[Quoth]

Why limit yourself to 8 hours a day?

[DNSB]

Originally Posted by Quoth

Why limit yourself to 8 hours a day?

I picked it as an example, if you want to use a higher or lower number feel free to modify my equation to reflect your choice. I do have one acquaintance who reads more than 8 hours per day though I have wondered how their spouse tolerates them spending most of the day in a recliner in the den and doing virtually nothing around the house.

[ownedbycats]

Originally Posted by DNSB

It was the only SDXC µSD card that was not in use I had on hand.

What were the other cards used for? Depending on what, I would've probably transferred the data from a smaller card and then just swap them out.

[DNSB]

Originally Posted by ownedbycats

What were the other cards used for? Depending on what, I would've probably transferred the data from a smaller card and then just swap them out.

The other cards I had available were too small to be SDXC and I did not feel like swapping cards around since the one device I have with a 64GB µSD card would not have benefited from upgrading to 128GB. My Raspberry Pi is where the 128GB card came from when I replaced it with a larger card and so was conveniently at hand when I decided to see how well the Clara HD would handle a SDXC card.

[i_am_not_a_duck]

I'm back and I have the log.
I think the relevant part is this. An error box also pops up with this message.

calibre, version 7.17.0
ERROR: Error: Error communicating with device

CorruptError: database disk image is malformed

Traceback (most recent call last):
File "calibre\gui2\device.py", line 112, in run
File "calibre\gui2\device.py", line 566, in _books
File "calibre\devices\kobo\driver.py", line 2083, in books
File "C:\t\t\apsw-42s16m2x\src\cursor.c", line 240, in resetcursor
apsw.CorruptError: CorruptError: database disk image is malformed

In the debug log Calibre opened after I exited, this was there after it. Not sure if it's relevant.

'list' object has no attribute 'set_debugging_title'

Traceback (most recent call last):
File "calibre\gui2\device.py", line 112, in run
File "calibre\gui2\device.py", line 596, in _sync_booklists
File "calibre_plugins.kobotouch_extended.device.driver" , line 513, in sync_booklists
File "calibre\devices\kobo\driver.py", line 1020, in sync_booklists
File "calibre\devices\kobo\driver.py", line 2569, in update_device_database_collections
AttributeError: 'list' object has no attribute 'set_debugging_title'

[PeterT]

This is the classic malformed Kobo database. Unfortunately the solution is to either restore from a backup if you have one (many people use the Kobo Utilities plugin to copy the database each time the device is connected to calibre), or recreate the database.

Recreation is forced by logging out from the Kobo account (Settings | Accounts and Sign out). You'll then need to redo the Kobo setup.

Sent from my Pixel 7a using Tapatalk

[i_am_not_a_duck]

What causes it?
Is it preventable?
It's extraordinarily annoying to have to basically set it up all over again and losing reading positions, annotations, which books I've finished reading, and the saved dictionary entries. It's like starting from scratch every time.

[PeterT]

Unfortunately things like crashes can cause problems.

The database is located on the portion of the internal SD card that is shared with the PC. Before the database can be opened on the PC side it has to be fully closed by all processes on the Kobo side. There was a long standing issue in the 4.x code (and I don't remember which version fixed it) where the database was not closed on the Kobo side when the storage was shared to the PC.

Sent from my Pixel 7a using Tapatalk

[ownedbycats]

For a bit there was a firmware issue where the database sometimes left open when connecting to USB. It's fixed now but if you're not up-to-date on the firmware you might be affected.

It might also be a sign of a failing SD card. My Aura HD corrupted and spontaneously reset several times before it died.

[Aleron Ives]

The broken firmware versions are 4.37 (all) and 4.38.21908. The fix was added in 4.38.23038.

[PeterT]

Originally Posted by i_am_not_a_duck

So everything works relatively well up to this point. Occasionally if I'm trying to upload a LOT of books at once, the Kobo will fail to import some of them and I'm not sure why, but I'll deal with that after this problem has been resolved.

Now: with Kobo disconnected and Calibre not currently running, if I connect the Kobo to the computer and browse to it, I see the metadata.calibre file is still intact. I open Calibre, and as soon as it detects the device, it overwrites metadata.calibre with a fresh empty one and therefore cannot see what is already on the device. It does not pop up any errors.

I realize that no one commented on the calibre.metadata file.

This is created and managed by calibre on the device and is obnly used by calibre.

It is written to the device after books have been transferred.

I don't think the Kobo driver used data from this file; rather it processes data directly from the KoboReader.sqlite database. This database is maintained by the Kobo, and is populated with new content during the post-disconnect processing on the device; the period where you see the message about the device importing new content.

The Kobo driver does update this database with items like the series info and any tags / collections associated with the book. This update does requite the database records to be present, which is why the Kobo requires a second connection to the PC after the initial transfer.

[i_am_not_a_duck]

Originally Posted by PeterT

I realize that no one commented on the calibre.metadata file.

This is created and managed by calibre on the device and is obnly used by calibre.

It is written to the device after books have been transferred.

I don't think the Kobo driver used data from this file; rather it processes data directly from the KoboReader.sqlite database. This database is maintained by the Kobo, and is populated with new content during the post-disconnect processing on the device; the period where you see the message about the device importing new content.

The Kobo driver does update this database with items like the series info and any tags / collections associated with the book. This update does requite the database records to be present, which is why the Kobo requires a second connection to the PC after the initial transfer.

So seeing as I was going to go through another reset, I figure this was a good time to update things and play around to see if I could manage to figure anything out. Since you mentioned the Kobo Driver and the database, I decided not to install Kobo Utilities this time thinking that maybe there was a bug in that plugin responsible for metadata file problem.

Originally Posted by Aleron Ives

The broken firmware versions are 4.37 (all) and 4.38.21908. The fix was added in 4.38.23038.

I updated the firmware, Calibre, and TouchExtended and now I no longer get the "Attribute Error" from my previous log.

NOW, I get this:

calibre, version 7.18.0
ERROR: Error: Error communicating with device

Not a ZIP file

Traceback (most recent call last):
File "calibre\ebooks\oeb\polish\container.py", line 1181, in __init__
File "calibre\utils\zipfile.py", line 774, in __init__
File "calibre\utils\zipfile.py", line 809, in _GetContents
File "calibre\utils\zipfile.py", line 824, in _RealGetContents
calibre.utils.zipfile.BadZipfile: File is not a zip file

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre\gui2\device.py", line 112, in run
File "calibre\gui2\device.py", line 646, in _upload_books
File "calibre_plugins.kobotouch_extended.device.driver" , line 343, in upload_books
File "calibre\devices\kobo\driver.py", line 2275, in upload_books
File "calibre_plugins.kobotouch_extended.device.driver" , line 177, in _modify_epub
File "calibre_plugins.kobotouch_extended.container" , line 117, in __init__
File "calibre\ebooks\oeb\polish\container.py", line 1188, in __init__
File "calibre\utils\localunzip.py", line 255, in extractall
File "calibre\utils\localunzip.py", line 242, in _extractall
ValueError: Not a ZIP file

hmmm.

[i_am_not_a_duck]

Ah, sorry. False alarm. The culprit was a corrupted epub.
I'll report back after I try the most recent plugin again.

[PeterT]

The metadata management is actually within the kobo driver. The utilility is actually pretty benign, and in your case I would suggest installing it and using the option to backup the database on every connect.