Kobo Database backup failed

[Filmbrain]

Hello --

Long time Calibre user, never really had a problem.

Updated this morning to 5.16.1. I have a Clara HD, with Calibre running on MacBook Pro (2019), Big Sur OS.

Not sure what to do. Any thoughts?

Error I receive:

calibre, version 5.16.1 (darwin, embedded-python: True)
Failed to backup device database: Failed: Backing up Kobo device database


*** in database main ***
Page 27882 is never used
Page 27897 is never used
Page 27899 is never used
Page 34776 is never used
Page 34843 is never used
Page 34864 is never used
Page 34882 is never used
Page 34926 is never used
Page 34978 is never used
Page 34979 is never used
Page 34980 is never used
Page 34981 is never used
Page 34985 is never used
Page 34986 is never used
Page 34988 is never used
Page 34989 is never used
Page 34992 is never used
Page 34993 is never used
Page 35006 is never used
Page 35008 is never used
Page 35013 is never used
Page 35017 is never used
Page 35018 is never used
Page 35020 is never used
Page 35054 is never used

Traceback (most recent call last):
  File "calibre/gui2/device.py", line 89, in run
  File "calibre_plugins.koboutilities.jobs", line 200, in do_device_database_backup
    raise Exception(check_result)
Exception: 
*** in database main ***
Page 27882 is never used
Page 27897 is never used
Page 27899 is never used
Page 34776 is never used
Page 34843 is never used
Page 34864 is never used
Page 34882 is never used
Page 34926 is never used
Page 34978 is never used
Page 34979 is never used
Page 34980 is never used
Page 34981 is never used
Page 34985 is never used
Page 34986 is never used
Page 34988 is never used
Page 34989 is never used
Page 34992 is never used
Page 34993 is never used
Page 35006 is never used
Page 35008 is never used
Page 35013 is never used
Page 35017 is never used
Page 35018 is never used
Page 35020 is never used
Page 35054 is never used

[PeterT]

Unfortunately it means a corrupted database on the Kobo. Two options
1) Logout on the Kobo which will drop the database, any content downloaded from Kobo and require you to resetup the device
2) restore the database from the last backup that calibre made

Sent from my Pixel 4a using Tapatalk

[davidfor]

As @PeterT said, this means the database on the device is corrupted. The database backup does a full check on the database after taking the copy to make sure nothing is wrong. Usually, if the backup reports problems, there will be issues seen elsewhere. Such as calibre reporting problems when getting the book list. But, those actions will only have errors if the parts of the database they use have problems.

You should also see the above errors if you use the database check function in the Kobo Utilities plugin. That is checking the database in the same way as the backup does. Or you can use any SQLite database manage to check as well.

For this error, it is not causing any errors at this point. But, it usually will lead to worse things later. And it can be fixable. You can try compressing the database, or a VACUUM in SQLite terms. The Kobo Utilities plugin can do this. It basically dumps the database and then reloads it. That reorganises everything and removes any unused space such as this. That should fix things. The alternative is to restore your last good backup.

[Filmbrain]

Thank you so much, this fixed it.

In case you're interested, I think I know how the database became corrupted.

I had a book on the device that would not open. (I forget the exact error message, but something about a bad file.) I connected my device in Calibre, deleted the bad book, and then copied it again to the device. (Same file.) It was after that (the next time I connected) that I received the database error message.

Originally Posted by davidfor

As @PeterT said, this means the database on the device is corrupted. The database backup does a full check on the database after taking the copy to make sure nothing is wrong. Usually, if the backup reports problems, there will be issues seen elsewhere. Such as calibre reporting problems when getting the book list. But, those actions will only have errors if the parts of the database they use have problems.

You should also see the above errors if you use the database check function in the Kobo Utilities plugin. That is checking the database in the same way as the backup does. Or you can use any SQLite database manage to check as well.

For this error, it is not causing any errors at this point. But, it usually will lead to worse things later. And it can be fixable. You can try compressing the database, or a VACUUM in SQLite terms. The Kobo Utilities plugin can do this. It basically dumps the database and then reloads it. That reorganises everything and removes any unused space such as this. That should fix things. The alternative is to restore your last good backup.

[davidfor]

Originally Posted by Filmbrain

Thank you so much, this fixed it.

In case you're interested, I think I know how the database became corrupted.

I had a book on the device that would not open. (I forget the exact error message, but something about a bad file.) I connected my device in Calibre, deleted the bad book, and then copied it again to the device. (Same file.) It was after that (the next time I connected) that I received the database error message.

That shouldn't have caused a problem. Deleting a book from the calibre works, or does whenever I do it. It might be related to whatever caused the book not to open. Or it might have just been some bad luck or timing with the eject.

[ownedbycats]

Any chance that you had disconnected without safely ejecting first? I believe Kobo uses FAT32 filesystems, which do not like sudden interruptions. I've corrupted my DB and some books a few times from it.

It's also possible that it's just a regular old bad block in the memory and it was touching both the DB and your specific book file. The good news is that it should mark that block as unusable and leave it alone.

[Filmbrain]

I did eject safely, but what I did notice is that the Kobo didn't do the usual post-eject "Importing new content" (or whatever the message is.) I thought because I had deleted and replaced the same book with the same file it wasn't necessary, so I didn't think much about it. Then when I plugged it in the next time I had the problem.

Originally Posted by ownedbycats

Any chance that you had disconnected without safely ejecting first? I believe Kobo uses FAT32 filesystems, which do not like sudden interruptions. I've corrupted my DB and some books a few times from it.

It's also possible that it's just a regular old bad block in the memory and it was touching both the DB and your specific book file. The good news is that it should mark that block as unusable and leave it alone.

[ownedbycats]

Do you have the "do not treat replacements as new books" option enabled in the driver settings? If you do and re-upload a book already on device, it won't run the import.

Otherwise, I can't say I've ever seen it fail. Sometimes it's a little slow to detect new books though, I've seen it take a full minute before the importing screen appears on a particularly large batch.

[davidfor]

Originally Posted by Filmbrain

I did eject safely, but what I did notice is that the Kobo didn't do the usual post-eject "Importing new content" (or whatever the message is.) I thought because I had deleted and replaced the same book with the same file it wasn't necessary, so I didn't think much about it. Then when I plugged it in the next time I had the problem.

As I said, it should work and does work here. When you delete the book using calibre, it should remove the database entries for the book. And after the eject, it should detect there were changes and import the book as a new one.

What happened after you ejected suggests either you didn't do the delete first (I know I frequently forget what I have and haven't done when fiddling with these devices) or the database corruption had happened by this point. Unfortunately, it is probably impossible to work out what actually happened and caused the problem.