Aura HD KoboUtilities Plugin: Database backup failure

[Skydog]

First, my thanks to David Forrester for his outstanding plugin.

I setup the plugin to backup the database upon connection and it worked but only twice. Recently, when connecting my Aura HD to Calibre the error below appears. What might be causing this? And are any subsequent manual backups good or corrupt?

Thank you.

Edit: I am also experiencing the same result with my Mini


calibre, version 1.34.0 (win32, isfrozen: True)
Failed to backup device database: Failed: Backing up Kobo device database

Backing up Kobo device database
Python function terminated unexpectedly

rowid 739 missing from index sqlite_autoindex_ShelfContent_1
wrong # of entries in index sqlite_autoindex_ShelfContent_1 (Error Code: 1)
Traceback (most recent call last):
File "site.py", line 132, in main
File "site.py", line 109, in run_entry_point
File "site-packages\calibre\utils\ipc\worker.py", line 195, in main
File "site-packages\calibre\utils\ipc\worker.py", line 139, in arbitrary_n
File "calibre_plugins.koboutilities.jobs", line 173, in do_device_database_backup
Exception:
rowid 739 missing from index sqlite_autoindex_ShelfContent_1
wrong # of entries in index sqlite_autoindex_ShelfContent_1

[DNSB]

Originally Posted by Skydog

First, my thanks to David Forrester for his outstanding plugin.

I setup the plugin to backup the database upon connection and it worked but only twice. Recently, when connecting my Aura HD to Calibre the error below appears. What might be causing this? And are any subsequent manual backups good or corrupt?

Thank you.

Edit: I am also experiencing the same result with my Mini


calibre, version 1.34.0 (win32, isfrozen: True)
Failed to backup device database: Failed: Backing up Kobo device database

Backing up Kobo device database
Python function terminated unexpectedly

rowid 739 missing from index sqlite_autoindex_ShelfContent_1
wrong # of entries in index sqlite_autoindex_ShelfContent_1 (Error Code: 1)
Traceback (most recent call last):
File "site.py", line 132, in main
File "site.py", line 109, in run_entry_point
File "site-packages\calibre\utils\ipc\worker.py", line 195, in main
File "site-packages\calibre\utils\ipc\worker.py", line 139, in arbitrary_n
File "calibre_plugins.koboutilities.jobs", line 173, in do_device_database_backup
Exception:
rowid 739 missing from index sqlite_autoindex_ShelfContent_1
wrong # of entries in index sqlite_autoindex_ShelfContent_1

At a wild eyed guess, your database has been corrupted.

When you run the "check device database" option, what does it return?

You might want to check the Kobo Utilities thread in the Calibre/GUI Plugins area;

Here's a link near the end of the thread where some database issues were being discussed: https://www.mobileread.com/forums/sho...&postcount=496

In theory, you can repair some corruption in an SQLite database but it is often easier and faster to recreate the database. For a Kobo ereader, logging out of your account is one way of doing a database rebuild. You will lose quite a bit of information such as reading position.

[Skydog]

Thank you. I should have mentioned that I did indeed perform a database check on both the Aura HD and Mini. Both resulted ok as shown below:

Result of running 'PRAGMA integrity_check' on database on the Kobo device:


ok

I only side load epubs and never turn on the wifi.

[PeterT]

I get the feeling David might like to see this database to try to understand what might have happened to it; especially since the integrity check seems to work while the backup fails.

[PeterT]

I also wonder if issuing a sqlite "reindex" command to force a recreation of all the indices would help.

[davidfor]

If the backup shows as corrupt but the database on the device doesn't, it is probably a timing issue. The way the backup job is run at the moment it could run at the same time as one of the device jobs. As the error is in an index for the ShelfContent table, the copy was probably done while shelf management was being done.

I have seen this a couple of time and it has been reported by at least one other. I have been testing a version of the plugin that runs the jobs as device jobs. That prevents this problem. I'll be releasing this today or tomorrow.

[Skydog]

Ah, that makes sense even to me. The only job running at the time was the updating of my "finished" shelf as I have a custom column created for books reaching 100% read.

Thank you very much, davidfor.

Originally Posted by davidfor

If the backup shows as corrupt but the database on the device doesn't, it is probably a timing issue. The way the backup job is run at the moment it could run at the same time as one of the device jobs. As the error is in an index for the ShelfContent table, the copy was probably done while shelf management was being done.

I have seen this a couple of time and it has been reported by at least one other. I have been testing a version of the plugin that runs the jobs as device jobs. That prevents this problem. I'll be releasing this today or tomorrow.

[davidfor]

I have just released the fix for this. See https://www.mobileread.com/forums/showthread.php?p=2827309 for details.