[KOBO] Calibre has issue automatically backing up to remote disk

[Derf]

Hi,
I have a 2017 iMac, and I have Calibre set up to backup to a remote disk on my LAN

afp://192.168.1.2/Media/Books/Calibre/

This is also where Calibre's database is and where it imports and stores my ePub files to without issue when I drag them into the Calibre window.

When I connect my Kobo Clara HD, and it attempts to automatically backup to this location, it fails with the error

Spoiler:

Failed: Backing up the Kobo device database

calibre, version 3.31.0 (darwin, embedded-python: True)
Failed to backup device database: Failed: Backing up Kobo device database

CantOpenError: unable to open database file

Traceback (most recent call last):
File "site-packages/calibre/gui2/device.py", line 87, in run
File "calibre_plugins.koboutilities.jobs", line 196, in do_device_database_backup
File "calibre_plugins.koboutilities.action", line 4722, in check_device_database
File "src/cursor.c", line 1019, in APSWCursor_execute.sqlite3_prepare
File "src/statementcache.c", line 386, in sqlite3_prepare
CantOpenError: CantOpenError: unable to open database file

A file is created with the resulting name
KoboReader-ClaraHD-N249840032138-20180909-193456_CORRUPT.sqlite
The filesize is 19.3 MB

But if I go to Kobo Utilities and choose to manually backup, it completes, and the file is called
KoboReader-ClaraHD-N249840032138-20180909-193456.sqlite
The filesize is 19.3 MB

I have changed the backup location to my local drive to save the backup file to my desktop, and it will auto backup when I connect the device. Can anyone point me in the direction of solving why auto-backup to a remote disk fails, but manually backing up to a remote disk work?

[davidfor]

That error has absolutely nothing to do with where the calibre database and directory are. In fact, it has nothing to do with calibre except that it was run by a plugin in calibre.

You are using the Kobo Utilities plugin to do a backup of the database that is on the Kobo device. What this does is copy the database file from the device to a directory that you configure in the plugin. After the copy has been done, the plugin checks the database is valid. In this case, that failed.

What this usually means is that the database on the device is corrupt in some way. Exactly what will happen depends on how bad the problem is and were it is. Frequently, if you get the error during the backup, then you will also get an error when calibre access the database to get the list of books. But, the check is for all of the database and the problem could be in an area that is not access by calibre.

With a corrupt database, usually what happens is that if you restart the device, you will notice the problems. For a lot of things, the device runs on in memory copies of the data and writes to the database as needed. And only reads when the device is started or when some of the less common functions are done. That means that if you restart, you might lose the reading status or collections or books might need to be reimported.

There is a possibility that the corruption was caused during the copy. So, I suggest checking the database on the device. There is a function in the Kobo Utilities plugin to do this. If there are errors, then it should be OK and hopefully it is a once-off copy problem. If there are errors, you will need to fix it. Either restore a recent database backup, and restart the device as soon as you remove it from the PC. Or signout from the Kobo account on the device. When you signin again, a new database will be created. But, you will lose reading status for sideloaded books.

[Derf]

The database on the device is fine. I have all my reading stats, all my current reading locations. Everything is fine on the device.

When I use the plugin to fix errors on the device database it simply shows

Spoiler:

Result of running 'PRAGMA integrity_check' on database on the Kobo device:


ok

When the "automatic" backup happens when I plug in the device (if I have it so configured) happens, it throws an error. BUT, when there is no automatic backup, or when I manually use the plugin to create a backup, the backup completes with no error.

Again, as noted in my troubleshooting, if I change the target location of the backup from a remote disk on my LAN to a local hard disk, the problem vanishes. But when I set it back to the remote disk again, the problem reappears.

I wouldn't say the problem is with the remote disk, per-se, since manual backups complete without error, and Calibre is storing its database and ePub files there. But something is off with regards to the automatic backup on a remote disk.

[davidfor]

Originally Posted by Derf

The database on the device is fine. I have all my reading stats, all my current reading locations. Everything is fine on the device.

When I use the plugin to fix errors on the device database it simply shows

Spoiler:

Result of running 'PRAGMA integrity_check' on database on the Kobo device:


ok

When the "automatic" backup happens when I plug in the device (if I have it so configured) happens, it throws an error. BUT, when there is no automatic backup, or when I manually use the plugin to create a backup, the backup completes with no error.

Again, as noted in my troubleshooting, if I change the target location of the backup from a remote disk on my LAN to a local hard disk, the problem vanishes. But when I set it back to the remote disk again, the problem reappears.

I wouldn't say the problem is with the remote disk, per-se, since manual backups complete without error, and Calibre is storing its database and ePub files there. But something is off with regards to the automatic backup on a remote disk.

In that case it has to be a problem with copying the database file to the server. The copy is done using OS functions, so there isn't really anything that can be done. Unless the issue is timing and the server is still doing something when the check is started. If that is the case, then adding a delay between the copy and the check. It might be interesting toc o a separate check of one of the backups that were reported to be corrupt.