[Kobo] Malformed database...

Liudprand · 03-08-2022, 10:11 PM

Hi.

I'm getting the following error when Calibre tries to back up the database on my Kobo Libra 2:

calibre, version 5.38.0 (win32, embedded-python: True)
Failed to backup device database: Failed: Backing up Kobo device database

CorruptError: database disk image is malformed

Traceback (most recent call last):
File "calibre\gui2\device.py", line 87, in run
File "calibre_plugins.koboutilities.jobs", line 197, in do_device_database_backup
File "calibre_plugins.koboutilities.action", line 6095, in check_device_database
apsw.CorruptError: CorruptError: database disk image is malformed

Presumably, since this backup process happens every time I connect the device, I can just restore the most recent (non-corrupt) backup... ? If so, could someone let me know how to do that? The only instructions I can find are to select "Export/import calibre data" from the "Calibre Library" toolbar button - but that button (as opposed to the "Library" button) doesn't appear on the toolbar, and I can't work out how to add it. But, in any case, I'm thinking maybe the organisation of the toolbar buttons has been tweaked since this instruction was composed(?).

Liudprand · 03-08-2022, 10:29 PM

Just to clarify: the references to this kind of issue that I've found on this forum and elsewhere seem to suggest that you have to "sign out" from the device, thus losing, and then having to restore, all side-loaded books.

BUT: I've noticed for a while that the database backup task takes place every time I plug the device in; so I'm hoping I can restore a previous backup - maybe missing just the last few books I uploaded to the device, which I can then re-send to it. Does that make sense? If not, then I don't think I understand what the "Backing up database" task actually does.

ownedbycats · 03-09-2022, 12:44 AM

Restoring the last non-corrupt backup should work; sign-out/sign-in will also work but will wipe your existing bookmarks and annotations and stuff.

No option in Calibre/Kobo Utilities; iirc that's a deliberate choice by the plugin author. You need to unzip your last good backup and then overwrite the files on the drive (look in the .kobo folder, I think it is) and then eject. Reboot the device for good measure.

If you're using 4.31 firmware, head over to the Kobo forum; there's been reports of it being a bit more prone to corrupt the db -- I've experienced it myself. If you don't need wifi or sync, reverting to 4.30 is more stable.

davidfor · 03-09-2022, 01:59 AM

Yes, just copy the "KoboReader.sqlite" file to the ".kobo" directory on the device. It will replace the existing one. Once you disconnect, I recommend shutting down to make sure the database is reread properly.

And it was a deliberate choice not make it a manual job. Apart from not wanting to write the code for something that would hopefully never be used, I also wanted it to be a deliberate action by the user.

Signing out of the device is the official way to fix problems like this. It would recreate the database and then import your sideloaded books and download the purchased books. But, you lose the reading status and annotations for sideloaded books, and reading stats overall. If you have a recent good database backup, restoring it usually works.

Liudprand · 03-09-2022, 04:49 AM

Quote:

Originally Posted by davidfor

Yes, just copy the "KoboReader.sqlite" file to the ".kobo" directory on the device. It will replace the existing one. Once you disconnect, I recommend shutting down to make sure the database is reread properly.

And it was a deliberate choice not make it a manual job. Apart from not wanting to write the code for something that would hopefully never be used, I also wanted it to be a deliberate action by the user.

Signing out of the device is the official way to fix problems like this. It would recreate the database and then import your sideloaded books and download the purchased books. But, you lose the reading status and annotations for sideloaded books, and reading stats overall. If you have a recent good database backup, restoring it usually works.

When I search my system, the most recently modified version of "KoboReader.sqlite" is from 10 November last year.

This is confusing, because, as I say, the database backs up (or, at least, that task appears to be performed) every time I connect the device. Is it possible that the latest backup is stored in a different file somewhere?

davidfor · 03-09-2022, 04:54 AM

Quote:

Originally Posted by Liudprand

When I search my system, the most recently modified version of "KoboReader.sqlite" is from 10 November last year.

This is confusing, because, as I say, the database backs up (or, at least, that task appears to be performed) every time I connect the device. Is it possible that the latest backup is stored in a different file somewhere?

Look at the configuration of the plugin. That has a directory for the backups. That is where they copied. If there is no directory specified, the backup is not done.

Liudprand · 03-09-2022, 05:08 AM

Ah, okay - thanks. Just FYI, the file name is in the form: "KoboReader-KoboLibra2-N418180043829-20220308-002515.sqlite" - that's why my file system couldn't find it.

I'll give this a go and report back...

Liudprand · 03-09-2022, 05:15 AM

In the .kobo folder on the device, there are already two .sqlite files:

BookReader.sqlite

KoboReader.sqlite

- plus BookReader.sqlite-shm and BookReader.sqlite-wal

Do I need to remove any of these files, and/or rename the backup file to match one of them?

JSWolf · 03-09-2022, 05:23 AM

If you are running 4.31, you should go back to 4.30 and do not sync or you will get 4.31 back.

Liudprand · 03-09-2022, 05:39 AM

Quote:

Originally Posted by JSWolf

If you are running 4.31, you should go back to 4.30 and do not sync or you will get 4.31 back.

But I need to sync regularly to update Pocket articles... ?

ownedbycats · 03-09-2022, 05:44 AM

Yes, if you need to sync or use wifi you're stuck with 4.31. Keep frequent backups is the main suggestion I have.

Liudprand · 03-09-2022, 05:54 AM

Quote:

Originally Posted by Liudprand

In the .kobo folder on the device, there are already two .sqlite files:

BookReader.sqlite

KoboReader.sqlite

- plus BookReader.sqlite-shm and BookReader.sqlite-wal

Do I need to remove any of these files, and/or rename the backup file to match one of them?

OK - I deleted the KoboReader.sqlite and renamed the backup file with that name - and it seems to have worked.

Thanks for your help.

JSWolf · 03-09-2022, 06:20 AM

Quote:

Originally Posted by Liudprand

But I need to sync regularly to update Pocket articles... ?

What you might be able to do is to install 4.30 and in your home router, block the Kobo update server. I don't know the address of this, but others on the forum do know. I'm hoping this will work.

Also, use Kobo Utilities with Calibre to backup your database every time you connect to Calibre. Then if something ever happens, you'll have a database you can restore.

ownedbycats · 03-09-2022, 06:53 AM

Quote:

Originally Posted by JSWolf

What you might be able to do is to install 4.30 and in your home router, block the Kobo update server. I don't know the address of this, but others on the forum do know. I'm hoping this will work.

Also, use Kobo Utilities with Calibre to backup your database every time you connect to Calibre. Then if something ever happens, you'll have a database you can restore.

Wouldn't blocking Kobo servers also break the Pocket sync?

davidfor · 03-09-2022, 08:11 AM

Quote:

Originally Posted by Liudprand

Ah, okay - thanks. Just FYI, the file name is in the form: "KoboReader-KoboLibra2-N418180043829-20220308-002515.sqlite" - that's why my file system couldn't find it.

I'll give this a go and report back...

The file name is changed for people with multiple devices (model and serial number) and then the time stamp added to allow multiple backups.

Quote:

Originally Posted by Liudprand

In the .kobo folder on the device, there are already two .sqlite files:

BookReader.sqlite

KoboReader.sqlite

- plus BookReader.sqlite-shm and BookReader.sqlite-wal

Do I need to remove any of these files, and/or rename the backup file to match one of them?

As you have worked out, replace the "KoboReader.sqlite". But, you should also remove the other "KoboReader.sqlite*" files. Leaving them shouldn't harm, but, it is safer.

The "BookReader.sqlite" is used for Kobo Plus subscription books and is encrypted. If you do not subscribe to Kobo Plus, it should be something like 500KB.

As to rolling back to 4.30, if you are syncing to Kobo frequently, you won't want to. You can sync and if an update is downloaded, press the "later" button. Then connect to the PC and delete the update files. It's a nuisance, but it can be done. I thought there was a patch to block it, but, it isn't being mentioned, so I'm probably wrong.

And I don't think the issue is anywhere near as bad as @JSWolf makes out. I can't reliably reproduce it. I can go days without issues with multiple connections a day with no issues. And I have spent a reasonable amount of time working with a Kobo developer trying to reproduce the problem consistently. That I said "trying" means we weren't able to. I hope he has some ideas, but, it isn't obvious. Or affecting everyone.

03-08-2022, 10:11 PM	#1
Liudprand Zealot Posts: 133 Karma: 10 Join Date: Nov 2021 Device: Kobo Libra 2	[Kobo] Malformed database... Hi. I'm getting the following error when Calibre tries to back up the database on my Kobo Libra 2: calibre, version 5.38.0 (win32, embedded-python: True) Failed to backup device database: Failed: Backing up Kobo device database CorruptError: database disk image is malformed Traceback (most recent call last): File "calibre\gui2\device.py", line 87, in run File "calibre_plugins.koboutilities.jobs", line 197, in do_device_database_backup File "calibre_plugins.koboutilities.action", line 6095, in check_device_database apsw.CorruptError: CorruptError: database disk image is malformed Presumably, since this backup process happens every time I connect the device, I can just restore the most recent (non-corrupt) backup... ? If so, could someone let me know how to do that? The only instructions I can find are to select "Export/import calibre data" from the "Calibre Library" toolbar button - but that button (as opposed to the "Library" button) doesn't appear on the toolbar, and I can't work out how to add it. But, in any case, I'm thinking maybe the organisation of the toolbar buttons has been tweaked since this instruction was composed(?).

03-09-2022, 12:44 AM	#3
ownedbycats Custom User Title Posts: 10,973 Karma: 75337983 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	Restoring the last non-corrupt backup should work; sign-out/sign-in will also work but will wipe your existing bookmarks and annotations and stuff. No option in Calibre/Kobo Utilities; iirc that's a deliberate choice by the plugin author. You need to unzip your last good backup and then overwrite the files on the drive (look in the .kobo folder, I think it is) and then eject. Reboot the device for good measure. If you're using 4.31 firmware, head over to the Kobo forum; there's been reports of it being a bit more prone to corrupt the db -- I've experienced it myself. If you don't need wifi or sync, reverting to 4.30 is more stable. Last edited by ownedbycats; 03-09-2022 at 12:55 AM.

03-09-2022, 01:59 AM	#4
davidfor Grand Sorcerer Posts: 24,905 Karma: 47303824 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	Yes, just copy the "KoboReader.sqlite" file to the ".kobo" directory on the device. It will replace the existing one. Once you disconnect, I recommend shutting down to make sure the database is reread properly. And it was a deliberate choice not make it a manual job. Apart from not wanting to write the code for something that would hopefully never be used, I also wanted it to be a deliberate action by the user. Signing out of the device is the official way to fix problems like this. It would recreate the database and then import your sideloaded books and download the purchased books. But, you lose the reading status and annotations for sideloaded books, and reading stats overall. If you have a recent good database backup, restoring it usually works. Last edited by davidfor; 03-09-2022 at 02:02 AM.

03-09-2022, 05:23 AM	#9
JSWolf Resident Curmudgeon Posts: 79,750 Karma: 145864619 Join Date: Nov 2006 Location: Roslindale, Massachusetts Device: Kobo Libra 2, Kobo Aura H2O, PRS-650, PRS-T1, nook STR, PW3	If you are running 4.31, you should go back to 4.30 and do not sync or you will get 4.31 back. Last edited by JSWolf; 03-09-2022 at 05:34 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[Kobo] database disk image is malformed	franklekens	Devices	30	11-19-2021 12:27 PM
Malformed database and saving Kobo stats	blar	Kobo Reader	1	05-23-2021 12:44 AM
Error in Kobo Glo: database disk image is malformed	Carpidor	Devices	2	08-07-2015 07:17 AM
Kobo error: database disk image is malformed	RSaunders	Devices	7	07-28-2011 09:52 PM
help: 'malformed database' with kobo	prairiegal	Devices	2	03-10-2011 09:50 PM

03-08-2022, 10:29 PM	#2
Liudprand Zealot Posts: 133 Karma: 10 Join Date: Nov 2021 Device: Kobo Libra 2	Just to clarify: the references to this kind of issue that I've found on this forum and elsewhere seem to suggest that you have to "sign out" from the device, thus losing, and then having to restore, all side-loaded books. BUT: I've noticed for a while that the database backup task takes place every time I plug the device in; so I'm hoping I can restore a previous backup - maybe missing just the last few books I uploaded to the device, which I can then re-send to it. Does that make sense? If not, then I don't think I understand what the "Backing up database" task actually does.

03-09-2022, 05:08 AM	#7
Liudprand Zealot Posts: 133 Karma: 10 Join Date: Nov 2021 Device: Kobo Libra 2	Ah, okay - thanks. Just FYI, the file name is in the form: "KoboReader-KoboLibra2-N418180043829-20220308-002515.sqlite" - that's why my file system couldn't find it. I'll give this a go and report back...

03-09-2022, 05:15 AM	#8
Liudprand Zealot Posts: 133 Karma: 10 Join Date: Nov 2021 Device: Kobo Libra 2	In the .kobo folder on the device, there are already two .sqlite files: BookReader.sqlite KoboReader.sqlite - plus BookReader.sqlite-shm and BookReader.sqlite-wal Do I need to remove any of these files, and/or rename the backup file to match one of them?

03-09-2022, 05:44 AM	#11
ownedbycats Custom User Title Posts: 10,973 Karma: 75337983 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	Yes, if you need to sync or use wifi you're stuck with 4.31. Keep frequent backups is the main suggestion I have.

Advert

Advert