Libra 2 All bookmarks, highlights, annotations vanished inexplicably :(

[ogdredweary]

Kobo Libra 2, version 4.33.19611, side-loaded books from Calibre

I'm new to Kobo, and from the start I had the related problem of bookmarks, highlights, and annotations simply not saving. Somehow, I got it all working properly, maybe it was the device reset—or the second device reset.

Last night, I was reading normally and came across another bit I wanted to highlight to find again later. I had to increase the font size in order to get it all on one page to highlight it, and when I decreased the font back to my preferred size, the new highlight vanished. The previous highlight was also gone. I checked the annotations list and all my bookmarks, highlights, and annotations in this book were gone.

When checking the system files on the Kobo, I found that KoboReader.sqlite, Analytics.conf, Kobo eReader.conf, and BookReader.sqlite all had the same created, modified, and last open date which read 'Tomorrow, 3:44'. Bizarre. After rebooting the Kobo, two of the files self corrected, but the other two were still time travelling about 4 hours into the future.

Kobo support basically shrugged and said that if annotations are lost there's no way to recover them. They seemed uninterested in the problem or the details I shared.

Frustratingly, the book in question will now not save any new highlights, bookmarks, or annotations—but other books will. This would seem to suggest it's an issue with this particular book, except for the weirdness with the system files.

Following some initial search results, I spent some time installing KoReader onto the device and Kobo Utilities in Calibre. I would prefer to use Kobo's native reader, but I would also like for my notes and bookmarks to be reliably saved and backed up (preferably wirelessly). Is there a way to do this that I haven't found yet? I'm resigned to the fact that all my hours of notes are gone, but is there anything else I could try to recover them?

I really love reading on Kobo Libra 2. The design is wonderfully ergonomic for my hands. It's really disappointing that it's been so buggy.

[JSWolf]

Are you reading ePub or KePub. I just tested this with the eBook I am reading in ePub and the highlight had no problems when I change the text size. I have not tested this with KePub.

[ogdredweary]

Originally Posted by JSWolf

Are you reading ePub or KePub. I just tested this with the eBook I am reading in ePub and the highlight had no problems when I change the text size. I have not tested this with KePub.

I knew I was forgetting something: ePubs.

This was new to me, too, as I've been using the font resize quite a lot to highlight across pages.

[davidfor]

Firstly, the timestamp on the files will not be an issue. You mention a difference of 4 hours. The file timestamp will be UTC, your timezone will be four hours away from that. Though exactly what you will see depends on how you are looking at it and the relative timezones involved.

Annotations/highlights in epubs should not be affected by font size. They do not support highlighting over a screen/page break. So, you do need to change the font size to do that. If you use kepub, you can highlight across screens.

There shouldn't be any issues with annotations in epubs. I use them all the time to mark errors or things I want to look up later. There can be problems if the database has issues. But, you have tested that. The only other thing I can think of is if there are errors in the epub. Or the ToC has changed. The latter can happen if you replace the epub using calibre.

[ogdredweary]

Originally Posted by davidfor

Firstly, the timestamp on the files will not be an issue. You mention a difference of 4 hours. The file timestamp will be UTC, your timezone will be four hours away from that. Though exactly what you will see depends on how you are looking at it and the relative timezones involved.

Even so, this doesn't explain why two of the files in question reverted to a time in the past after restarting the device while two of them remained in the future.

Annotations/highlights in epubs should not be affected by font size. They do not support highlighting over a screen/page break. So, you do need to change the font size to do that. If you use kepub, you can highlight across screens.

They shouldn't be, no. And I have been doing it throughout this book with no issues until now.

There shouldn't be any issues with annotations in epubs. I use them all the time to mark errors or things I want to look up later. There can be problems if the database has issues. But, you have tested that. The only other thing I can think of is if there are errors in the epub. Or the ToC has changed. The latter can happen if you replace the epub using calibre.

Right, I don't know what could have happened in this one instance of saving a highlight that resulted in all highlights, annotations, and bookmarks being deleted—and now no new notes being saved. If it was a problem with the book, then wouldn't it have had the problem right at the start? I mean, it did, but I managed to get them working somehow (not sure which of my troubleshooting steps fixed it). Why now that I'm halfway through?

I keep my Kobo offline most of the time, so it wasn't an update or anything like that. I'm utterly baffled and not sure how to proceed. I haven't read anything since this incident because I can't save any annotations, and I'm afraid of losing more notes in other books.

[JSWolf]

Originally Posted by ogdredweary

Right, I don't know what could have happened in this one instance of saving a highlight that resulted in all highlights, annotations, and bookmarks being deleted—and now no new notes being saved. If it was a problem with the book, then wouldn't it have had the problem right at the start? I mean, it did, but I managed to get them working somehow (not sure which of my troubleshooting steps fixed it). Why now that I'm halfway through?

I keep my Kobo offline most of the time, so it wasn't an update or anything like that. I'm utterly baffled and not sure how to proceed. I haven't read anything since this incident because I can't save any annotations, and I'm afraid of losing more notes in other books.

Run Calibre if you do not have Kobo Utilities installed, install it. It's a plugin. It will allow you to plug in your Kobo and check if the database is corrupt or not.

[ogdredweary]

Originally Posted by JSWolf

Run Calibre if you do not have Kobo Utilities installed, install it. It's a plugin. It will allow you to plug in your Kobo and check if the database is corrupt or not.

I had already installed Kobo Utilities, but I didn't know it could do this. Just ran the check, and the database is, in fact, corrupted (details below). I don't see a way to repair the database. Will I have to reset the device again?

calibre, version 5.44.0
ERROR: Unhandled exception: <b>CorruptError</b>:CorruptError: database disk image is malformed

calibre 5.44 embedded-python: True is64bit: True
macOS-10.16-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '21.5.0', 'Darwin Kernel Version 21.5.0: Tue Apr 26 21:08:22 PDT 2022; root:xnu-8020.121.3~4/RELEASE_X86_64')
Python 3.8.5
OSX: ('10.16', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (1, 48, 0) && DeDRM (10, 0, 2) && Package KFX (from KFX Input) (1, 48, 0) && Annotations (1, 17, 0) && Find Duplicates (1, 9, 4) && KFX metadata reader (from KFX Input) (1, 48, 0) && KFX Input (1, 48, 0) && Kobo Utilities (2, 16, 3)
Traceback (most recent call last):
File "calibre_plugins.koboutilities.action", line 1757, in check_device_database
check_result = self._check_device_database()
File "calibre_plugins.koboutilities.action", line 3738, in _check_device_database
return check_device_database(self.device_database_path())
File "calibre_plugins.koboutilities.action", line 6099, in check_device_database
result = cursor.fetchall()
apsw.CorruptError: CorruptError: database disk image is malformed

[DNSB]

Originally Posted by ogdredweary

I had already installed Kobo Utilities, but I didn't know it could do this. Just ran the check, and the database is, in fact, corrupted (details below). I don't see a way to repair the database. Will I have to reset the device again?

A signout/signin will replace the database and should not require you to reload any sideloaded books though all reading status, annotations, etc. will be lost. Using Kobo Utilities to back up the database will help to reduce any future losses to a minimal level.

You can try using an SQLite utility to repair the database using vacuum and/or re-index won't hurt and might help but the chances are pretty slim.

[davidfor]

Originally Posted by ogdredweary

Even so, this doesn't explain why two of the files in question reverted to a time in the past after restarting the device while two of them remained in the future.

Yes it could. It will depend on what needed to be updated, and when. And how you are looking at the timestamps. What I reported is what I know happens. It comes from spending a lot of time looking at the files on these devices from Windows or using telnet to access the device.

Or are you saying that the time difference does not agree with your timezone difference to UTC?

They shouldn't be, no. And I have been doing it throughout this book with no issues until now.



Right, I don't know what could have happened in this one instance of saving a highlight that resulted in all highlights, annotations, and bookmarks being deleted—and now no new notes being saved. If it was a problem with the book, then wouldn't it have had the problem right at the start? I mean, it did, but I managed to get them working somehow (not sure which of my troubleshooting steps fixed it). Why now that I'm halfway through?

Simple, because the error that triggered the problem only happened half-way through the book. I gave several reasons for the problem. The database becoming corrupted would cause this to start. And The exact way in which the database is corrupted will give different results. If there is an error in the book, it might be fine in the first half, but, not in the latter. If you replaced the book to fix something after starting to read it, that might be when the error started. If you go back to earlier in the book, and try to annotate, you might find you can't make new ones. Or if they do work, it suggests an error in the second half of the book.

I keep my Kobo offline most of the time, so it wasn't an update or anything like that. I'm utterly baffled and not sure how to proceed. I haven't read anything since this incident because I can't save any annotations, and I'm afraid of losing more notes in other books.

Take a copy of the database to backup everything up if you are concerned. If you lose more you can restore that backup or do some work to extract the annotations. Or, backup the contents of "Digital Edition/annot". That has a copy of all annotations made in ePubs and PDFs. They can be restored though you need to open the book to trigger rereading the files. I don't know if this will contain the missing annotations. That will depend on what the real issue is.

[mikeana]

[QUOTE=davidfor;4231719]Firstly, the timestamp on the files will not be an issue. You mention a difference of 4 hours. The file timestamp will be UTC, your timezone will be four hours away from that. Though exactly what you will see depends on how you are looking at it and the relative timezones involved.

I have had this problem with my annotations dating 6 hours ahead at 6:01pm saying tomorrow, and spent hours trying to figure out what I was doing wrong.
Why would Kobo do it that way? It doesn't seem practical or logical. Kobo should fix this.