Annotations not working for some books (all kepubs on a Libra 2)

[Liudprand]

I have a problem with annotations on my Kobo Libra 2.

Initially, I found (I think after a recent firmware update) that annotations weren't working in any of my books - including the ones purchased from Kobo. I could highlight text, and even add a note - but if I clicked on the icon in the margin after adding a note, nothing would happen; and, similarly, if I went to my annotations using the icon at the bottom of the page, it would tell me I had no annotations in the book - even though, when I went back to the page, the text remained highlighted.

I spoke to Kobo support via chat, and they advised me to reset the device, which I did, and reinstalled everything from scratch.

Now I have the same probelm as before, but with a twist: many books will accept highlighting, including (so far) all of the purchased ones. But several of the side-loaded books I've converted to kepubs (using KoboTouchExtended in Calibre) are displaying the behaviour described above. In fact, if I highlight some text in one of these books that has this error, and then use the Annotations plugin to import annotations into Calibre from the device, these "ghost" annotations in fact do show up. But then, when I go back to the device, not only are those particular annotations not listed as annotations when you click on the icon - they've disappeared altogether: no highlighting in the text at all.

So, something weird is going on. I was wondering if it's something to do with the conversion to kepub - but I've tried it several times with one of the problematic books, and it's the same every time.

Any ideas?

[davidfor]

The initial issues sound like a corrupted database on the device. The factory reset would have fixed that. Though just signing out of the Kobo account and back in would have done it as well.

For the later problem with sideloaded kepubs could be an error in the book. The annotations rely on the ToC and if that is out of sync with the book, the annotations will not display properly. This could happen when the book is edited and the ToC not updated inside the book. Or if you update the ToCo and structure of the book and resend it to the device. In either case, the annotations will probably not be displayed in the book, but, should appear in the annotations list.

The other reason for annotations not appearing in kepubs is if the books have not been properly processed and do not have the Kobo specific spans. These are used for the locations within the book that are used by both annotations and the reading position. This should only happen if there are errors in the books. In some cases, the books will be sent as epubs instead. In other cases, the changes might only be partially done.

For either of these, the annotations should still be there when you eject the device. For them to disappear, usually means a corrupt database. But, the annotations plugin would probably have had errors if that was the case. The other reason for annotations to disappear is if the book is replace. The drivers can handle this, but, there is an option to turn off this handling and the device will import any replaced book as a new book. And that removes the annotations and reading status for the books.

To give a better idea, I would need to see the kepub version of the book. This will be a problem if the book is under copyright. If you have a book that is legal to post, that would be good. There is also the ScrambleEbook plugin that can be used to generate a postable version of the book. As th text is scrambled, that can make it hard to discuss where problems occur, but, it lets me see the book structure. It might also be useful to see a debug log from calibre when fetching the annotations. That might give some clues as to what is happening.

[Liudprand]

Thanks for this.

Kobo did recommend signing out and in again first, but it didn't have any effect, so I had to do a full reset (left that out of my original post for simplicity!).

I've attached a Project Gutenberg edition of Bleak House, which does have this problem.

I should maybe mention that, apart from using the KTExtended plugin to convert to kepub when I send books to the device, the only other thing I do with them is Polish them, with everything selected except 'Add/replace metadata as a "book jacket" page' and 'Add soft hyphens'. Otherwise, this was just downloaded from Gutenberg and then sent to my Libra 2 using KTE.

Thanks again...

[Liudprand]

One further detail: In a book that has this problem, if I highlight something (with or without adding a note) and then close the book and re-open it, the annotation has disappeared - i.e. in addition to not showing up if you click the "Annotations" icon, it no longer appears as highlighting on the page...

BUT - even after that, when everything seems to have disappeared - if I fetch annotations into Calibre from the device, the "disappeared" annotations are uploaded successfully! So, they're still "there" in some form, I guess...

[Liudprand]

Sorry to bombard this thread, davidfor, but I've just worked something out.

If I upload the original file to my Libra 2 in the normal way, but without polishing it, everything seems to work fine.

I guess I can mess around unticking options in there to discover which one is causing the problem - I guess either "Remove unused CSS rules..." or "Upgrade book internals" are the most likely candidates? It's odd, though, that it doesn't cause the same probelm with all of my books.

[Strether]

As a way around the problem, you might download Harry T's excellent version from the Mobileread library.

Jim

[Liudprand]

Thanks - but it's not just that book. I have 4,800 books on my Libra 2 (I know, I know... ) and it affects a lot of them. But it seems I may now have a workaround (see above)...

[Liudprand]

OK - I think I've solved the mystery. I've added the same book several times, proceeding in the same way each time except for the options selected under "Polish". I selectively omitted one of a few options I had suspicions about each time - and when I excluded "Upgrade book internals", it worked fine - no problem at all with annotations!

Hopefully might be useful for others...

[theducks]

Originally Posted by Liudprand

OK - I think I've solved the mystery. I've added the same book several times, proceeding in the same way each time except for the options selected under "Polish". I selectively omitted one of a few options I had suspicions about each time - and when I excluded "Upgrade book internals", it worked fine - no problem at all with annotations!

Hopefully might be useful for others...

Care to share what Polish option? (I see some books don't always look the same as Preview/Viewer, but the code appears the same as some that work)

[DNSB]

One possible issue with the copy of Bleak House is that epubcheck barfed on it. Not to mention Gutenberg's love for running chapters together and splitting files at random spots.

[JSWolf]

Originally Posted by Liudprand

Sorry to bombard this thread, davidfor, but I've just worked something out.

If I upload the original file to my Libra 2 in the normal way, but without polishing it, everything seems to work fine.

I guess I can mess around unticking options in there to discover which one is causing the problem - I guess either "Remove unused CSS rules..." or "Upgrade book internals" are the most likely candidates? It's odd, though, that it doesn't cause the same probelm with all of my books.

Is it just Remove unused CSS rules and Upgrade book internals that you have selected? If so, there's no need for Upgrade book internals. In this case, if it's ePub 2 there's no need to convert to ePub 3. There's no need to change whatever changes in an ePub 3.

[Liudprand]

Originally Posted by theducks

Care to share what Polish option? (I see some books don't always look the same as Preview/Viewer, but the code appears the same as some that work)

I don't think I understand the question. My message that you quote contains the answer to your question: i.e., if I untick "Upgrade book internals", the problem doesn't seem to arise.

I'm afraid the rest of what's in your message is too technical for me to parse!

[Liudprand]

Originally Posted by JSWolf

Is it just Remove unused CSS rules and Upgrade book internals that you have selected? If so, there's no need for Upgrade book internals. In this case, if it's ePub 2 there's no need to convert to ePub 3. There's no need to change whatever changes in an ePub 3.

If you read the initial post in this thread, that will answer your question. Originally I had EVERYTHING selected except:

"Add/replace metadata as a "book jacket" page"

and

"Add soft hyphens".

I still have EVERYTHING ELSE selected EXCEPT FOR:

"Upgrade book internals".

And - for some reason that's way beyond my pay-grade - that seems to have fixed the issue.

[Liudprand]

Originally Posted by DNSB

One possible issue with the copy of Bleak House is that epubcheck barfed on it. Not to mention Gutenberg's love for running chapters together and splitting files at random spots.

Interesting. As it turns out (as my recent posts make clear), it turns out that's not the problem - once I unselected "Upgrade book internals" in "Polish", it stopped being an issue.

[davidfor]

Originally Posted by Liudprand

Thanks for this.

Kobo did recommend signing out and in again first, but it didn't have any effect, so I had to do a full reset (left that out of my original post for simplicity!).

I've attached a Project Gutenberg edition of Bleak House, which does have this problem.

I should maybe mention that, apart from using the KTExtended plugin to convert to kepub when I send books to the device, the only other thing I do with them is Polish them, with everything selected except 'Add/replace metadata as a "book jacket" page' and 'Add soft hyphens'. Otherwise, this was just downloaded from Gutenberg and then sent to my Libra 2 using KTE.

I tried the book and I can see the same problem. Looking at it, I think it is caused by the internal file names. The first is "7061549018157950677_1023-h-0.htm.html". That, plus there a fragments in the ToC entries. I'm pretty sure that is breaking the annotations handling. I have seen something like this before, but, I can't remember the details.

Can you point me to the original? I should be able to find it, but, there are always multiple versions and I want to make sure I get the right one. I'm curious about what changes the upgrade to ePub 3 made to the file. But, doing that is not likely to add anything to the files. It would only be useful if you were using an ereader that did not support ePub 2.