[GUI Plugin] Annotations

[davidfor]

Originally Posted by kawls

Thank you for puzzling this out with me. The problem seems simpler to communicate now that I have the terminology. This is the problem: annotations embedded in an epub's contents.opf file (in <meta name="calibre:user_metadata:#mm_annotations">) are not being passed into the Kobo device's sqlite database when the epub is sideloaded to it from Calibre. Other metadata like the book's title, author, etc. are passed into the database, but annotations metadata are not.

.
.
.

Now that I know of your Kobo Utilities plugin that backs-up the Kobo database, I seemingly won't get in this situation again. But I am in it at present, alas. Any ideas why the annotations aren't being passed into the Kobo database?

The plugin is one way only. It fetches the annotations from the device and stores them as HTML in a column in the calibre library. There is absolutely no attempt to pass them back to the device. And the plugin does not store enough details for the annotation to be restored to a device.

Plus, the Kobo devices only reads Dublin Core metadata from sideloaded books, and even then, not all of them. Annotations, do not have any sort of standard for storage (or at least not one that I have seen and seen implemented). Hence there is no way to add them to the book that the device would read.

To be clear, there is absolutely nothing in the plugin or the thread that suggests annotations can be restored to the device. It is a common question it this thread and elsewhere but the answer is always the same: NO. This might change after the release for calibre version 5. That includes annotation support within the viewer and the calibre database. But, the key word here is "after" with no other suggestion of when, or even a promise that it will be done.

[droopy]

Originally Posted by davidfor

The modified timestamp is updated. So, you can sort by that. But, you asked if you could sort by the timestamps within the annotations. That you cannot do.

I searched for #mm_annotations (I dunno why there's the "mm_" part in my annotations field), and I see just my books with annotations.

The problem is that the "Modified" date for all books is July 31 2020, the date I synced the annotations when my Kobo was connected to Calibre, even if July 31 is not the date I made the latest annotation on books X, Y or Z.

[davidfor]

Originally Posted by droopy

I searched for #mm_annotations (I dunno why there's the "mm_" part in my annotations field), and I see just my books with annotations.

Sorry, I can't help there as you are the person who created the column and gave it that name.

The problem is that the "Modified" date for all books is July 31 2020, the date I synced the annotations when my Kobo was connected to Calibre, even if July 31 is not the date I made the latest annotation on books X, Y or Z.

The modified date is changed by calibre whenever you change a book or something related to it. Renaming a tag will change the modified date for all books that have the tag. If all books have the same modified timestamp, it probably means you changed a custom column. That will trigger an update for all books when calibre restarts which means the modified timestamp changes for all books at that time.

There isn't much that can be done here. The modified timestamp is associated with changes to the books. There is no way to know if that was a fetch of the annotations or some other edit. And there is no way to block the timestamp from being updated as you make changes. Anything else that could be done is complicating things for a small number of users.

[droopy]

Until today, I had no problems syncing my annotations from my Kobo to my Calibre.
But today, it now says

"Unable to import annotations as the annotations column does not exist."
As the screenshots show, it does appear that the annotations column does in fact exist.

Can anyone please help?

[DNSB]

Originally Posted by droopy

Until today, I had no problems syncing my annotations from my Kobo to my Calibre.
But today, it now says

"Unable to import annotations as the annotations column does not exist."
As the screenshots show, it does appear that the annotations column does in fact exist.

Can anyone please help?

Try using #annotations as the lookup name.

To quote from the calibre documentation: "Names for custom fields (columns you have created yourself) always have a # as the first character."

The number of custom column names that have been posted starting with a # were not just for looks.

[davidfor]

Originally Posted by droopy

Until today, I had no problems syncing my annotations from my Kobo to my Calibre.
But today, it now says

"Unable to import annotations as the annotations column does not exist."
As the screenshots show, it does appear that the annotations column does in fact exist.

Can anyone please help?

The first screenshot is the wizard to create the column. Did you restart calibre after creating the column? If you, the important screenshot is what is in the customization dialog of the plugin. Is the correct column selected there?

If that is correct, then please run calibre in debug mode, try the fetch and post the log after it fails. And turn the debug option in the plugin on before doing this.

[droopy]

Originally Posted by davidfor

The first screenshot is the wizard to create the column. Did you restart calibre after creating the column? If you, the important screenshot is what is in the customization dialog of the plugin. Is the correct column selected there?

If that is correct, then please run calibre in debug mode, try the fetch and post the log after it fails. And turn the debug option in the plugin on before doing this.

Hi DavidFor.
Thanks for replying.
I enabled debug mode in the Annotations plugin and then restarted Calibre in Debug Mode.
I then ran “Fetch Annotations from Device” and now it’s successfully fetching the annotations. Strange. but a happy result.

[DNSB]

Originally Posted by droopy

Hi DavidFor.
Thanks for replying.
I enabled debug mode in the Annotations plugin and then restarted Calibre in Debug Mode.
I then ran “Fetch Annotations from Device” and now it’s successfully fetching the annotations. Strange. but a happy result.

We don't believe in miracles, we just rely on them.

[OrinocoDream]

Hi! I love that this tool exists. Is there a github for it somewhere?

One book whose annotations I want to read is not appearing in the list when I "Fetch annotations from connected device". It's not green/yellow/red, it's just missing altogether.

I notice if I "send to device" from Calibre, then add a note on that new copy, then the book does show up in the annotation plugin's list. However, I want the highlights/notes from my original version of the book. By the way, prior to clicking "send to device", I did update the metadata using Calibre's "Edit Metadata > Download Metadata" function. I was trying make the book more recognizeable.

I tried searching for how to update metadata for a title on the kindle itself. People suggest deleting the book and re-uploading it. Of course, if I do that I lose the notes.

Anyone know how to fix this? I do see entries for the title in the debug log,

DEBUG:   15.0  resolve_paths. id=38, path=/<storage>/Terner, Janet R. & Pew, W. L. & Aird, Robert A_/Courage to Be Imperfect_ The Life and Work of Rudolf Dreikurs, The - Janet R. Terner & W. L. Pew & Robert A. Aird.bookmark
DEBUG:   15.0  resolve_paths. fmt=epub
DEBUG:   15.0  resolve_paths. looking for book on device: book_path=/media/rob/Kindle/documents/Terner, Janet R. & Pew, W. L. & Aird, Robert A_/Courage to Be Imperfect_ The Life and Work of Rudolf Dreikurs, The - Janet R. Terner & W. L. Pew & Robert A. Aird.bookmark
DEBUG:   15.0  resolve_paths. id=39, path=/<storage>/1897-1972, Dreikurs, Rudolf,/Challenge of Parenthood (1979), The - Dreikurs, Rudolf, 1897-1972.bookmark
DEBUG:   15.0  resolve_paths. fmt=epub

Thanks for any help!

[JSWolf]

Is this going to be updated to work with Calibre 5?

[ownedbycats]

I think davidfor mentioned in the Kobo Utilities thread wanting to update this for Calibre 5 and possibly implement annotation syncing. The latter on sounded like a long-term thing though if it's even doable.

[davidfor]

I have most of it ported to calibre 5. I got stuck on one change and haven't had time to fix it and maintain compatibility with older versions of calibre. I'll hopefully do this in the next few days.

As to importing to the calibre annotations system, that's a completely separate thing and I haven't started looking at it.

[bopuc]

@davidfor
Excited for the future possibilities of these changes and just want to say I am rooting for you. Thank you for keeping Annotations alive! ����

[notonyourtintype]

As a newbie to this forum and this plugin... I installed it on my Calibre (Windows 64bit), it imported annotations just fine from my Kobo Clara.

Now... I'd like to be able to read the book on the Windows PC with the annotations in place and on screen, rather than only see the annotations on the Kobo itself. But I think I'm missing something.

help?

[milady133]

Originally Posted by notonyourtintype

Now... I'd like to be able to read the book on the Windows PC with the annotations in place and on screen, rather than only see the annotations on the Kobo itself. But I think I'm missing something.

That's not how the plugin works. And it's not possible to do that. There's not a standard way to treat annotations in EPUB, so each program treats annotations differently, making it impossible to create a tool to import annotations from different programs.
If you read EPUB (not kepubs) in your Kobo, maybe the annotations are compatible with how Adobe treats the annotations in the PC program, maybe someone can point if it's possible