[GUI Plugin] Annotations

[davidfor]

Originally Posted by Terisa de morgan

Well, I have a question about relation between plugins. Would it be posible to call the Annotation plugins from other plugin (namely Kobo Utility). I usually lose notes because I don't remember to run the plugin after finishing the book before removing it. Before trying to analyze this plugin (as I've never checked the code), I wanted to ask davidfor's opinion about it (opinion, hands to develop it will be mine )

Originally Posted by Terisa de morgan

Or another option would be to start it automatically after connecting the device. I don't know, just not forgetting it

I have thought of adding an automatic start, but there is one little problem with this. The annotations import requires one book to be selected. This is used with the Kindles and the other import methods as there the plugin does the matching between the books the annotations are for and the library. For Kindles, the annotations come from the clippings file. This has the title and author, but no direct reference to the book. The plugin does the matching to the library and can prompt to accept the match based on a confidence level for the match. If the plugin can't find a match, it suggests the selected book as the destination for the annotations. This was something the original author of the plugin did and I've never known if it was useful or not.

For the Kobo devices, the selected book won't be used. The plugin only fetches annotations for books that are already matched, and it uses this link to get the right book. Removing this for Kobo devices should be safe.

Calling the plugin from another plugin isn't hard. I call a function in the Goodreads sync from Kobo Utilities. Calling the annotations should be OK as well. But, an automatic fetch of the annotations is probably better. A problem with doing this is that the annotations fetch runs in the foreground. If it was a background job, it would be less intrusive especially when no annotations are found.

[Terisa de morgan]

Thank you, I was thinking in adding a new checkbox when you read the percentage, and extract annotations only for that book. I'll try to go with that idea because I think a job is less useful (if I haven't open the book I cannot have added any annotation). I'll tell you if I'm able to do it (another customization )

[schnesim]

Originally Posted by Terisa de morgan

Well, I have a question about relation between plugins. Would it be posible to call the Annotation plugins from other plugin (namely Kobo Utility). I usually lose notes because I don't remember to run the plugin after finishing the book before removing it. Before trying to analyze this plugin (as I've never checked the code), I wanted to ask davidfor's opinion about it (opinion, hands to develop it will be mine )

According to this thread calling a plugin from another plugin should be possible:

https://www.mobileread.com/forums/sh...ad.php?t=54761

[Terisa de morgan]

It's possible but depending on how it's coded, it's useful or not.

[Terisa de morgan]

Originally Posted by davidfor

For the Kobo devices, the selected book won't be used. The plugin only fetches annotations for books that are already matched, and it uses this link to get the right book. Removing this for Kobo devices should be safe.

It's safe till now

Originally Posted by davidfor

Calling the plugin from another plugin isn't hard. I call a function in the Goodreads sync from Kobo Utilities. Calling the annotations should be OK as well. But, an automatic fetch of the annotations is probably better. A problem with doing this is that the annotations fetch runs in the foreground. If it was a background job, it would be less intrusive especially when no annotations are found.

Thank you, I've already done it. Some minor modifications at this plugin for getting confidence but it already makes its job. I've to check but I think there is a problem if you want to change the css and apply it to the existing notes, but I have to debug it a bit more (now I get an exception).

But I have my foreground check for annotations, so I'm good to go. Thank you to the original creator and the current maintainer (and I have to finish the translation).

[dsestero]

First of all I have updated the italian translation that presented some minor errors and omissions; it is attached.

Then I would like to share what I understood until now about why only some annotated books are shown when I select ‘Fetch annotations from device’ (not much really).
The problem may be related to the matching of books in the Calibre library and on device but rather to the format/title of the books. In fact, if an annotated book does not match I expect that it will be shown in red and require manual match but not that it is not shown at all.
To check this hypothesis I made different trials sending the same book to the device from Calibre in different formats. I discovered that books in the new Amazon KFX format are not considered in the list of annotated books. Unfortunately, this is the format that most books are nowadays downloaded to the device from Amazon and it is very convenient too because it supports typographic enhancements, better speed and better navigation.
Furthermore, but here I have to investigate more, it seems that AZW3 books sent to the kindle from Calibre are recognized while AZW3 books downloaded from Amazon not or not always. MOBI seems OK.
Finally, other things that I still don't understand occur, like this:
I send a book in KFX to Kindle, annotate it and cannot see any annotation. I remove that book from kindle, I send the book in AZW3 and still cannot see any annotation. I add another annotation to the book and when fetch annotations finally I can see it with 2 annotations, but then when I preview them nothing is shown.

I still have not found a rule to understand which annotated book are shown and which not.

Well I wished to share these first findings, I will investigate more and will report but any suggestion is welcome!

Ciao,
Dario

[schnesim]

Originally Posted by dsestero

I still have not found a rule to understand which annotated book are shown and which not.

Did you try running the plugin with debug messages on? Would be interesting to see, if you see the same error messages which I posted.

[dsestero]

Yes, it is attached to post #356. Actually I got several messages like yours 'Title not found in books on device' for annotated books. Now I would like to understand what is the rule by which an annotated book is fetched or not by the plugin. It is not only a question of whether the book is matched to the library.

Ciao,
Dario

[schnesim]

Maybe I find the time at the weekend to add a bunch of more log messages to figure out, what's actually going wrong.

[glire]

So, the pluguin (v.1.6.0) stopped working suddenly. It fetch the books but when I try to import the annotations I get this error message:

Traceback (most recent call last):
File "calibre_plugins.annotations.action", line 293, in fetch_device_annotations
File "calibre_plugins.annotations.action", line 1012, in process_selected_books
File "calibre_plugins.annotations.action", line 150, in add_annotations_to_calibre
File "calibre_plugins.annotations.annotations", line 251, in merge_annotations
File "calibre_plugins.annotations.annotations_db", line 241, in capture_content
TypeError: 'NoneType' object has no attribute '__getitem__'

I tried using the last beta but the error continue appering.

[davidfor]

Originally Posted by glire

So, the pluguin (v.1.6.0) stopped working suddenly. It fetch the books but when I try to import the annotations I get this error message:

Traceback (most recent call last):
File "calibre_plugins.annotations.action", line 293, in fetch_device_annotations
File "calibre_plugins.annotations.action", line 1012, in process_selected_books
File "calibre_plugins.annotations.action", line 150, in add_annotations_to_calibre
File "calibre_plugins.annotations.annotations", line 251, in merge_annotations
File "calibre_plugins.annotations.annotations_db", line 241, in capture_content
TypeError: 'NoneType' object has no attribute '__getitem__'

I tried using the last beta but the error continue appering.

The error is in code that is interpreting the annotations previously fetched ready to add the new annotations. It looks like one of the existing annotations has been changed and no longer contains the expected HTML code. If you can work out which book it is, then you can fix or remove the annotation.

I'll have a think of how to handle this better, but, I'm not sure if there is anything that can be done except removing the existing annotation or reporting an error and not updating it.

[dsestero]

Any news about better understanding what makes an annotation fetchable? (i.e. understanding why some annotated books are fetched for annotations and most other no?).

And I would like to ask if it would be possible, in a future version, to have the possibility to merge existing annotations already stored in the Calibre library with the annotations for the same book fetched from the device. This would allow to empty, from time to time, the file MyClippings.txt that otherwise continue to grow up (furthermore, when annotations are removed from the kindle, they still remain in the file, so clearing it from time to time seems to me a good idea).

Ciao,
Dario

[schnesim]

Originally Posted by dsestero

Any news about better understanding what makes an annotation fetchable? (i.e. understanding why some annotated books are fetched for annotations and most other no?).

According to my investigation so far the cause for the issue appears to be the parsing of the timestamp.

If never done any Python coding but nevertheless my plan right now is, extract the timestamp parsing method so I can run it in an IDE and debug what's actually happening. (Also there is a nasty rege which I need to dissect.)

[dsestero]

Schnesim, I don't know Python but in the past I was a Java programmer. May be I can help with the regular expression if you point it to me.

Good night,
Dario

[davidfor]

Originally Posted by dsestero

Any news about better understanding what makes an annotation fetchable? (i.e. understanding why some annotated books are fetched for annotations and most other no?).

Sorry, I haven't had a chance to have a real look. My assumption is that the book matching isn't as good enough. But, a little change in the line that is being matched might be it.

And I would like to ask if it would be possible, in a future version, to have the possibility to merge existing annotations already stored in the Calibre library with the annotations for the same book fetched from the device. This would allow to empty, from time to time, the file MyClippings.txt that otherwise continue to grow up (furthermore, when annotations are removed from the kindle, they still remain in the file, so clearing it from time to time seems to me a good idea).

The plugin is already designed to do this. It should be able to take the annotations from multiple devices and merge them into one. Are you seeing problems?