[GUI Plugin] Annotations

[jopping]

Thanks for making such a great plugin @davidfor!

I'm trying to customize the Annotations plugin to do this:

If you'd prefer to add imported annotations to a custom column (recommended), open the Customize Annotations dialog by selecting Customize plugin… from the plugin's context menu. Use the custom column wizard in the Annotation options section to create a custom column.

However, every time I click on the Customize plugin... button, Calibre returns the following error:

Spoiler:

calibre, version 4.0.0
ERROR: Must restart: You must restart calibre before you can configure the Annotations plugin

This occurs despite me trying to restart Calibre multiple times.

I was wondering if you could give me any help into how to solve this problem? It'd be amazing if I could separate my annotations into a custom field! Thanks!

Here's the debug log when I try to customize the plugin in debug mode:

Spoiler:

calibre Debug log
calibre 4.0 embedded-python: True is64bit: True
Linux-5.0.0-29-generic-x86_64-with-debian-buster-sid Linux ('64bit', 'ELF')
('Linux', '5.0.0-29-generic', '#31-Ubuntu SMP Thu Sep 12 13:05:32 UTC 2019')
Python 2.7.16
Linux: ('debian', 'buster/sid', '')
Interface language: None
Successfully initialized third party plugins: KoboTouchExtended (3, 1, 5) && KePub Output (3, 1, 3) && Annotations (1, 10, 0)
calibre 4.0 embedded-python: True is64bit: True
Linux-5.0.0-29-generic-x86_64-with-debian-buster-sid Linux ('64bit', 'ELF')
('Linux', '5.0.0-29-generic', '#31-Ubuntu SMP Thu Sep 12 13:05:32 UTC 2019')
Python 2.7.16
Linux: ('debian', 'buster/sid', '')
Interface language: None
Successfully initialized third party plugins: KoboTouchExtended (3, 1, 5) && KePub Output (3, 1, 3) && Annotations (1, 10, 0)
Not controlling automatic hidpi scaling
devicePixelRatio: 1.1
logicalDpi: 96.3636363636 x 96.3636363636
physicalDpi: 84.1043643264 x 84.2662162162
Using calibre Qt style: True
[0.00] Starting up...
[0.00] Showing splash screen...
[0.10] splash screen shown
[0.10] Initializing db...
[0.14] db initialized
[0.14] Constructing main UI...
DEBUG: 0.0 Annotations plugin: Error loading libiMobileDevice. This hasn't worked for a while, and is blacklisted in calibre v3.
DEBUG: 0.0 Annotations plugin: Error is: No module named idevice.libimobiledevice
DEBUG: 0.0 Error loading QT5: No module named QtWebKitWidgets
Traceback (most recent call last):
File "site-packages/calibre/gui2/ui.py", line 157, in __init__
File "site-packages/calibre/gui2/ui.py", line 171, in init_iaction
File "site-packages/calibre/customize/__init__.py", line 612, in load_actual_plugin
File "importlib/__init__.py", line 37, in import_module
File "site-packages/calibre/customize/zipplugin.py", line 185, in load_module
File "calibre_plugins.annotations.action", line 43, in <module>
File "site-packages/calibre/customize/zipplugin.py", line 185, in load_module
File "calibre_plugins.annotations.annotated_books", line 24, in <module>
ImportError: No module named PyQt4
Looking for desktop notifier support from: org.freedesktop.Notifications
org.freedesktop.Notifications found in 0.0 seconds
[0.75] main UI initialized...
[0.75] Hiding splash screen
[9.39] splash screen hidden
[9.39] Started up in 9.39 seconds with 40 books
Icon theme "gnome" not found.

[DNSB]

Originally Posted by jopping

Thanks for making such a great plugin @davidfor!

I'm trying to customize the Annotations plugin to do this:

However, every time I click on the Customize plugin... button, Calibre returns the following error:
This occurs despite me trying to restart Calibre multiple times.

I was wondering if you could give me any help into how to solve this problem? It'd be amazing if I could separate my annotations into a custom field! Thanks!

Here's the debug log when I try to customize the plugin in debug mode:

To quoted davidfor's post #645 immediately prior to your post (bolding mine):

Originally Posted by davidfor

I have just released version 1.10.0 of the plugin. I should have done this before, but kept forgetting. The release of calibre 4.0 has triggered this release as it has a change that breaks this plugin. I'll be fixing this soon, but am making this release as the last version to support calibre 3.x and earlier.

[DEADWILDER]

Would it be possible to re-add support for Marvin since the Marvin XD plugin is no longer working? Thank you!

[Philantrop]

Originally Posted by DEADWILDER

Would it be possible to re-add support for Marvin since the Marvin XD plugin is no longer working? Thank you!

No, sadly Apple and Marvin's developer made that impossible or I would gladly fix both plugins.

[DEADWILDER]

Originally Posted by Philantrop

No, sadly Apple and Marvin's developer made that impossible or I would gladly fix both plugins.

Oh, I see. Would it be possible to add support for KyBook 3 instead?

[davidfor]

Originally Posted by DEADWILDER

Oh, I see. Would it be possible to add support for KyBook 3 instead?

Possibly. It depends on where the data is. If it is visible when the device is connected, the plugin can access it. Otherwise, it needs to be implemented in the same way as for GoodReader where the data is emailed and the plugin reads that file.

It also needs knowledge of the format and someone willing to do it. I don't use KyBook, so it isn't something I'm particularly interested in implementing myself. If someone wants to do it, I will be happy to give advice and help.

[Kicweed]

Hey @davidfor, thanks for all the effort you've put in this :-)

I've been having the following issue with this plugin lately and I don't know if there's something I can do from my end to fix it.

I'm on Calibre 3.48, with the 1.10.0 version of the plugin and a Kobo Forma on 4.18.13737.

The Import Annotations screen appears normally (the one where you see a list of all the books with the number of annotations for each and you can selectively preview for each book).

Unfortunately, during the next step (after pressing the Import Annotations button) the process starts but fails while importing, with the following message:

calibre, version 3.48.0
Error fetching annotations: Unable to fetch annotations from Kobo Forma.

Traceback (most recent call last):
  File "calibre_plugins.annotations.action", line 311, in fetch_device_annotations
  File "calibre_plugins.annotations.action", line 1052, in process_selected_books
  File "calibre_plugins.annotations.action", line 160, in add_annotations_to_calibre
  File "calibre_plugins.annotations.annotations", line 297, in merge_annotations
  UnboundLocalError: local variable 'regurgitated_soup' referenced before assignment

I've enabled the "Enable debug" option but I don't know where the log is kept to see whether there's any additional helpful information.

Do you have any idea what could be breaking so I could try to fix it?

Thanks in advance!

[davidfor]

Originally Posted by Kicweed

Hey @davidfor, thanks for all the effort you've put in this :-)

I've been having the following issue with this plugin lately and I don't know if there's something I can do from my end to fix it.

I'm on Calibre 3.48, with the 1.10.0 version of the plugin and a Kobo Forma on 4.18.13737.

The Import Annotations screen appears normally (the one where you see a list of all the books with the number of annotations for each and you can selectively preview for each book).

Unfortunately, during the next step (after pressing the Import Annotations button) the process starts but fails while importing, with the following message:

calibre, version 3.48.0
Error fetching annotations: Unable to fetch annotations from Kobo Forma.

Traceback (most recent call last):
  File "calibre_plugins.annotations.action", line 311, in fetch_device_annotations
  File "calibre_plugins.annotations.action", line 1052, in process_selected_books
  File "calibre_plugins.annotations.action", line 160, in add_annotations_to_calibre
  File "calibre_plugins.annotations.annotations", line 297, in merge_annotations
  UnboundLocalError: local variable 'regurgitated_soup' referenced before assignment

I've enabled the "Enable debug" option but I don't know where the log is kept to see whether there's any additional helpful information.

Do you have any idea what could be breaking so I could try to fix it?

That looks like a problem in the code. A case isn't being handled properly when attempting to merge the existing annotations with new ones. But, I need a little bit more information to work out what it is. Can you run calibre in debug mode and do the fetch again? You can do this by right-clicking on the preferences button and choosing "Restart in debug mode". Calibre will restart and display a message about the log. Then connect the device and try to fetch the annotations. After the error is displayed, close calibre and it will display a log. Post that or PM me the contents to look at.

[Kicweed]

I've PMed you the log, I hope it's helpful.

The only peculiarity of my setup (which was not a problem up to now), is that I'm using a 3-column setup for my annotations:

• Column A: Annotations added by me manually, inside Calibre
• Column B: Annotations used by the GUI Annotations plugin, not touched by me at all
• Column C: The result of A + B, used to export my annotations to an ebook by Calibre's "Export Catalog" feature.

The above setup has been working without issues, I don't know what might have changed and at what point.

[davidfor]

Originally Posted by Kicweed

I've PMed you the log, I hope it's helpful.

I will have a look at it over the weekend. But, I think I know what the problem is.

The only peculiarity of my setup (which was not a problem up to now), is that I'm using a 3-column setup for my annotations:

• Column A: Annotations added by me manually, inside Calibre
• Column B: Annotations used by the GUI Annotations plugin, not touched by me at all
• Column C: The result of A + B, used to export my annotations to an ebook by Calibre's "Export Catalog" feature.

The above setup has been working without issues, I don't know what might have changed and at what point.

I doubt that is affecting anything. Only "Column B" should be referenced in the plugin. The others are just other custom columns that have similar sort of data in them.

[davidfor]

Originally Posted by Kicweed

I've PMed you the log, I hope it's helpful.

I can see the problem. It looks like the annotation has been edited in the library. In fact, it looks like the text in the annotation column was never put there by the plugin. Is this the case?

The code is looking for a div that wraps the annotation. It extracts them and merges them with the new annotations. In this case, it cannot find the old annotation as the div with the class "user_annotations" is missing.

This is fragile, and is even worse with calibre 4.x. The change in the HTML editor use is making a lot of changes to the code that will break the way the plugin updates the annotations. I'll have to look at it, but, at this point, I don't have an idea of what to do.

[Terisa de morgan]

Originally Posted by davidfor

This is fragile, and is even worse with calibre 4.x. The change in the HTML editor use is making a lot of changes to the code that will break the way the plugin updates the annotations. I'll have to look at it, but, at this point, I don't have an idea of what to do.

I feel your pain about this. I've had to adapt a personal plugin for handling the restricted (and... let´s say.... easily modified by the editor) html that can be used now.

[davidfor]

Originally Posted by Terisa de morgan

I feel your pain about this. I've had to adapt a personal plugin for handling the restricted (and... let´s say.... easily modified by the editor) html that can be used now.

I read those posts. And I was thinking that it had been fixed. But, in this case it definitely hasn't. There is a div wrapped around all the annotations that loses it's class. There is another that wraps each annotation and is actually replace by a paragraph with a break inside it.

I do want people to be able to edit the annotations. At the least, being able to remove any that are no longer needed is needed function. I really don't want to implement an editor of any sort.

[kovidgoyal]

Qt's rich text editor does not preserve html classes sadly. It might preserve id/name/title attributes, you could try with those.

[Terisa de morgan]

Originally Posted by davidfor

I read those posts. And I was thinking that it had been fixed. But, in this case it definitely hasn't. There is a div wrapped around all the annotations that loses it's class. There is another that wraps each annotation and is actually replace by a paragraph with a break inside it.

I do want people to be able to edit the annotations. At the least, being able to remove any that are no longer needed is needed function. I really don't want to implement an editor of any sort.

It cannot be fixed because the editor is the main problem. It says that it allows something but it's not really true, in the sense that modifies the changes you have done in the html view even if it is according to its own rules, to code it at the editor thinks it is the proper way, ignoring the user input.