[GUI Plugin] Annotations - Page 55

thiago.eec · 12-27-2020, 09:38 AM

Hi, David.

I having some issues with Kindle Paperwhite.

1) When I click to fetch annotations, the list of Annotations is loaded and then this error appears:

Spoiler:

2) After the error above, if I click on the Help button (on annotations window), it freezes calibre. This is the whole log from calibre-debug, after forcing it to close:

Spoiler:

UMNiK · 12-27-2020, 11:58 AM

Can't seem to find the pm system on this forum (but hey, found some nostalgia for the early 2000s). Since the logs are clean of anything personal, here they are.
https://www.dropbox.com/s/qcorlw29gf...pings.txt?dl=0
https://www.dropbox.com/s/gb0euwarcg...umwwz.txt?dl=0

In fact, I also had the error that thiago.eec has, but restarting with disable cache seems to have moved it to my situation (no error, but no annotations either).

thiago.eec · 12-27-2020, 12:24 PM

Quote:

Originally Posted by UMNiK

Can't seem to find the pm system on this forum (but hey, found some nostalgia for the early 2000s). Since the logs are clean of anything personal, here they are.
https://www.dropbox.com/s/qcorlw29gf...pings.txt?dl=0
https://www.dropbox.com/s/gb0euwarcg...umwwz.txt?dl=0

In fact, I also had the error that thiago.eec has, but restarting with disable cache seems to have moved it to my situation (no error, but no annotations either).

Actually, despite the error, the plugin still fetches the annotations here.

davidfor · 12-28-2020, 12:23 AM

Quote:

Originally Posted by thiago.eec

Hi, David.

I having some issues with Kindle Paperwhite.

1) When I click to fetch annotations, the list of Annotations is loaded and then this error appears:

That is an error sorting the list after it is displayed. Do you know which column the sort was done one? And are any of the columns empty? Or show as something like "None"?

Quote:

2) After the error above, if I click on the Help button (on annotations window), it freezes calibre. This is the whole log from calibre-debug, after forcing it to close:

My bad. It has been a long time since I pressed that help button. And honestly, I didn't remember what it actually showed. I had to use calibre 4.23 to see.

davidfor · 12-28-2020, 02:12 AM

Quote:

Originally Posted by UMNiK

Can't seem to find the pm system on this forum (but hey, found some nostalgia for the early 2000s). Since the logs are clean of anything personal, here they are.
https://www.dropbox.com/s/qcorlw29gf...pings.txt?dl=0
https://www.dropbox.com/s/gb0euwarcg...umwwz.txt?dl=0

In fact, I also had the error that thiago.eec has, but restarting with disable cache seems to have moved it to my situation (no error, but no annotations either).

I'm baffled. The books appear to be there and they should match. And it seems to only affect KFX books. The attached beta has more debugging to try and see what is going on.

The beta also includes fixes for the problems with the sorting and help.

And for PMs, it will probably get enabled after a few more posts, but sometimes it isn't. Post in https://www.mobileread.com/forums/sh...d.php?t=319819 to ask it to be enabled and a moderator will fix it.

UMNiK · 12-28-2020, 06:16 AM

Same issue, no error, no annotations. Here are the new logs and clippings.

https://www.dropbox.com/s/grz6e618a6...7_gtk.txt?dl=0
https://www.dropbox.com/s/k4pqyyvdnh...pings.txt?dl=0

davidfor · 12-28-2020, 07:28 AM

Quote:

Originally Posted by UMNiK

Same issue, no error, no annotations. Here are the new logs and clippings.

https://www.dropbox.com/s/grz6e618a6...7_gtk.txt?dl=0
https://www.dropbox.com/s/k4pqyyvdnh...pings.txt?dl=0

I wasn't expecting that to fix the problem with fetching the annotations. I was adding more logging to try and see where it was going wrong. And it has pointed out a problem. Unfortunately, I'm not sure if this is the issue, and if it is, exactly how to fix it.

The attached beta has more logging and a couple of changes that might solve it. But, before using it, could you add an annotation to an AZW3 or MOBI format book. The problem seems to be for KFX books. I want to see an example of a working annotation for the other formats to check how they work.

UMNiK · 12-28-2020, 08:37 AM

Quote:

Originally Posted by davidfor

I wasn't expecting that to fix the problem with fetching the annotations. I was adding more logging to try and see where it was going wrong. And it has pointed out a problem. Unfortunately, I'm not sure if this is the issue, and if it is, exactly how to fix it.

The attached beta has more logging and a couple of changes that might solve it. But, before using it, could you add an annotation to an AZW3 or MOBI format book. The problem seems to be for KFX books. I want to see an example of a working annotation for the other formats to check how they work.

Look like you won't be seeing that example just yet. Also no annotations or errors for AZW3.

https://www.dropbox.com/s/pdthw4trnx...qmpkj.txt?dl=0
https://www.dropbox.com/s/to4cqbflqd...pings.txt?dl=0

jhowell · 12-28-2020, 10:06 AM

I don't know if this is relevant, but I happened to notice that the My Clippings.txt file in the post above has a Unicode BOM character at the beginning of each book title. Those might not be visible depending on how you view those files.

thiago.eec · 12-28-2020, 06:58 PM

Quote:

Originally Posted by davidfor

That is an error sorting the list after it is displayed. Do you know which column the sort was done one? And are any of the columns empty? Or show as something like "None"?

Actually, no column was marked as sorted (no arrow on column headers). Then, I clicked on Title column and tried again, but the error remained. Also tried other columns, but with no luck.

There is no empty columns (or filled with 'none').

Quote:

Originally Posted by davidfor

My bad. It has been a long time since I pressed that help button. And honestly, I didn't remember what it actually showed. I had to use calibre 4.23 to see.

Thanks. This is fixed.

And just so you know: colors are not well suited for Dark Mode, as seen in the attachment.

davidfor · 12-28-2020, 09:41 PM

Quote:

Originally Posted by jhowell

I don't know if this is relevant, but I happened to notice that the My Clippings.txt file in the post above has a Unicode BOM character at the beginning of each book title. Those might not be visible depending on how you view those files.

Of course. I looked for blanks and things, but, I didn't think of that.

It looks like the Kindle is adding the BOM character at the beginning of each annotation block. They are probably building the complete string for the block and then dumping it into the file without worrying about this. It might be deliberate as a way of separating the annotations, but, I don't think so. And my brain hurts whenever I play with this stuff.

In any case, the attached beta does "record.encode().decode('utf-8-sig')" where record is the annotation block. Though I am sure there is a better way to handle this, it seems to work in the test harness against @UMNiK, but that just parses the file and doesn't do the matching to the actual book. It produces a title without the BOM character, so it should work.

The attached beta has this change, so we will see.

davidfor · 12-28-2020, 09:51 PM

Quote:

Originally Posted by thiago.eec

Actually, no column was marked as sorted (no arrow on column headers). Then, I clicked on Title column and tried again, but the error remained. Also tried other columns, but with no luck.

There is no empty columns (or filled with 'none').

I realised after posting my comments that the default sort was actually a hidden column. And with the way the list is built it would be possible to display values everywhere, but, the sort values might not be set.

Did the beta fix the problem? The changes in the beta should have fixed it. Or possibly changed the error.

Quote:

Thanks. This is fixed.

And just so you know: colors are not well suited for Dark Mode, as seen in the attachment.

Sorry, it's not really a problem I am interested in fixing. It either means coming up with a colour combination that always works, adding code to use different colours depending on the styles, or code to configure the colour choices.

UMNiK · 12-29-2020, 03:16 AM

Despite throwing a "nonetype" error, it found and imported the annotations correctly!

https://www.dropbox.com/s/4ngxopang1...kg_s3.txt?dl=0
https://www.dropbox.com/s/0ryj3wk7vh...pings.txt?dl=0

davidfor · 12-29-2020, 04:55 AM

Quote:

Originally Posted by UMNiK

Despite throwing a "nonetype" error, it found and imported the annotations correctly!

Woops. Was sorting before setting the column for the sort. And the default column was the worst column to sort on.

Quote:

https://www.dropbox.com/s/4ngxopang1...kg_s3.txt?dl=0
https://www.dropbox.com/s/0ryj3wk7vh...pings.txt?dl=0

YES!!!! Though I am horrified at the size of that log file.

The attached beta should be it:

Fixes the sorting problem
Has the matching correct for Kindle devices.
Includes a change the "confidence" level for matching the book. More should be green in the fetched annotations list and not prompt to check the book when doing the actual import.
Removed the bulk of the logging.

Again, please tell me how it goes. If there are no issues, I will try and release it at the end of the week.

UMNiK · 12-29-2020, 06:05 AM

It all went perfectly (except for a single book with messed up metadata (author-title switch), which is my fault)! The new confidence level also saved some time by not requiring an ID for a green match. I even added an older MyClippings.txt on top of the existing one, and it all matched up no problem! Thanks for your hard (and quick) work on this!

https://www.dropbox.com/s/7tdfm1l18h...qbr70.txt?dl=0
https://www.dropbox.com/s/blqxe0f0g1...pings.txt?dl=0

12-27-2020, 09:38 AM	#811
thiago.eec Wizard Posts: 1,217 Karma: 1419583 Join Date: Dec 2016 Location: Goiânia - Brazil Device: iPad, Kindle Paperwhite, Kindle Oasis	Hi, David. I having some issues with Kindle Paperwhite. 1) When I click to fetch annotations, the list of Annotations is loaded and then this error appears: Spoiler: calibre, version 5.8.1 ERRO: Exceção sem tratamento: <b>TypeError</b>:'<' not supported between instances of 'NoneType' and 'NoneType' calibre 5.8.1 [64bit] embedded-python: True is64bit: True Windows-10-10.0.19041 Windows ('64bit', 'WindowsPE') ('Windows', '10', '10.0.19041') Python 3.8.5 Windows: ('10', '10.0.19041', '', 'Multiprocessor Free') Interface language: pt_BR Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (1, 36, 0) && Package KFX (from KFX Input) (1, 36, 0) && ACE (1, 1, 1) && Annotations (1, 14, 0) && Calibre Power Search Plugin (1, 7, 0) && Count Pages (1, 11, 0) && Diaps Editing Toolbag (0, 3, 7) && EpubCheck (0, 2, 3) && EpubMerge (2, 11, 0) && EpubSplit (3, 0, 0) && Find Duplicates (1, 8, 3) && Goodreads (1, 5, 1) && Goodreads Sync (1, 15, 0) && KFX metadata reader (from KFX Input) (1, 36, 0) && KFX Input (1, 36, 0) && Set KFX metadata (from KFX Output) (1, 47, 0) && KFX Output (1, 47, 0) && KindleUnpack - The Plugin (0, 82, 3) && LanguageTool (0, 1, 1) && Open With (1, 5, 13) && ScrambleEbook (0, 4, 5) && Skoob Books (1, 5, 0) && Skoob Sync (0, 3, 6) && Sum Column (0, 3, 1) && TTS to MP3 (0, 5, 8) Traceback (most recent call last): File "calibre_plugins.annotations.annotated_books", line 168, in sort TypeError: '<' not supported between instances of 'NoneType' and 'NoneType' 2) After the error above, if I click on the Help button (on annotations window), it freezes calibre. This is the whole log from calibre-debug, after forcing it to close: Spoiler: calibre 5.8.1 [64bit] embedded-python: True is64bit: True Windows-10-10.0.19041 Windows ('64bit', 'WindowsPE') ('Windows', '10', '10.0.19041') Python 3.8.5 Windows: ('10', '10.0.19041', '', 'Multiprocessor Free') Interface language: pt_BR Failed to initialize plugin: 'C:\\Users\\go_th\\AppData\\Roaming\\calibre\\plug ins\\DeDRM.zip' Traceback (most recent call last): File "calibre\customize\ui.py", line 703, in initialize_plugins File "calibre\customize\ui.py", line 61, in load_plugin File "calibre\customize\zipplugin.py", line 293, in load File "importlib\__init__.py", line 127, in import_module File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "calibre\customize\zipplugin.py", line 185, in exec_module File "calibre\customize\zipplugin.py", line 181, in get_code File "calibre_plugins.dedrm.__init__", line 168 print u"{0} v{1}: Copying needed library files from plugin's zip".format(PLUGIN_NAME, PLUGIN_VERSION) ^ SyntaxError: invalid syntax Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (1, 36, 0) && Package KFX (from KFX Input) (1, 36, 0) && ACE (1, 1, 1) && Annotations (1, 14, 0) && Calibre Power Search Plugin (1, 7, 0) && Count Pages (1, 11, 0) && Diaps Editing Toolbag (0, 3, 7) && EpubCheck (0, 2, 3) && EpubMerge (2, 11, 0) && EpubSplit (3, 0, 0) && Find Duplicates (1, 8, 3) && Goodreads (1, 5, 1) && Goodreads Sync (1, 15, 0) && KFX metadata reader (from KFX Input) (1, 36, 0) && KFX Input (1, 36, 0) && Set KFX metadata (from KFX Output) (1, 47, 0) && KFX Output (1, 47, 0) && KindleUnpack - The Plugin (0, 82, 3) && LanguageTool (0, 1, 1) && Open With (1, 5, 13) && ScrambleEbook (0, 4, 5) && Skoob Books (1, 5, 0) && Skoob Sync (0, 3, 6) && Sum Column (0, 3, 1) && TTS to MP3 (0, 5, 8) Turning on automatic hidpi scaling devicePixelRatio: 1.0 logicalDpi: 120.0 x 120.0 physicalDpi: 141.7674418604651 x 141.4020618556701 Using calibre Qt style: True [0.00] Starting up... [0.00] Showing splash screen... [0.66] splash screen shown [0.66] Initializing db... [0.74] db initialized [0.74] Constructing main UI... DEBUG: 0.0 Annotations plugin: Error loading libiMobileDevice. This hasn't worked for a while, and is blacklisted in calibre v3. DEBUG: 0.0 Annotations plugin: Error is: No module named 'calibre.devices.idevice' DEBUG: 0.0 Annotations::reader_app_support.py - loading translations DEBUG: 0.0 Annotations::common_utils.py - loading translations DEBUG: 0.0 Annotations::appearance.py - loading translations DEBUG: 0.0 Annotations::config.py - loading translations DEBUG: 0.0 Annotations::annotated_books.py - loading translations DEBUG: 0.1 Annotations::action.py - loading translations DEBUG: 0.0 Goodreads Sync::dialogs.py - loading translations DEBUG: 0.0 GoodreadsSync::action.py - loading translations EpubMerge: DEBUG: 2020-12-27 10:19:52,920: calibre_plugins.epubmerge.epubmerge_plugin(158): macmenuhack file_path:C:\Users\go_th\AppData\Roaming\calibre\p lugins\fanficfare_macmenuhack.txt DEBUG: 1.3 HttpHelper::__init__: proxy=None [2.59] main UI initialized... [2.59] Hiding splash screen Starting QuickView [3.53] splash screen hidden [3.53] Started up in 3.53 seconds with 1328 books Storage number map: {(7, 0): [(3, 'C')], (7, 1): [(1, 'D'), (2, 'Z')], (7, 2): [(1, 'F')], (7, 4): [(1, 'G')]} Storage number for USBSTOR\DISK&VEN_KINDLE&PROD_INTERNAL_STORAGE&REV_ 0100\G090KB03824406RQ&0: StorageDeviceNumber(type=7, number=4, partition_number=0) Drive letters for USBDevice(vendor_id=0x1949 product_id=0x4 bcd=0x100 devid=usb\vid_1949&pid_0004&rev_0100 devinst=5) {'drive_letters': ['G'], 'pnp_id_map': {'G': '\\\\?\\usbstor#disk&ven_kindle&prod_internal_stor age&rev_0100#g090kb03824406rq&0#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'}, 'readonly_drives': set(), 'sort_map': {'G': (4, 1)}} Syncing cover thumbnails to workaround amazon cover bug Restored 0 cover thumbnails that were destroyed by Amazon Shortcut for 'Interface Action: Annotations (Annotations) - Find annotations' already registered by Find annotations Job: 0 Obter informações do dispositivo started DeviceJob: 0 Obter informações do dispositivo done, calling callback DeviceJob: 0 Obter informações do dispositivo callback returned Job: 1 Definir informações da biblioteca started DeviceJob: 1 Definir informações da biblioteca done, calling callback DeviceJob: 1 Definir informações da biblioteca callback returned Job: 2 Obter listagem de livros do dispositivo started DEBUG: 6.6 USBMS: Fetching list of books from device. Device= KINDLE2 oncard= None DEBUG: 6.6 USBMS: dirs are: G:\ documents DEBUG: 6.7 USBMS: scan from root False G:\documents DEBUG: 6.9 USBMS: count found in cache: 181, count of files in metadata: 181, need_sync: False DEBUG: 6.9 USBMS: Finished fetching list of books from device. oncard= None DEBUG: 6.9 USBMS: Fetching list of books from device. Device= KINDLE2 oncard= carda DEBUG: 6.9 USBMS: Fetching list of books from device. Device= KINDLE2 oncard= cardb DeviceJob: 2 Obter listagem de livros do dispositivo done, calling callback DeviceJob: metadata_downloaded: Starting set_books_in_library DeviceJob: set_books_in_library: books to process= 181 DeviceJob: set_books_in_library finished: time= 0.009002447128295898 DeviceJob: metadata_downloaded: updating views DeviceJob: metadata_downloaded: syncing DeviceJob: metadata_downloaded: refreshing ondevice DeviceJob: metadata_downloaded: sending metadata_available signal DeviceJob: 2 Obter listagem de livros do dispositivo callback returned Job: 0 Obter informações do dispositivo finished Nenhum detalhe disponível. Job: 2 Obter listagem de livros do dispositivo finished Nenhum detalhe disponível. Job: 1 Definir informações da biblioteca finished Nenhum detalhe disponível. Shortcut for 'Interface Action: Annotations (Annotations) - Find annotations' already registered by Find annotations Shortcut for 'Interface Action: Annotations (Annotations) - Find annotations' already registered by Find annotations Job: 2 Obter listagem de livros do dispositivo finished Nenhum detalhe disponível. calibre 5.8.1 [64bit] embedded-python: True is64bit: True Windows-10-10.0.19041 Windows ('64bit', 'WindowsPE') ('Windows', '10', '10.0.19041') Python 3.8.5 Windows: ('10', '10.0.19041', '', 'Multiprocessor Free') Interface language: pt_BR Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (1, 36, 0) && Package KFX (from KFX Input) (1, 36, 0) && ACE (1, 1, 1) && Annotations (1, 14, 0) && Calibre Power Search Plugin (1, 7, 0) && Count Pages (1, 11, 0) && Diaps Editing Toolbag (0, 3, 7) && EpubCheck (0, 2, 3) && EpubMerge (2, 11, 0) && EpubSplit (3, 0, 0) && Find Duplicates (1, 8, 3) && Goodreads (1, 5, 1) && Goodreads Sync (1, 15, 0) && KFX metadata reader (from KFX Input) (1, 36, 0) && KFX Input (1, 36, 0) && Set KFX metadata (from KFX Output) (1, 47, 0) && KFX Output (1, 47, 0) && KindleUnpack - The Plugin (0, 82, 3) && LanguageTool (0, 1, 1) && Open With (1, 5, 13) && ScrambleEbook (0, 4, 5) && Skoob Books (1, 5, 0) && Skoob Sync (0, 3, 6) && Sum Column (0, 3, 1) && TTS to MP3 (0, 5, 8) Traceback (most recent call last): File "calibre_plugins.annotations.annotated_books", line 168, in sort TypeError: '<' not supported between instances of 'NoneType' and 'NoneType' calibre 5.8.1 [64bit] embedded-python: True is64bit: True Windows-10-10.0.19041 Windows ('64bit', 'WindowsPE') ('Windows', '10', '10.0.19041') Python 3.8.5 Windows: ('10', '10.0.19041', '', 'Multiprocessor Free') Interface language: pt_BR Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (1, 36, 0) && Package KFX (from KFX Input) (1, 36, 0) && ACE (1, 1, 1) && Annotations (1, 14, 0) && Calibre Power Search Plugin (1, 7, 0) && Count Pages (1, 11, 0) && Diaps Editing Toolbag (0, 3, 7) && EpubCheck (0, 2, 3) && EpubMerge (2, 11, 0) && EpubSplit (3, 0, 0) && Find Duplicates (1, 8, 3) && Goodreads (1, 5, 1) && Goodreads Sync (1, 15, 0) && KFX metadata reader (from KFX Input) (1, 36, 0) && KFX Input (1, 36, 0) && Set KFX metadata (from KFX Output) (1, 47, 0) && KFX Output (1, 47, 0) && KindleUnpack - The Plugin (0, 82, 3) && LanguageTool (0, 1, 1) && Open With (1, 5, 13) && ScrambleEbook (0, 4, 5) && Skoob Books (1, 5, 0) && Skoob Sync (0, 3, 6) && Sum Column (0, 3, 1) && TTS to MP3 (0, 5, 8) Traceback (most recent call last): File "calibre_plugins.annotations.annotated_books", line 434, in show_annotated_books_dialog_clicked File "calibre_plugins.annotations.annotated_books", line 442, in show_help File "calibre_plugins.annotations.common_utils", line 510, in __init__ TypeError: QTextEdit.setHtml(): argument 1 has unexpected type 'bytes'

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] KindleUnpack - The Plugin	DiapDealer	Plugins	523	07-15-2025 06:45 PM
[GUI Plugin] Open With	kiwidude	Plugins	404	02-21-2025 05:42 AM
[GUI Plugin] Annotations (closed)	GRiker	Plugins	68	06-17-2014 06:11 AM
[GUI Plugin] KiNotes	-axel-	Plugins	0	07-14-2013 06:39 PM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 12:27 PM

12-27-2020, 11:58 AM	#812
UMNiK Member Posts: 12 Karma: 10 Join Date: Oct 2020 Device: Kindle PW4	Can't seem to find the pm system on this forum (but hey, found some nostalgia for the early 2000s). Since the logs are clean of anything personal, here they are. https://www.dropbox.com/s/qcorlw29gf...pings.txt?dl=0 https://www.dropbox.com/s/gb0euwarcg...umwwz.txt?dl=0 In fact, I also had the error that thiago.eec has, but restarting with disable cache seems to have moved it to my situation (no error, but no annotations either).

12-28-2020, 06:16 AM	#816
UMNiK Member Posts: 12 Karma: 10 Join Date: Oct 2020 Device: Kindle PW4	Same issue, no error, no annotations. Here are the new logs and clippings. https://www.dropbox.com/s/grz6e618a6...7_gtk.txt?dl=0 https://www.dropbox.com/s/k4pqyyvdnh...pings.txt?dl=0

12-28-2020, 10:06 AM	#819
jhowell Grand Sorcerer Posts: 7,078 Karma: 91577715 Join Date: Nov 2011 Location: Charlottesville, VA Device: Kindles	I don't know if this is relevant, but I happened to notice that the My Clippings.txt file in the post above has a Unicode BOM character at the beginning of each book title. Those might not be visible depending on how you view those files.

12-29-2020, 03:16 AM	#823
UMNiK Member Posts: 12 Karma: 10 Join Date: Oct 2020 Device: Kindle PW4	Despite throwing a "nonetype" error, it found and imported the annotations correctly! https://www.dropbox.com/s/4ngxopang1...kg_s3.txt?dl=0 https://www.dropbox.com/s/0ryj3wk7vh...pings.txt?dl=0

12-29-2020, 06:05 AM	#825
UMNiK Member Posts: 12 Karma: 10 Join Date: Oct 2020 Device: Kindle PW4	It all went perfectly (except for a single book with messed up metadata (author-title switch), which is my fault)! The new confidence level also saved some time by not requiring an ID for a green match. I even added an older MyClippings.txt on top of the existing one, and it all matched up no problem! Thanks for your hard (and quick) work on this! https://www.dropbox.com/s/7tdfm1l18h...qbr70.txt?dl=0 https://www.dropbox.com/s/blqxe0f0g1...pings.txt?dl=0