[GUI Plugin] Annotations - Page 62

davidfor · 03-05-2021, 08:57 PM

Quote:

Originally Posted by Noirtier

Hi David,

I've been having another play with this today after updating the Kobo driver, Calibre to 5.12 and the annotations plugin to 1.14.10. It looks like the issue with some annotations appearing in the preview, but not being added to the metadata correctly still remains unfortunately. I imported annotations from five new books: Two imported all correctly, one imported mostly fine apart from some ordering differences between the preview and saved metadata, while the remaining two had significant chunks missing. Let me know if you would like me to try anything else, but no worries either way. It is not a big problem to work around by copying from the preview (though changing the preview window so it is not always forced on top of all other windows would help a bit with this).

If there are still issues, then I need to see a debug log and details of what is missing. And the database from the device.

Quote:

The bigger problem is really having to re-stitch together all the sections that crossed over page-breaks, and thus needed to be marked with two separate highlights in the Kobo (I think this might be where the ordering differences sometimes crop up between the Kobo/preview and imported metadata versions – I guess one uses the location in the book to order, while the other uses the timestamp of the highlight to order perhaps?). I assume there is nothing anyone can do about the buggy highlighting functionality other than Kobo themselves.

Yes, the ordering in the preview should be by timestamp and in the metadata by location. With the latter, new annotations are merged with any existing annotations. I think the location order works better for that.

hollowpoint · 03-08-2021, 12:13 PM

Hey David, thanks for your work on this extension.

I'm running Win10, Calibre 5.12(x64), Annotations 1.14.

Cannot get Annotations to work with any of the following 3 devices. Despite creating the annotations column, etc. I have had it working in the past, though not with these newer devices.

Issue per device:

-PW4 (5.13.4). Connect by USB, device recognized and can see books, Annotations tries and says it cannot import any ("No annotated books found on device: Unable to find any annotations on Amazon Kindle matching books in your library."). There are definitely annotations on certain Kindle books that are synced between all these devices, and there's also a MyClippings.txt file with some annotations--opened and checked it.

-Samsung Galaxy phone (Android 11). Annotations are all synced to books in Kindle. Device connected by USB and recognized by Calibre. However "Fetch annotations" option is grayed out.

-Ipad Mini v5 (iOS 14.14). Connected by lightning cable. Device not recognized by Calibre.

So, the questions boil down to these:

- Is there any way to attach one's current-gen smart devices (Android 10/11, iOS 14.x) to Calibre and *directly* import the synced Kindle annotations?

- Thots on what might be blocking the PW4 from importing annotations? This one I expected to work, even though I had doubts about the smart devices.

davidfor · 03-08-2021, 11:19 PM

Quote:

Originally Posted by hollowpoint

So, the questions boil down to these:

- Is there any way to attach one's current-gen smart devices (Android 10/11, iOS 14.x) to Calibre and *directly* import the synced Kindle annotations?

For iOS the answer is definitely no. The plugin did support it, but, Apple made it extremely hard or possibly impossible to do so. They locked down the access when the device was connected and from what I saw, no one could manage to do it.

For Android, I don't know, but, definitely not at the moment. There are absolutely no standards for annotations. Each ereader app handles them differently. They have to store them somewhere, and for the plugin to get them, they need to be in a place that the plugin can see them. And someone needs to write code to do this. I can do that for anything I use, but, if I don't use the app or device, I hope that someone steps up to do it. I'll help, but, without being able to test something, makes it hard to prove it works.

Quote:

- Thots on what might be blocking the PW4 from importing annotations? This one I expected to work, even though I had doubts about the smart devices.

Try one of the betas in the thread. There was a problem with the last release with Kindles. It was partly due to handling for KFX but there were some other things happening.

And for the record, I keep meaning to release this properly, but, something keeps coming up that means maybe it needs a little more work. I'll try to get one out this weekend as I think the problems it fixes outweighs those that are still occurring.

hollowpoint · 03-09-2021, 10:45 AM

Ok thanks. I noticed both the Android and iOS Kindle apps have an export function for annotations that emails you an HTML file with the annotations data. It's pretty clean output, very predictable and compliant HTML. Plus the Kindle itself will email you the annotations in the form of CSV and PDF, as well.

Wonder if it'd almost be easier to write a plugin that consumes the HTML and CSV files, and import the annotations that way. Then you'd have no device connections to deal with, just the 2 file types for input options. One could use the Pandoc plugin to clean up the input or convert it to something else, before it gets stuffed into a Calibre custom column. As it's a one-way sync (it only imports the annotations to my knowledge, it doesn't look like they get synced back to connected devices), the option of importing file input only versus having to worry about app, device, and OS support, seems vastly simplified.

ETA: The problem with importing the HTML or CSV annotations output, I just remembered, is the fact that users can only export one book at a time. I haven't yet seen a way to "export all annotations from all books." If there was a way to do that, this would seem such a streamlined way to get annotations into Calibre.

davidfor · 03-09-2021, 08:17 PM

Quote:

Originally Posted by hollowpoint

Ok thanks. I noticed both the Android and iOS Kindle apps have an export function for annotations that emails you an HTML file with the annotations data. It's pretty clean output, very predictable and compliant HTML. Plus the Kindle itself will email you the annotations in the form of CSV and PDF, as well.

Wonder if it'd almost be easier to write a plugin that consumes the HTML and CSV files, and import the annotations that way. Then you'd have no device connections to deal with, just the 2 file types for input options. One could use the Pandoc plugin to clean up the input or convert it to something else, before it gets stuffed into a Calibre custom column. As it's a one-way sync (it only imports the annotations to my knowledge, it doesn't look like they get synced back to connected devices), the option of importing file input only versus having to worry about app, device, and OS support, seems vastly simplified.

ETA: The problem with importing the HTML or CSV annotations output, I just remembered, is the fact that users can only export one book at a time. I haven't yet seen a way to "export all annotations from all books." If there was a way to do that, this would seem such a streamlined way to get annotations into Calibre.

The plugin already supports doing this. The annotations can be fetched from devices, files somewhere or via dialog. The only one currently in the plugin is for Goodreader. That works by pasting text from an email into a dialog. The full details are in the first post of the thread.

The issue, as always, is that someone needs to write the code to read the file. That needs someone with an interest in doing it and the appropriate skills or interest in acquiring the skills. I'm happy to help and add the code to the plugin. But, I'm not very interested in writing code for ereaders or apps that I do not personally use. Having someone who is interested works a lot better, and, honestly, which one would I start with?

mdtanos · 03-12-2021, 10:32 PM

is it possible to highlight a book from the calibre ebook viewer and have it synced with the same My Clippings.txt from kindle? so you can switch the reading session back and forth between desktop and kindle device and still keep highlights organized?

thank you very much!

davidfor · 03-14-2021, 01:06 AM

Quote:

Originally Posted by mdtanos

is it possible to highlight a book from the calibre ebook viewer and have it synced with the same My Clippings.txt from kindle? so you can switch the reading session back and forth between desktop and kindle device and still keep highlights organized?

No, it isn't. The "My Clippings.txt" file is really just a log of the annotations and highlights on the device. When you make an annotation the device, it is added to the file, but, it is never read from it. The annotations are stored elsewhere.

mdtanos · 03-15-2021, 04:32 PM

Quote:

Originally Posted by davidfor

No, it isn't. The "My Clippings.txt" file is really just a log of the annotations and highlights on the device. When you make an annotation the device, it is added to the file, but, it is never read from it. The annotations are stored elsewhere.

thank you very much!

C Alberga · 03-26-2021, 03:02 PM

Should I report a possible "Annotations" error here? If not, where?

davidfor · 03-26-2021, 10:22 PM

Quote:

Originally Posted by C Alberga

Should I report a possible "Annotations" error here? If not, where?

This thread is for the Annotations Plugin, so any problems with it should be reported here. Problems with other annotations (calibre annotations, on device, etc) should be reported elsewhere.

robjlucas · 04-01-2021, 05:15 AM

Firstly, thank you very much for providing this plugin; your work is much appreciated. I couldn't believe it when I discovered there was no built-in way of exporting my annotations from non-DRMed documents in the Kindle.

However, I'm hitting a basic stumbling block with setup:

Running Calibre 4.99 on Ubuntu 20.04, with a connected Kindle, from which I am trying to fetch annotations:

1. I select the document that is on the Kindle, click "Annotations" > "Fetch annotations from connected device"
2. I get the error message "Unable to import annotations as the annotations column has not been configured..."
3. To set up the column, I click "Annotations" > "Customize plugin...".
4. Under "Add fetched annotations to", "Comments" is preselected; clicking the custom column wizard icon next to this does nothing, and the icon seems to be greyed out.

Is there some way around this, such as adding the appropriate column manually? One would, presumably need to know the technical requirements for such a column.

davidfor · 04-01-2021, 06:00 AM

Quote:

Originally Posted by robjlucas

Firstly, thank you very much for providing this plugin; your work is much appreciated. I couldn't believe it when I discovered there was no built-in way of exporting my annotations from non-DRMed documents in the Kindle.

However, I'm hitting a basic stumbling block with setup:

Running Calibre 4.99 on Ubuntu 20.04, with a connected Kindle, from which I am trying to fetch annotations:

1. I select the document that is on the Kindle, click "Annotations" > "Fetch annotations from connected device"
2. I get the error message "Unable to import annotations as the annotations column has not been configured..."
3. To set up the column, I click "Annotations" > "Customize plugin...".
4. Under "Add fetched annotations to", "Comments" is preselected; clicking the custom column wizard icon next to this does nothing, and the icon seems to be greyed out.

A dialog should show up when you click the button. Does anything else work on the configuration dialog after you do this? It is possible the wizard is opening off screen and you need to get it back in view to use it.

Quote:

Is there some way around this, such as adding the appropriate column manually? One would, presumably need to know the technical requirements for such a column.

You can create the column yourself. It is just a "Long text, like comments..." column. The configuration dialog will show all suitable columns in for you to choose from. Create the column and restart calibre and when you open the plugins configuration dialog, the column should be in the drop-down.

enaid1 · 04-01-2021, 09:41 PM

Thank you so much for this plugin! (At first I couldn't make it work, but then I tried the most recent beta and everything worked perfectly...)

Question: is there a way to mass delete annotations from a book (whether downloaded from the Kindle store or side loaded)? I am using my Kindle to read books in the language I am learning and I highlight a bunch of words to download and make flash cards out of. The only way to delete these that I've found so far is one at a time (such as on the Amazon Highlights page).

davidfor · 04-02-2021, 12:14 AM

Quote:

Originally Posted by enaid1

Thank you so much for this plugin! (At first I couldn't make it work, but then I tried the most recent beta and everything worked perfectly...)

Question: is there a way to mass delete annotations from a book (whether downloaded from the Kindle store or side loaded)? I am using my Kindle to read books in the language I am learning and I highlight a bunch of words to download and make flash cards out of. The only way to delete these that I've found so far is one at a time (such as on the Amazon Highlights page).

Sorry, I don't have a Kindle, and I don't know how they work. But, I would have expected that there was an annotations list on the device that would have an option to do it. That would not remove them from the My Clippings.txt file that the plugin reads, but, if the function is there it should remove them from everywhere else.

davidfor · 04-03-2021, 11:55 PM

I have just released version 1.15.0 of the plugin. This is a bug-fix release that covers the beta's since the last release.

The fixes in this release are:

Fix: Spanish My Clippings.txt has "La" for the annotations.
Fix: Change method of getting books on device to use a search similar to what is used for Kobo and Tolino devices.
Fix: Sorting of annotations for kepubs on Kobo devices.
Fix: Opening My Clippings.txt should have been binary. Missed this when reverted earlier change.
Fix: Display more details of an exception when parsing My Clippings.txt.
Fix: Goodreader import not working.
Fix: Broke Python 2 support with the open when reading My Clippings.txt.
Fix: If already have book id in library, use confidence of 5.
Fix: Encode and decode the title from My Clippings.txt to remove the Unicode BOM character.
Fix: A couple of changes to solve some obvious problems with the Kindle fetch. Plus more logging.
Fix: Python 3 compatibility: Help dialogs.
Fix: Sorting columns in fetched annotations dialog.

Calibre will announce the release in the next hour or so. If there are any issues, please report them here.

03-09-2021, 10:45 AM	#919
hollowpoint Groupie Posts: 155 Karma: 6672188 Join Date: Apr 2019 Device: Ipad Mini, Kindle PW5, Libra 2	Ok thanks. I noticed both the Android and iOS Kindle apps have an export function for annotations that emails you an HTML file with the annotations data. It's pretty clean output, very predictable and compliant HTML. Plus the Kindle itself will email you the annotations in the form of CSV and PDF, as well. Wonder if it'd almost be easier to write a plugin that consumes the HTML and CSV files, and import the annotations that way. Then you'd have no device connections to deal with, just the 2 file types for input options. One could use the Pandoc plugin to clean up the input or convert it to something else, before it gets stuffed into a Calibre custom column. As it's a one-way sync (it only imports the annotations to my knowledge, it doesn't look like they get synced back to connected devices), the option of importing file input only versus having to worry about app, device, and OS support, seems vastly simplified. ETA: The problem with importing the HTML or CSV annotations output, I just remembered, is the fact that users can only export one book at a time. I haven't yet seen a way to "export all annotations from all books." If there was a way to do that, this would seem such a streamlined way to get annotations into Calibre. Last edited by hollowpoint; 03-09-2021 at 12:01 PM.

04-01-2021, 05:15 AM	#926
robjlucas Junior Member Posts: 1 Karma: 10 Join Date: Apr 2021 Device: Kindle	Wizard disabled Firstly, thank you very much for providing this plugin; your work is much appreciated. I couldn't believe it when I discovered there was no built-in way of exporting my annotations from non-DRMed documents in the Kindle. However, I'm hitting a basic stumbling block with setup: Running Calibre 4.99 on Ubuntu 20.04, with a connected Kindle, from which I am trying to fetch annotations: 1. I select the document that is on the Kindle, click "Annotations" > "Fetch annotations from connected device" 2. I get the error message "Unable to import annotations as the annotations column has not been configured..." 3. To set up the column, I click "Annotations" > "Customize plugin...". 4. Under "Add fetched annotations to", "Comments" is preselected; clicking the custom column wizard icon next to this does nothing, and the icon seems to be greyed out. Is there some way around this, such as adding the appropriate column manually? One would, presumably need to know the technical requirements for such a column.

04-03-2021, 11:55 PM	#930
davidfor Grand Sorcerer Posts: 24,906 Karma: 47303748 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	Update to version 1.15.0 I have just released version 1.15.0 of the plugin. This is a bug-fix release that covers the beta's since the last release. The fixes in this release are: Fix: Spanish My Clippings.txt has "La" for the annotations. Fix: Change method of getting books on device to use a search similar to what is used for Kobo and Tolino devices. Fix: Sorting of annotations for kepubs on Kobo devices. Fix: Opening My Clippings.txt should have been binary. Missed this when reverted earlier change. Fix: Display more details of an exception when parsing My Clippings.txt. Fix: Goodreader import not working. Fix: Broke Python 2 support with the open when reading My Clippings.txt. Fix: If already have book id in library, use confidence of 5. Fix: Encode and decode the title from My Clippings.txt to remove the Unicode BOM character. Fix: A couple of changes to solve some obvious problems with the Kindle fetch. Plus more logging. Fix: Python 3 compatibility: Help dialogs. Fix: Sorting columns in fetched annotations dialog. Calibre will announce the release in the next hour or so. If there are any issues, please report them here.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] KindleUnpack - The Plugin	DiapDealer	Plugins	494	05-18-2024 02:34 AM
[GUI Plugin] Open With	kiwidude	Plugins	403	04-01-2024 08:39 AM
[GUI Plugin] Annotations (closed)	GRiker	Plugins	68	06-17-2014 06:11 AM
[GUI Plugin] KiNotes	-axel-	Plugins	0	07-14-2013 06:39 PM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 12:27 PM

03-08-2021, 12:13 PM	#917
hollowpoint Groupie Posts: 155 Karma: 6672188 Join Date: Apr 2019 Device: Ipad Mini, Kindle PW5, Libra 2	Hey David, thanks for your work on this extension. I'm running Win10, Calibre 5.12(x64), Annotations 1.14. Cannot get Annotations to work with any of the following 3 devices. Despite creating the annotations column, etc. I have had it working in the past, though not with these newer devices. Issue per device: -PW4 (5.13.4). Connect by USB, device recognized and can see books, Annotations tries and says it cannot import any ("No annotated books found on device: Unable to find any annotations on Amazon Kindle matching books in your library."). There are definitely annotations on certain Kindle books that are synced between all these devices, and there's also a MyClippings.txt file with some annotations--opened and checked it. -Samsung Galaxy phone (Android 11). Annotations are all synced to books in Kindle. Device connected by USB and recognized by Calibre. However "Fetch annotations" option is grayed out. -Ipad Mini v5 (iOS 14.14). Connected by lightning cable. Device not recognized by Calibre. So, the questions boil down to these: - Is there any way to attach one's current-gen smart devices (Android 10/11, iOS 14.x) to Calibre and directly import the synced Kindle annotations? - Thots on what might be blocking the PW4 from importing annotations? This one I expected to work, even though I had doubts about the smart devices.

03-12-2021, 10:32 PM	#921
mdtanos Junior Member Posts: 2 Karma: 10 Join Date: Mar 2021 Device: none	is it possible to highlight a book from the calibre ebook viewer and have it synced with the same My Clippings.txt from kindle? so you can switch the reading session back and forth between desktop and kindle device and still keep highlights organized? thank you very much!

03-26-2021, 03:02 PM	#924
C Alberga Enthusiast Posts: 38 Karma: 10 Join Date: Jul 2012 Device: none	Should I report a possible "Annotations" error here? If not, where?

04-01-2021, 09:41 PM	#928
enaid1 Member Posts: 12 Karma: 6256 Join Date: May 2017 Device: iPhone	Thank you so much for this plugin! (At first I couldn't make it work, but then I tried the most recent beta and everything worked perfectly...) Question: is there a way to mass delete annotations from a book (whether downloaded from the Kindle store or side loaded)? I am using my Kindle to read books in the language I am learning and I highlight a bunch of words to download and make flash cards out of. The only way to delete these that I've found so far is one at a time (such as on the Amazon Highlights page).