[GUI Plugin] Annotations

[Quoth]

The note seems to precede the highlight in "my clippings.txt" which seems odd.
A note without being associated with the highlighted source is useless.

You can have highlights without a note, but not vice-versa.

[Katja_hbg]

hi @davidfor
In an other thread you wrote «I actively discourage use of the "Copy annotations for Selected Books" function in my Kobo Utilities plugin. It is ugly, something I put together when I was first learning about this, and if I could, would drop. Each time someone mentions they use it, I am surprised.»
I switched from Kobo Utilities to Annotation after it shows all that strange additional lines.
After some setting learning I am now ok with the new method.

The only disadvantage of Annotation is the shortkey as I need to name one for each device.
I would prefer, like for KU, to have one shortkey running the annotation fetch. There is only one advice active at the the same time.

Could you imagine to provide one single shortkey to fetch independent of the connected device?

[davidfor]

Originally Posted by Katja_hbg

hi @davidfor
In an other thread you wrote «I actively discourage use of the "Copy annotations for Selected Books" function in my Kobo Utilities plugin. It is ugly, something I put together when I was first learning about this, and if I could, would drop. Each time someone mentions they use it, I am surprised.»
I switched from Kobo Utilities to Annotation after it shows all that strange additional lines.
After some setting learning I am now ok with the new method.

The only disadvantage of Annotation is the shortkey as I need to name one for each device.
I would prefer, like for KU, to have one shortkey running the annotation fetch. There is only one advice active at the the same time.

Could you imagine to provide one single shortkey to fetch independent of the connected device?

I hadn't thought of that. You can have only one device connected at a time, so using only one shortcut key makes send.

But, how did you set the shortcut key? The plugin isn't listed in the calibre preferences. I'm sure there is another way to set them, but, it escapes me at the moment.

[Katja_hbg]

oh sorry, I am wrong.
I did not find a possibility to add ashortcut. Of course I like to have one.
In the favourites plugin it is possible to add function of loading annotation. And this dependents from device i.e. two devices are two entries, which make no sense but does not matter. I mixed up this with shortcut.

Sorry for the confusion.
Do you like to offer a shortcut?

[davidfor]

Originally Posted by Katja_hbg

oh sorry, I am wrong.
I did not find a possibility to add ashortcut. Of course I like to have one.
In the favourites plugin it is possible to add function of loading annotation. And this dependents from device i.e. two devices are two entries, which make no sense but does not matter. I mixed up this with shortcut.

Sorry for the confusion.
Do you like to offer a shortcut?

I did think of the Favourites Plugin, but, when I looked, the fetch wasn't there. I just remembered that only currently available menu options are displayed. I needed to connect a device to see the fetch. I'll have a look at what can be done there. I have a feeling it won't be possible.

The plugin should be registering shortcuts. My other ones do, so I'll add this. But, the fetch is a little trickier as it would be best to register a single short cut for "Fetch from connected device". I have no idea if that is possible without actually displaying the menu item. Which I might do and have it disabled when a suitable device is not connected.

[Katja_hbg]

«The plugin should be registering shortcuts. My other ones do, so I'll add this. But, the fetch is a little trickier as it would be best to register a single short cut for "Fetch from connected device".»
I am looking forward for the shortcut. Favourite is only nice to have for me.
The KU-fetch works with one single Shortcut, that's why I miss that comfort.
Thanks for looking into that.

David, a couple of questions about using the plugin with larger ebooks:

1. If you need to import annotations on a book a second time (because you added more annotations after your first import to Calibre), I assume the best way to handle that is 'delete' all the existing annotations from the details view in the book, and then run import again?

2. This one is weird, a special case question. :-) Say that you have a book that you read a LOT, it's huge, you add annotations all over the place. And then you want to transfer the book, with the on-device annotations, to another Kobo device or app. Is there a way to do that (even as a manual hack) with a sideloaded book not purchased from Kobo? (I realize the simplest solution would be: if you want to read the book with notes on multiple Kobo devices that bad, just buy the book from Kobo in the first place. Problem is, I already owned some of the books that I want to do this with before I discovered Kobo devices :-) ).

*ETA: I see how you do item 2, it's basically munge the book and annotations data from the KoboReader.sqllite db file on the device, and move it onto another Kobo device. Here's an older thread how one user did it: https://www.mobileread.com/forums/sh...d.php?t=312974

[davidfor]

Originally Posted by maximus83

David, a couple of questions about using the plugin with larger ebooks:

1. If you need to import annotations on a book a second time (because you added more annotations after your first import to Calibre), I assume the best way to handle that is 'delete' all the existing annotations from the details view in the book, and then run import again?

You can, but the plugin should merge the annotations. It should have enough information to update any changed annotations and add new ones. It won't delete them.

2. This one is weird, a special case question. :-) Say that you have a book that you read a LOT, it's huge, you add annotations all over the place. And then you want to transfer the book, with the on-device annotations, to another Kobo device or app. Is there a way to do that (even as a manual hack) with a sideloaded book not purchased from Kobo? (I realize the simplest solution would be: if you want to read the book with notes on multiple Kobo devices that bad, just buy the book from Kobo in the first place. Problem is, I already owned some of the books that I want to do this with before I discovered Kobo devices :-) ).

*ETA: I see how you do item 2, it's basically munge the book and annotations data from the KoboReader.sqllite db file on the device, and move it onto another Kobo device. Here's an older thread how one user did it: https://www.mobileread.com/forums/sh...d.php?t=312974

That is the way I would do it. You can also copy the annotations file from the "Digital Editions/Annotations" directory. That will load the annotations the first time the book is loaded on the new device. But, it loses the name of the chapter the annotations are in.

Originally Posted by davidfor

You can also copy the annotations file from the "Digital Editions/Annotations" directory. That will load the annotations the first time the book is loaded on the new device. But, it loses the name of the chapter the annotations are in.

Oh interesting, that sounds a bit easier.

Is it also possible to use a PC client app, like ADE or Kobo, to locally sync the book data from the source device, then attach the new Kobo device and sync to that, all local? Or does 'local sync' not work with sideloaded books in those apps?

[davidfor]

Originally Posted by maximus83

Is it also possible to use a PC client app, like ADE or Kobo, to locally sync the book data from the source device, then attach the new Kobo device and sync to that, all local? Or does 'local sync' not work with sideloaded books in those apps?

Kobo doesn't sync anything about sideloaded books. You can add them to the desktop app, but, they don't get synced to the device.

For ADE, I don't know. I don't think it shows the annotations for sideloaded books on the device. It is possible it will for books sent from ADE, but, I don't do that. I pretty much only use ADE to download DRM protected books. I will occasionally send a book from it to the device to check that this works, but that's about all.

Originally Posted by davidfor

For ADE, I don't know. I don't think it shows the annotations for sideloaded books on the device. It is possible it will for books sent from ADE, but, I don't do that. I pretty much only use ADE to download DRM protected books. I will occasionally send a book from it to the device to check that this works, but that's about all.

You are right on ADE--just tested. It will sync local sideloaded books to and from devices, but ignores the device annotations.

Would it be reasonable to request a 'migrate-sideloaded-kobo-books-with-annotations-to-new-kobo-device' script or plugin? Or ping Kobo itself to offer this? Frankly, I'm surprised they haven't already provided this. They are a smart company, they built the reader so folks could sideload books obtained elsewhere. You'd think they'd provide an easy, built-in way to synchronize reading data for sideloaded books to a new Kobo device, that's in their best interest.

I mean, I can use that hack when the time comes, but probably a lot of users won't, and all the other big ebook providers make this much easier to do. Also, I'd think a lot of users would like a simple way to BACK UP their reading and annotations data auto-magically in the cloud (even for sideloaded books) so there's no risk of losing it should the current device be lost/damaged.

Right now given the situation with how annotations are so 'device locked' on Kobo, I'm considering doing something I've heard some users here say they do: only put sideloaded books on the Kobo, don't use their cloud and don't use local annotations either. When I want to annotate books and sync between devices, buy them from Google Play or sideload them into their books app, and use that on PC or phone to add annotations. That last part is kind of what I have been doing, for some time. And this approach should work, because I only need to annotate a subset of the books I read, maybe 1/4.

It's a workaround, and I realized something like that was gonna be necessary when I chose the Kobo over a Kindle. But it's fine, I like the device so much better for just relaxed reading and it's easier taking annotations on a PC (typing) or Android device (swiping) than trying to take notes on a clunky e-ink keyboard, anyway. :-) That said, I still think it'd be a win for Kobo's business to provide easier cross-device sideloaded book data migration for users--it'd give users an easier path to upgrade devices and stick with Kobo in the future.

[Savate]

HI - I finally figured out how to use this plug-in!

(1) I only realised that halfway through your help instructions (at the beginning of this htread) you mention 'clippings.txt' but it was not clear that this plug-in totally relies on it. I had to read through several times before the penny dropped.

I had deleted my clippings file recently (it was getting large) and it took a while to figure out that the plugin wasn't finding my annotations because the clippings file did not match the books in my library!

Maybe a clear statement at the beginning that this plugin uses the clippings.txt file so do not delete it until you have copied the annotations in to Calibre?

It's not obvious to dummies like me!

(2) The annotations list appears in a window of its own - very useful but it cannot be resized! Could this be fixed? (I can drag column widths but this does not change the overall window width.)

(3) Similar point: could the annotations window remain open until it's closed by the user? If I select one book to import the window is automatically closed so I must reload all the annotations to select another book.

(4) My annotations show 'Unknown' instead of the book title. It is not clear how to change that. Also, it would be useful to make explicit the labels 'Highlight', 'Note' and 'Bookmark' as well as the location. These are in the clippings file but not added to the annotations although this is important information. Can this be done? (e.g. add further regex options).

[Note: this may already be possible but I can't see how to do it]

(5) A luxury item: it would be handy to have a 'save to text file' function on the annotations window (I realise that it has copy so that I can paste highlights into an application - but sometimes I need to do several at one sitting it would be just easier). However, this is a luxury item so not essential (unlike 2 - 4 above).

Overall this is a really useful plugin for a reader who keeps a lot of annotations and notes.

Thanks

[davidfor]

Originally Posted by Savate

HI - I finally figured out how to use this plug-in!

(1) I only realised that halfway through your help instructions (at the beginning of this htread) you mention 'clippings.txt' but it was not clear that this plug-in totally relies on it. I had to read through several times before the penny dropped.

I had deleted my clippings file recently (it was getting large) and it took a while to figure out that the plugin wasn't finding my annotations because the clippings file did not match the books in my library!

Maybe a clear statement at the beginning that this plugin uses the clippings.txt file so do not delete it until you have copied the annotations in to Calibre?

It's not obvious to dummies like me!

There is a section labelled "Typical Kindle workflow". That states the details are coming from the "My Clippings.txt" file. I'm not immediately sure how to make it clearer.

(2) The annotations list appears in a window of its own - very useful but it cannot be resized! Could this be fixed? (I can drag column widths but this does not change the overall window width.)

The "Import annotations" window is sizable here. As are all the other windows in the plugin. You should be able to grab the edges of the windows and drag them.

(3) Similar point: could the annotations window remain open until it's closed by the user? If I select one book to import the window is automatically closed so I must reload all the annotations to select another book.

I have no idea why you would want to do this. The annotations list window should show the books it found annotations in. Select each with a check-box, and press the "Import annotations" button. The annotations will be applied to all the books listed. Depending on how well the book and title in the clippings file matches the books in the library, you might get prompted to confirm the book is correct.

The book you select before starting is only used if matches can't be find. The annotations that aren't otherwise matched can be added to this book.

(4) My annotations show 'Unknown' instead of the book title. It is not clear how to change that.

Where do you see this? I the "Import annotations" window? If so, I can't think of how it happens unless there is a problem in your library and this is the title of a book.

Or is it in the annotations? "Unknown" is used here if the plugin cannot determine the location of the annotation. It should be a page number or Kindle location, but if there is a problem getting these from the clippings file, "Unknown" will be used. If that is the case, there might be a problem with the file.

Also, it would be useful to make explicit the labels 'Highlight', 'Note' and 'Bookmark' as well as the location. These are in the clippings file but not added to the annotations although this is important information. Can this be done? (e.g. add further regex options).

[Note: this may already be possible but I can't see how to do it]

The plugin uses different styles for each. You can set this in the configuration.

(5) A luxury item: it would be handy to have a 'save to text file' function on the annotations window (I realise that it has copy so that I can paste highlights into an application - but sometimes I need to do several at one sitting it would be just easier). However, this is a luxury item so not essential (unlike 2 - 4 above).

I'll consider putting a "Copy to clipboard" option for the selected books in the list. But, the question then becomes exactly what to copy and the format to use. But, I don't know when I will do this.

[Savate]

Thanks for the reply

On point (1) It might be clearer to users like me (for whom the Kindle is a bit of black box) to state at the beginning of the guidance that the plugin uses the clippings.txt file. So, if that file is incomplete (or worse entirely missing if the Kindle has been reset) then the plugin has no data to work on.

In spite of a mention in the 'workflow' section, it took a few tries to work out why I wasn't getting all my annotations because, as I said, I had deleted an earlier clippings file and started a new one. Anyway, the point is a bit moot I guess because now that I know it's become obvious but it wasn't obvious to begin with (and others might be caught out).

On point (2) - blimey! You are right. I have no idea why but when I was using this earlier (before I posted) I simply could not resize the window by dragging window borders as per normal window behaviour. You just have to take my word for that! I have restarted all software, cleaned out system files, restarted the PC and now it's working. Very puzzling but 'my bad' as they say. All fine now, so apologies for the stupid observation :-)

On point (3) - well this might be a matter of operational taste. I have over 300 books in my library and so prefer to be selective about which annotations I import at any given moment; maybe I don't want to add them all at once but one or two at a time and keeping the annotations window open is a convenience. But it's OK, I can live with the way it is. A moot detail and not worth the work. :-)

On point (4) - this still seems to be a problem for me. I have reset my Kindle and have a new, clean clippings file.

I attach two sample files. One is the clippings text containing a highlight from the book Murder on the Orient Express (a kfx file), a note and a bookmark.

The second is an image of the annotations as they appear in my Calibre 'annotations' field after importing to Calibre.

Neither the labels nor the location information show up in the imported version (see the image file) and the book title is simply displayed as' 'Unknown'.

You note that "The plugin uses different styles for each. You can set this in the configuration", but I have no idea how I could use CSS to display the labels and location info (but happy to change colours, margins etc.). What I would like here is to include the book title and the labels 'highlight', 'note' or 'bookmark' along with the location value as they appear in the the clippings file. I can't see how to do that with CSS (but presumably this happens in the the Python code?).

Ignore point (5)!

Thanks for any further advice.

[davidfor]

Originally Posted by Savate

On point (1) It might be clearer to users like me (for whom the Kindle is a bit of black box) to state at the beginning of the guidance that the plugin uses the clippings.txt file. So, if that file is incomplete (or worse entirely missing if the Kindle has been reset) then the plugin has no data to work on.

In spite of a mention in the 'workflow' section, it took a few tries to work out why I wasn't getting all my annotations because, as I said, I had deleted an earlier clippings file and started a new one. Anyway, the point is a bit moot I guess because now that I know it's become obvious but it wasn't obvious to begin with (and others might be caught out).

I'll look at it again, but, I'm not sure how to make it clearer.

On point (3) - well this might be a matter of operational taste. I have over 300 books in my library and so prefer to be selective about which annotations I import at any given moment; maybe I don't want to add them all at once but one or two at a time and keeping the annotations window open is a convenience. But it's OK, I can live with the way it is. A moot detail and not worth the work. :-)

Sorry, not a change I'm interested in making. If I was making any changes there, I would be changing the import window to a modal dialog which means you couldn't interact with the main calibre window when it was opened.

On point (4) - this still seems to be a problem for me. I have reset my Kindle and have a new, clean clippings file.

I attach two sample files. One is the clippings text containing a highlight from the book Murder on the Orient Express (a kfx file), a note and a bookmark.

The second is an image of the annotations as they appear in my Calibre 'annotations' field after importing to Calibre.

Neither the labels nor the location information show up in the imported version (see the image file) and the book title is simply displayed as' 'Unknown'.

It's a Kindle firmware change. Try this beta. That should fix it. And I do have to release that properly.

You note that "The plugin uses different styles for each. You can set this in the configuration", but I have no idea how I could use CSS to display the labels and location info (but happy to change colours, margins etc.). What I would like here is to include the book title and the labels 'highlight', 'note' or 'bookmark' along with the location value as they appear in the the clippings file. I can't see how to do that with CSS (but presumably this happens in the the Python code?).

Try the beta and look at the results. But, adding the book title and author doesn't make sense as this is already part of the metadata when you view it in calibre. I don't know that adding the type helps either. The information displayed below tells me what it is.