[GUI Plugin] Annotations

[davidfor]

Originally Posted by GregLeBlonde

Thanks for your response. On a 30+ page thread that covers many iterations of the plugin it is tough to find of a question has been asked previously and know whether that answer is still true. If it is a common question, maybe it should be added to the FAQ on the plugin page; that was the first place I looked.

The first post of the thread says what the plugin does. And some details of how to use it. Nothing I can see in that suggests can send them back to the device. Does it need to be clearer? I'm loath to add a "This plugin doesn't" section, as that just means more to maintain without any real, to my mind, benefit.

[davidfor]

In this thread, the fetch from a Kindle is not working. From the sample "My Clippings.txt" file, it appears to have lower case for some of the terms being searched for to get the location. The attached beta is a quick fix for this.

[Quoth]

P.S. Thanks for quick response davidfor
Obviously it's closer to being useful for me. See other thread.

[Quoth]

I've tried the Kobo H2O again, with the book now on SD card and the Annotation Plugin version just posted:

​Chapter 1: Out of the Cold and Into the Darkness	Thu 10 Jan 2019 12:44:02 AM
chameleonic armour.

del ic?
​Chapter 1: Out of the Cold and Into the Darkness	Thu 10 Jan 2019 12:46:13 AM
actually being in control.

Style?

Just two of the 400+ annotations. The datestamp is correctly only once per annotation, but I can't see how to remove the blank paragraph between the highlight and my note. I'd rather it was after the note. I looked at and experimented with the CSS dialog in configuration.
The Kobo Utilities annotation plug in also gives a % progress and has "chapter" twice, first is internal number and second is the TOC/Heading text for the chapter (curiously the first number can start at 0 or have strange numbers for sections "pasted" after the main novel not on the TOC.) The Kobo Utilities version is faster. It used to give good format, now has a load of extra paragraphs and indents, but a few global Search & Replace fixes it in Sublime or Notepad++ (copied via clipboard and saved as plain text).

The other thread has current progress with the PW3.

[Dfaure]

Hi,
I just found this: https://www.albertopettarin.it/exportnotesodyssey.html

Which may be worth getting your plugin to work with Bookeen e-readers.

[davidfor]

Originally Posted by Dfaure

Hi,
I just found this: https://www.albertopettarin.it/exportnotesodyssey.html

Which may be worth getting your plugin to work with Bookeen e-readers.

That is basically just a script to parse the annot files that the Adobe RMSDK produces. Which means the Bookeen ereaders are using the RMSDK for reading epubs. Which is not that surprising as it is the only way that I know of to support books with Adobe DRM.

There is still a lot to do before it can be used. It reads only one annot file. But, I would expect there is one file per book that has been opened. The Kobo devices do this for epubs, as well as their database which the plugin reads.

Then there needs to be a way to map the annotations back to the books. Unless there is a database somewhere, that can be a pain and inaccurate. It depends on how they are stored and how the driver for the device handles it.

If someone wants to add support for the Bookeen devices, then this is probably the way to go. I'll be happy to help, but, without the device in my hands, it's not something I am interested in doing myself.

This does suggest something else, and I'm a little embarrassed I didn't think of it before. This could be used to add generic support for RMSDK readers. Or at least a framework to support them. That is a bit more interesting to me. But, again, it isn't something I'm likely to do without some help from someone with a device that would use it. At the least, someone is needed for testing. Ideally, I would like someone with the device for writing the device specific code.

[cheshyre]

I'm using the plugin with Calibre for Windows and a Kobo eReader.
The "Import Annotations" window is 3268 pixels wide and cannot be resized any narrower.
Since my display resolution is only 1920 px wide, it ends up larger than my screen.
I can either read the titles (left-column) or see the buttons (lower right), but not both.

[davidfor]

Originally Posted by cheshyre

I'm using the plugin with Calibre for Windows and a Kobo eReader.
The "Import Annotations" window is 3268 pixels wide and cannot be resized any narrower.
Since my display resolution is only 1920 px wide, it ends up larger than my screen.
I can either read the titles (left-column) or see the buttons (lower right), but not both.

The dialog should be sizable, or at least it is here. You should be able to grab the edges and resize it. And the size is stored when you press the OK button.

[Quoth]

Yes, I can resize it.
Still no use for Kindle PW3 unlike Kobo H2O every note & highlight is after the location, like in my clippings. Though the formatting is terrible the Kobo Utilities Fetch Annotation works better, it only needs a few global edits to fix it into a nice text file. Simply copying My cllppings.txt for the Kindle PW3 is faster and just as functional as this Annotations plug in.

Also there is limit on how many annotations this plugin will read from Kobo. I'm up to 1352 at 51% of the book and the KoboUtilities fetch gets them all, but this Annotations Plugin truncates at some point.

I've abandoned the idea of switching to PW3 for proofing as the imported info is poorer than Kobo Utilities which gives logical chapter, chapter title text, percent into chapter etc. The PW3 is much nicer to select, but I've found using the SD card and wider line spacing the text selection is now only a little slower on Kobo H2O than it used to be, though speed is erratic.

Selection is just wonderful on the Kindle PW3, pity the actual extracted data is poor.

[davidfor]

Originally Posted by FrustratedReader

Yes, I can resize it.
Still no use for Kindle PW3 unlike Kobo H2O every note & highlight is after the location, like in my clippings. Though the formatting is terrible the Kobo Utilities Fetch Annotation works better, it only needs a few global edits to fix it into a nice text file. Simply copying My cllppings.txt for the Kindle PW3 is faster and just as functional as this Annotations plug in.

The issue is that you are interpreting the contents of the My Clipping.txt file differently to the plugin, and apparently every other user of the plugin. The plugin is reading each section in the file and putting those into separate elements in the generated HTML. This means there is an element for each "Your Highlight" and "Your Note" in the file. You seem to be saying that these are actually connected together and should be treated as a highlight with a note. Looking at your examples, that doesn't seem unreasonable, but, it isn't completely obvious how they are connected. Also, the plugin has been working this way for a long time and no-one has reported an issue with this. Unless the file format has changed recently, I am loath to change the plugin without more reports that it is wrong.

Also there is limit on how many annotations this plugin will read from Kobo. I'm up to 1352 at 51% of the book and the KoboUtilities fetch gets them all, but this Annotations Plugin truncates at some point.

Not that I can think of. Is the code in the field complete? If not, it is probably a limitation on the long text field being used in calibre.

I've abandoned the idea of switching to PW3 for proofing as the imported info is poorer than Kobo Utilities which gives logical chapter, chapter title text, percent into chapter etc. The PW3 is much nicer to select, but I've found using the SD card and wider line spacing the text selection is now only a little slower on Kobo H2O than it used to be, though speed is erratic.

Selection is just wonderful on the Kindle PW3, pity the actual extracted data is poor.

A small correction: The extracted data is only as poor as the available data.

Edit:
I have fixed at least one of the problems you have mentioned. I just haven't gotten to finish testing it. Plus I'm still completely baffled about where the extra line you keep pointing at is coming from.

[Quoth]

The sample CSS view for the editing of Annotations appearance shows:
Location heading/Chapter + date-time
Highlight.
Note.

That is indeed what I get if I use this plugin with the Kobo. Though I find the Kobo Utilities is better.

What I was getting with the PW3 is

Location number only + date-time
Highlight with following Blankline (can't be moved)
Location number only + date-time
Note.

I'm now getting the blank line at the correct place!

378	Sun 13 Jan 2019 12:04:32 AM
silent for a while. “That would be acceptable, now we

379	Sun 13 Jan 2019 12:05:37 AM
" agreed Leodith, "
379	Sun 13 Jan 2019 12:05:38 AM
be acceptable, now we

390	Sun 13 Jan 2019 12:07:16 AM
a
390	Sun 13 Jan 2019 12:07:17 AM
have tape somewhere

406	Sun 13 Jan 2019 12:08:57 AM
aall
406	Sun 13 Jan 2019 12:08:58 AM
Anything at personal from

Having location numbers are useless on their own. I need chapter names.

Essentially the Location number only + date-time is the same for pairs of Highlight + note. You can have a highlight with no note.

I can't see why the Annotation plugin has LESS info than my clippings and repeats the heading as in My Clippings.





Note, I'm not interested in storing the annotations in Calibre, I have Notepad++ or SublimeText on left side of screen and Libre Office Writer on the right.
I mark off in the text file what I've done, in front of the note * = source edited, X = note discarded. Or some other comment.

For the import to Calibre or upload to Web, the header and footer removed, body is fully justified, TOC has page numbers removed etc.

[Quoth]

For the Kindle PW3 (aka PW 2015) does this plug in simply read "my clippings.txt"? If so, then I can process that directly without a Calibre Plugin.

[davidfor]

Originally Posted by FrustratedReader

For the Kindle PW3 (aka PW 2015) does this plug in simply read "my clippings.txt"? If so, then I can process that directly without a Calibre Plugin.

YESSSS!!!!!!!!!!!!!!!!

That is all it does. And I have said that in most of my responses to you about this. And I have explicitly stated that what you get is only what is in this file.

And for your previous post, this plugin is ONLY about storing the annotations in calibre. NOTHING else. If it happens to help for something else, then good. And I will consider some tweaks to help that, but, not if they interfere with the purpose of the plugin.

In your previous post, you state that the plugin gets less info from the My Clippings.txt file, but, you haven't stated what. And your samples seem to show that the information does match.

And yes, it is possible that there are highlights and notes in the My Clippings.txt file that can be connected together. But, as I stated in my last post, NO ONE as complained about this before. This plugin has been working this way for years. If I am to make a change like this, then I need confirmation from others that this conclusion is correct, that it works and is the desired behaviour for more than just one person.

[Quoth]

the annotation import plugin only shows first location number, not page or range.

Snippet from "my clippings.txt"

Séadna, Kevin, Órlaith, Shoshanna Muirí, Beth and her
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Highlight on page 193 | location 2957-2958 | Added on Tuesday, 15 January 2019 02:49:44

Beth and her husband Niall. Anrhi
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Note on page 193 | location 2958 | Added on Tuesday, 15 January 2019 02:50:24

Niall or Brian?
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Highlight on page 194 | location 2960-2961 | Added on Tuesday, 15 January 2019 02:51:02

Órlaith’s parents, Danny and Laragh. The two older
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Note on page 194 | location 2961 | Added on Tuesday, 15 January 2019 02:51:50

De two older
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Highlight on page 194 | location 2960-2961 | Added on Tuesday, 15 January 2019 02:51:50

The two older
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Highlight on page 194 | location 2961-2961 | Added on Tuesday, 15 January 2019 20:54:18

frail old Lord Conor Aldiare.
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Note on page 194 | location 2961 | Added on Tuesday, 15 January 2019 20:55:00

HTML source from Calibre Metadata

<tbody><tr><td class="location" style="text-align:left">2958</td><td class="timestamp" uts="1547520624.0" style="text-align:right">Tue 15 Jan 2019 02:50:24 AM </td></tr></tbody></table><p class="note" style="font-size:100%;margin:2.0;">Niall or Brian?</p></div><div class="annotation" genre="Romance, Gothic fiction (Literary genre), fantasy adventure" hash="b6d73458e76e3cbc299260d751591216" location_sort="002960" reader="Kindle" style="margin:0 0 0.5em 0"><table cellpadding="0" width="100%" style="background-color:LightGray;color:black;font-size:75%;margin:0em;" color="Gray"><tbody><tr><td class="location" style="text-align:left">2960</td><td class="timestamp" uts="1547520662.0" style="text-align:right">Tue 15 Jan 2019 02:51:02 AM </td></tr></tbody></table><p class="highlight" style="margin:1.4em;">Órlaith’s parents, Danny and Laragh. The two older</p></div><div class="annotation" genre="Romance, Gothic fiction (Literary genre), fantasy adventure" hash="cbe2b96e03205670f9625c177b9c2707" location_sort="002960" reader="Kindle" style="margin:0 0 0.5em 0"><table cellpadding="0" width="100%" style="background-color:LightGray;color:black;font-size:75%;margin:0em;" color="Gray"><tbody><tr><td class="location" style="text-align:left">2960</td><td class="timestamp" uts="1547520711.0" style="text-align:right">Tue 15 Jan 2019 02:51:51 AM </td></tr></tbody></table><p class="highlight" style="margin:1.4em;">The two older</p></div><div class="annotation" genre="Romance, Gothic fiction (Literary genre), fantasy adventure" hash="aad16432f5fb0a5fdce48d6971cb4ed8" location_sort="002961" reader="Kindle" style="margin:0 0 0.5em 0"><table cellpadding="0" width="100%" style="background-color:LightGray;color:black;font-size:75%;margin:0em;" color="Gray"><tbody><tr><td class="location" style="text-align:left">2961</td><td class="timestamp" uts="1547520710.0" style="text-align:right">Tue 15 Jan 2019 02:51:50 AM </td></tr></tbody></table><p class="note" style="font-size:100%;margin:2.0;">De two older</p></div></div>

Text from Metadata

2957	Tue 15 Jan 2019 02:49:44 AM
Beth and her husband Niall. Anrhi

2958	Tue 15 Jan 2019 02:50:24 AM
Niall or Brian?
2960	Tue 15 Jan 2019 02:51:02 AM
Órlaith’s parents, Danny and Laragh. The two older

2960	Tue 15 Jan 2019 02:51:51 AM
The two older

2961	Tue 15 Jan 2019 02:51:50 AM
De two older

I actually find your explanations and replies hard to follow, so it wasn't clear that all you were doing was reading "my clippings.txt", though it seems very slow to be only doing that. I thought you'd found access to something else asiIt's not even accurately copying what is there, compared with the Kindle DXG I had. I can't test it as I passed it on to someone with Macular Degeneration.
I hope to have an old 6" Kindle Keyboard soon to test mobi formating. Someone getting the current basic Kindle.

So anyway, I'm back to proofing people's texts on the Kobo as it's now at least usable again with combination of larger margin, line spacing and SD Card. Though poorer than when I bought it. Sadly though the Kindle is great for source selection and note typing compared to Kobo, the "my clippings.txt" is too limited lacking chapter & chapter %. Page numbers are a bit stupid on eBooks. The "location" might be good if it was something 100% reliable based on paragraph count. I've no idea what it is, except it's not affected by device screen, font size or page settings.

It's up to you what you change, but certainly it's needlessly repeating identical time stamps for note and source highlight, contrary to CSS preview and only putting location, not location range or page, at least I think so.

Maybe you need a friend with a Kindle PW3 or something using current FW to test.

I thought of writing a program to process "my clippings.txt"

Edit:
The "my clippings.txt" clip is newer than the Calibre import as it ends that book with

==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Highlight on page 282 | location 4320-4320 | Added on Tuesday, 15 January 2019 23:35:03

including yourselves if we take on the obligation of supplying the Reservations.”
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Note on page 282 | location 4320 | Added on Tuesday, 15 January 2019 23:35:10

Fix
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Highlight on page 283 | location 4332-4332 | Added on Tuesday, 15 January 2019 23:35:56

“You’ll have to see. Don’t
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Note on page 283 | location 4332 | Added on Tuesday, 15 January 2019 23:36:42

They are going to Emarineer sehortly

and the parts from Calibre are at the end.

However you can see the rather useless Page is missing (can be used I suppose on Kindle if no settings changed) and that locations for highlight can have a range.

[davidfor]

Originally Posted by FrustratedReader

the annotation import plugin only shows first location number, not page or range.

Yes, you are right, it is only taking one of the location descriptions. I think there is a situation where it uses the page number if there is no location number. And yes, it just uses the start location from the range. This is consistent with the other devices and apps the plugin can fetch annotations from. And I'm not sure if there is an actual advantage in having the end of the range.

Snippet from "my clippings.txt"

Séadna, Kevin, Órlaith, Shoshanna Muirí, Beth and her
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Highlight on page 193 | location 2957-2958 | Added on Tuesday, 15 January 2019 02:49:44

Beth and her husband Niall. Anrhi
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Note on page 193 | location 2958 | Added on Tuesday, 15 January 2019 02:50:24

Niall or Brian?
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Highlight on page 194 | location 2960-2961 | Added on Tuesday, 15 January 2019 02:51:02

Órlaith’s parents, Danny and Laragh. The two older
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Note on page 194 | location 2961 | Added on Tuesday, 15 January 2019 02:51:50

De two older
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Highlight on page 194 | location 2960-2961 | Added on Tuesday, 15 January 2019 02:51:50

The two older
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Highlight on page 194 | location 2961-2961 | Added on Tuesday, 15 January 2019 20:54:18

frail old Lord Conor Aldiare.
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Note on page 194 | location 2961 | Added on Tuesday, 15 January 2019 20:55:00

HTML source from Calibre Metadata

Text from Metadata

2957	Tue 15 Jan 2019 02:49:44 AM
Beth and her husband Niall. Anrhi

2958	Tue 15 Jan 2019 02:50:24 AM
Niall or Brian?
2960	Tue 15 Jan 2019 02:51:02 AM
Órlaith’s parents, Danny and Laragh. The two older

2960	Tue 15 Jan 2019 02:51:51 AM
The two older

2961	Tue 15 Jan 2019 02:51:50 AM
De two older

Thanks for the HTML as that allows me to see what the plugin produces. But, going through the sample from the My Clippings.txt, it isn't completely obvious which highlight and notes are connected. I would assume the ones with the same timestamp and location. But, that doesn't appear to be it. Most of the notes have a later timestamp than the highlight. It is only a few seconds, and probably represents how long it took to tap in the note. But that makes it harder to be sure the note and highlight are associated. And in the Kobo, I can make a highlight or bookmark and add a note to it later. Is this possible with the Kindle? If so, it makes it much harder to be sure of the association between the two.

And which highlight? The location is something like a paragraph. It would be easy to separately highlight the first and last words. And then knowing which word the note is attached to becomes harder.

I actually find your explanations and replies hard to follow,

If there is something in my posts that you don't understand, then ask for clarification. And if you use the quoting that is part of the forum, you can be specific about which parts.

so it wasn't clear that all you were doing was reading "my clippings.txt", though it seems very slow to be only doing that.

Well, I have only mentioned the My Clippings.txt and several times said it was the source of the data. I would have said if it came from somewhere else. Or asked for samples.

As to speed, I can't comment on that as I don't have a Kindle to test with. It is processing the complete file, so the more entries the longer. There is also some clumsiness in matching that back to the book on the device and calibre library. There isn't a direct link, so it actually has to look at the books. That is probably what is slow.

I thought you'd found access to something else asiIt's not even accurately copying what is there, compared with the Kindle DXG I had. I can't test it as I passed it on to someone with Macular Degeneration.
I hope to have an old 6" Kindle Keyboard soon to test mobi formating. Someone getting the current basic Kindle.

So anyway, I'm back to proofing people's texts on the Kobo as it's now at least usable again with combination of larger margin, line spacing and SD Card. Though poorer than when I bought it. Sadly though the Kindle is great for source selection and note typing compared to Kobo, the "my clippings.txt" is too limited lacking chapter & chapter %. Page numbers are a bit stupid on eBooks. The "location" might be good if it was something 100% reliable based on paragraph count. I've no idea what it is, except it's not affected by device screen, font size or page settings.

I have been told what the location is, but, as I don't use it, haven't bothered to remember. I'm sure it has been discussed here.

It's up to you what you change

As I said, until someone else confirms it, I'm not making a change.

but certainly it's needlessly repeating identical time stamps for note and source highlight,

Where is it repeating identical time stamps? There is one repeated timestamp in your My Clippings.txt sample ("15 January 2019 02:51:50"). There are none in either the HTML or the text version.

contrary to CSS preview and only putting location, not location range or page, at least I think so.

Maybe you need a friend with a Kindle PW3 or something using current FW to test.

I rely on the people who use the plugin to help with testing. They are the ones who find the bugs and know what is happening.

And my friends have better taste in devices

I thought of writing a program to process "my clippings.txt"

Be my guest.

Edit:
The "my clippings.txt" clip is newer than the Calibre import as it ends that book with

==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Highlight on page 282 | location 4320-4320 | Added on Tuesday, 15 January 2019 23:35:03

including yourselves if we take on the obligation of supplying the Reservations.”
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Note on page 282 | location 4320 | Added on Tuesday, 15 January 2019 23:35:10

Fix
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Highlight on page 283 | location 4332-4332 | Added on Tuesday, 15 January 2019 23:35:56

“You’ll have to see. Don’t
==========
Goths and Rooks (v6) (Ray McCarthy)
- Your Note on page 283 | location 4332 | Added on Tuesday, 15 January 2019 23:36:42

They are going to Emarineer sehortly

and the parts from Calibre are at the end.

However you can see the rather useless Page is missing (can be used I suppose on Kindle if no settings changed) and that locations for highlight can have a range.

Sorry, I'm baffled by this. Is this an extra section to demonstrate something? Just another sample? Or are you saying that when you ran the annotations plugin it didn't process all of the file? And what do you mean by "the parts from Calibre are at the end"? Nothing in that segment comes from calibre. So, the end of what? What parts from calibre? Was it just to show the range? If so, your other examples already showed it well enough. And the plugin handles ranges. But, it only takes the first number. Again, I'm not sure what use the full range would be in the fetched data.