Forma how to export highlights like kindle

[davidfor]

Originally Posted by Deobulakenyo

Sorry I edited my post while you were responding.

I see that now, and the change sort of makes sense, but, it turns out that you and @Quoth were talking about completely different things.

I really don't understand hahahahaha

I have a book which i finished reading last night and there are lots of highlights. I want to export the highlights because i want to delete the book already. It is a sideloaded kepub.

There is no option on my libra to export the highlights when i long press the book.

Officially, there is no way. But, there is a config file entry that can be added to enable this. You need to add the following to the "Kobo eReader.conf" file on the device:

[FeatureSettings]
BookSpecificStats=true
ExportHighlights=true

The "FeatureSettings" might already be there. If it is, add the other lines after it. And if you do this on Windows, make sure you use a Linux line-end aware editor.

The "BookSpecificStats" is not needed for exporting the annotations. It turns on an option to see the Reading Stats for books other than the one you are currently reading. Those lines were together in my config file, so I copied them all.

With these options in the config file, when you long-press on a book in the lists, it will have "Export Annotations" and "Reading Stats" at the end.

I do not understand how Calibre can export/get the highlights from the book in the device when connected.

The Annotations plugin can be used for that. When you connect the device, the Annotations plugin button will have the option "Fetch annotations from connected device". Pressing that will start the fetch. The plugin gets annotations form the device for books in the calibre library. When finished, it displays the list of books with annotations. You can look at them and decide to keep them or not. If you import them, the annotations are added to the metadata for the books. Which means they are added to a column in calibre. If you do this again, the new annotations are merged with the existing annotations.

But, @Quoth doesn't use that. His annotations tend to be ephemeral. He marks up errors in the books, fixes them and then removes the annotations. For this, the uses "Copy annotations for Selected Book" which is part of the Kobo Utilities plugin. That reads the annotations from the selected books and shows them in a dialog. You can read them there or copy them to the clipboard.

Which of these is best depends on what you the annotations are for. If you want to keep them long term, the Annotations plugin is probably better. But, for short-lived annotations, the other probably is. If you are going to copy them elsewhere, the Kobo Utilities method might be better. I suggest trying them both and seeing what suits you better.

[Deobulakenyo]

Originally Posted by davidfor

I see that now, and the change sort of makes sense, but, it turns out that you and @Quoth were talking about completely different things.

Officially, there is no way. But, there is a config file entry that can be added to enable this. You need to add the following to the "Kobo eReader.conf" file on the device:

[FeatureSettings]
BookSpecificStats=true
ExportHighlights=true

The "FeatureSettings" might already be there. If it is, add the other lines after it. And if you do this on Windows, make sure you use a Linux line-end aware editor.

The "BookSpecificStats" is not needed for exporting the annotations. It turns on an option to see the Reading Stats for books other than the one you are currently reading. Those lines were together in my config file, so I copied them all.

With these options in the config file, when you long-press on a book in the lists, it will have "Export Annotations" and "Reading Stats" at the end.

Thanks for this. I was able to do this.

The Annotations plugin can be used for that. When you connect the device, the Annotations plugin button will have the option "Fetch annotations from connected device". Pressing that will start the fetch. The plugin gets annotations form the device for books in the calibre library. When finished, it displays the list of books with annotations. You can look at them and decide to keep them or not. If you import them, the annotations are added to the metadata for the books. Which means they are added to a column in calibre. If you do this again, the new annotations are merged with the existing annotations.

But, @Quoth doesn't use that. His annotations tend to be ephemeral. He marks up errors in the books, fixes them and then removes the annotations. For this, the uses "Copy annotations for Selected Book" which is part of the Kobo Utilities plugin. That reads the annotations from the selected books and shows them in a dialog. You can read them there or copy them to the clipboard.

Which of these is best depends on what you the annotations are for. If you want to keep them long term, the Annotations plugin is probably better. But, for short-lived annotations, the other probably is. If you are going to copy them elsewhere, the Kobo Utilities method might be better. I suggest trying them both and seeing what suits you better.

This one is a negative for me. Even though my device (libra) is connected to Calibre, and the device book is highlighted (one with highlights), all options are greyed out including Fetch annotations in the Annotations plugin. It is unfortunate as this is the method that i want more.

[davidfor]

Originally Posted by Deobulakenyo

This one is a negative for me. Even though my device (libra) is connected to Calibre, and the device book is highlighted (one with highlights), all options are greyed out including Fetch annotations in the Annotations plugin. It is unfortunate as this is the method that i want more.

That's strange, and wrong. If a Kobo device is connected, and identified as a Kobo device, the Annotations plugin should enable the fetch. And it would also do this for the other supported devices. And I did just check that it was working.

Have you configured the plugin? I can't remember if this is needed before it can be used. The configuration should default to using the comments field for annotations, but, it might not be enabled until you choose this yourself.

Can you run calibre in debug mode and post the log after connecting the device? To do that, right click the Preferences button in calibre and select the "Restart in debug" mode option. Calibre will restarted and display a message about the log. Then connect the device and let the device jobs run. When they finish, close calibre. The debug log should be displayed. Post the full log for me to look at. If you don't want to post it, upload it somewhere and PM me the link. Hopefully I will be able to see what the issue is.


And for the record, I maintain the Annotations plugin, created the Kobo Utilities plugin and maintain the KoboTouch driver in calibre. I am involved in one way or another with most things in calibre related to Kobo. And I do some other work, mainly on plugins, in calibre.

[Deobulakenyo]

Originally Posted by davidfor

That's strange, and wrong. If a Kobo device is connected, and identified as a Kobo device, the Annotations plugin should enable the fetch. And it would also do this for the other supported devices. And I did just check that it was working.

Have you configured the plugin? I can't remember if this is needed before it can be used. The configuration should default to using the comments field for annotations, but, it might not be enabled until you choose this yourself.

Can you run calibre in debug mode and post the log after connecting the device? To do that, right click the Preferences button in calibre and select the "Restart in debug" mode option. Calibre will restarted and display a message about the log. Then connect the device and let the device jobs run. When they finish, close calibre. The debug log should be displayed. Post the full log for me to look at. If you don't want to post it, upload it somewhere and PM me the link. Hopefully I will be able to see what the issue is.


And for the record, I maintain the Annotations plugin, created the Kobo Utilities plugin and maintain the KoboTouch driver in calibre. I am involved in one way or another with most things in calibre related to Kobo. And I do some other work, mainly on plugins, in calibre.

I rebooted my PC and tried again. Now Fetch annotations from device is now working. however, these are the problems:

1. annotations from 4 books were retrieved, the one book i intend to get the annotation from isn't there (there should be 5)

2. After retrieving the annotations, where should/would they be found?

3. are highlights different from annotations per se? The one book i am speaking about has highlights of words that I want to review the definitions of after reading.

And thank you very much for the patience in helping a non-techy person like me

[davidfor]

Originally Posted by Deobulakenyo

I rebooted my PC and tried again. Now Fetch annotations from device is now working. however, these are the problems:

1. annotations from 4 books were retrieved, the one book i intend to get the annotation from isn't there (there should be 5)

The first thing is to check that calibre has matched that book to the library. The plugin works annotations for books that calibre has that are also on the device. If there isn't a green tick next to the book in the library list, then the plugin cannot retrieve the annotations. That can be fixed.

If the tick is there, then the plugin should get the annotations. I need to see a debug log. Can you turn on the debugging option in the plugins configuration and then run calibre in debug mode and try the fetch again. Post the log and tell me which book is the problem book so I can read through the log.

2. After retrieving the annotations, where should/would they be found?

That will depend on the configuration. By default they are appended to the comments column. But, you can add a custom column and put them in that. In either case, you will see them in the calibre interface. But, they are not added to the actual book.

3. are highlights different from annotations per se? The one book i am speaking about has highlights of words that I want to review the definitions of after reading.

Only in how the device shows them and the HTML the plugin generates. The database on the device stores them in the same place with indications of what type it is. That used this when fetching them. If you just highlight text, that is all that is in the annotation that is fetched. If there is a note attached to it, that is shown under the highlight.

[Numnumnum]

Originally Posted by davidfor

I haven't looked to see if the highlights are there after a library book was returned. But, I had been told by Kobo that one of the reasons for adding the book as a preview after it is returned is to have somewhere for the annotations to be kept in case the person bought the book later. I assumed this would work with borrowing the book again.

FWIW I can confirm that the highlights remain on an OverDrive book when re-borrowed .

However, I’m now seeing that highlights and annotations which I have made separately for *purchased books* on Kobo and on the KOBO iOS app on my iphone are not syncing properly and, I assume, therefore not downloadable in their entirety through Calibre. Does anyone know how to force a sync of those annotations for exporting ?

[davidfor]

Originally Posted by Numnumnum

FWIW I can confirm that the highlights remain on an OverDrive book when re-borrowed .

Good to have someone confirm that.

However, I’m now seeing that highlights and annotations which I have made separately for *purchased books* on Kobo and on the KOBO iOS app on my iphone are not syncing properly and, I assume, therefore not downloadable in their entirety through Calibre. Does anyone know how to force a sync of those annotations for exporting ?

Why? The methods in calibre to get annotations from Kobo devices work with the device. If the annotations are on the device, they should be fetched. If they are not fetched, then there is a bug that needs to be looked at. And see above for how to start debugging them.

For the iOS and Android apps, there have been problems on and off with annotations syncing to and from them. I haven't tested this for a while, so I don't know the status. One thing that does trip me up is that the syncing isn't necessarily immediate. From memory, there can be a 12 hour delay. Kobo told me that was deliberate, but, I'm not sure why they did it that way. They might have reduced the timing, but, I don't think it will be immediate.

[pgilster]

Originally Posted by davidfor

Did you delete the preview that Kobo added? If you had removed that from your library, that would probably explain it.

That would explain my problem. I never looked at the preview file to see if it held my annotations. I have to hang onto the next preview that comes in after a library return to see what I can find out. Thanks!

[Numnumnum]

Originally Posted by davidfor

For the iOS and Android apps, there have been problems on and off with annotations syncing to and from them. I haven't tested this for a while, so I don't know the status. One thing that does trip me up is that the syncing isn't necessarily immediate. From memory, there can be a 12 hour delay. Kobo told me that was deliberate, but, I'm not sure why they did it that way. They might have reduced the timing, but, I don't think it will be immediate.

OK thanks. I appreciate you responding with your expertise. I’ve been testing out a bit and it seems like annotations from the kobo iOS app are syncing with the kobo device but not vice versa. I’m confused why that would be the case. But at any rate I suppose what really matters is whether the annotations are getting on the kobo. Although it does make me a bit nervous that things aren’t syncing properly. For the most part I’ll be reading on the kobo it’s just those moments on the subway on the bus were I want to pull up my phone and do some catch-up reading. But if things consistently I think with the kobo even after 12 hours or 24 hours or however long it takes I think that that would work fine with calibre export .

[Deobulakenyo]

Originally Posted by Numnumnum

FWIW I can confirm that the highlights remain on an OverDrive book when re-borrowed .

However, I’m now seeing that highlights and annotations which I have made separately for *purchased books* on Kobo and on the KOBO iOS app on my iphone are not syncing properly and, I assume, therefore not downloadable in their entirety through Calibre. Does anyone know how to force a sync of those annotations for exporting ?

I think it is fine now. will test it out more and then seek your expertise if I run into new/old problems.

Thanks for the patience and the help.

[davidfor]

Originally Posted by pgilster

That would explain my problem. I never looked at the preview file to see if it held my annotations. I have to hang onto the next preview that comes in after a library return to see what I can find out.

I am not sure if you can see the annotations when you look at the preview. I was told by Kobo that one of the reasons for adding the preview was to have a place to store the annotations in their database. That doesn't necessarily mean they will be usable by you.

And I think I should have been clearer about what I meant by "delete", or as Kobo uses, "remove". There are three levels:

• Remove download - this deletes the file from the device but leaves the details. You can tap it to download again.
• Remove from My Books - removes the file and details and puts the book in the "Archive" section of your library. You have to visit the store and library to get it back.
• Delete from Archive - Books in the archive can be delete and will be removed from your account completely. Something is still kept in Kobos database as, if I remember correctly, if you try to buy the book again, they know and restore the book.

I don't know when the annotations are actually removed. I would expect they are still there if you restored a book from the Archive. But, I would expect them to be removed if you deleted the book from the Archive.

[pgilster]

Thanks, davidfor. I haven't had the chance to explore this yet but will try to look into it over the weekend. I take a lot of annotations and really want to be able to retrieve them from library books. I appreciate your help.

[Numnumnum]

Originally Posted by Quoth

Kobo works better.
1) Use Calibre and Kobo Utilites (Best soiution and why I'd not go back to a Kindle). Select book, Kobo Utility menu Import Annotations, copy to clipboard and paste into any program or a custom Calibre column. Windows, Mac or Linux.
OR
2) Enable menu item on long press of book to export highlights, annotations & bookmarks by adding a line to the kobo config file. This creates a text file, better than the Kindle one, but not as good as option 1.

Sorry, I really feel daft here, but I'm trying to use method 1) here on an OverDrive book, but Calibre says that "Virtual books cannot be added to the Library" - Does this mean that method 1) doesn't work with OverDrive books and will have to use method 2) ?

[davidfor]

Originally Posted by Numnumnum

Sorry, I really feel daft here, but I'm trying to use method 1) here on an OverDrive book, but Calibre says that "Virtual books cannot be added to the Library" - Does this mean that method 1) doesn't work with OverDrive books and will have to use method 2) ?

Method 1 will only work if the book on the device matches a book in the calibre library. For books from Kobo, including Overdrive books, you cannot use the "Add to library" method from the device list (the "Virtual books" is a misnomer due to a generic message being used). You can add an empty book by right-clicking the add button in calibre and select the option to add a book. You then enter the title and author and calibre will create a book entry in the library. The next time you connect the device, it should match the added book to the book on the device and you can use the copy annotations function. Or many of the other functions in the plugin.

[Skydog]

Originally Posted by davidfor

2. After retrieving the annotations, where should/would they be found?

That will depend on the configuration. By default they are appended to the comments column. But, you can add a custom column and put them in that. In either case, you will see them in the calibre interface. But, they are not added to the actual book.

What is the difference between "Fetching annotations (experimental)" and the "Annotations" button?

When "fetching" annotations, I see a one-second dialog box appear "amending annotations......" or something like that but where do they go? Nothing shows up when selecting the "Browse annotations" button (nor in the column). The "Annotations" feature works as expected, providing a list as well as column data.