How do I export notes (annotation) to my computer?

[kwokhin]

Originally Posted by davidfor

I don't suppose I have posted an overview of the methods for getting annotations from the Kobo devices. I would just point to the last time I posted this, but looking at it, it is missing some options and details.

Firstly, I an not happy with any of these methods. They all have limitations or need extra work or something. But, I also don't use any of them much. Most annotations I make are to mark errors in a book or something I want to look up later. Once I have fixed the problem or looked up the info, I delete the annotation. I know that some people want permanent annotations, but I haven't seen anything that comes close to a highlighter pen and a paper book.

Enough preamble, on to the methods...

Firstly, there is the method already discussed: run some SQL against the database and save the output. If you have any SQL skills, this is simple and can be easily customised. There have been also been several scripts posted here that do the same thing.

On the device, you can add "ExportHighlights=true" into the FeatureSettings section of the config file. With this, the long press menu of the book will have an extra option, "Export highlights" (or something like this). Using this will prompt you for a file name and export the annotations to that file in the root directory of the device.

If you are a calibre user, there is "Fetch annotations" (or something like that) on the device menu. This appends the annotations to the comments field of the metadata in the calibre library. It shows the highlighted text plus the annotation.

Again for calibre users, there is the Annotation plugin. Like the built-in calibre function, this fetches the annotations, but it can put them into a custom column. It is also handles them better if you use multiple devices or change them in the calibre library. This supports Kobo and Kindle devices plus some ereader apps. This plugin was created by someone else, but I am maintaining it.

As meera said, my Kobo Utilities plugin has a annotations function. It is basically the calibre "Fetch annotations" but written to a dialog that you can copy the text from. I wrote this before the Annotations plugin existed and do not recommend this. But, if I remove it, someone is sure to complain.

tshering wrote a nice little tool for Windows that extracts the annotations. I don't think it has been updated for a while, but the way annotations are stored hasn't changed, so it should still work.

Lastly, if the epub is on the main memory of the device, there is a file with the annotations in it. This is produced by the Adobe RMSDK side of the reader. It is in the directory "Digital Editions/Annotations". The rest of the path matches the path to the epub and the file has the extension "annot". This is an XML file that can easily be read to see the annotations. And with a little care, can be copied to the PC to use with ADE.

First, thank you so much for taking your time and reply with such a detailed and various methods. I don't have any SQL skills and I don't quite use Calibre. It doesn't recognize my device and it doesn't seem to work and I never bothered to solve it.

but I REALLY LOVE THE PROGRAM tshering WROTE! AMAZING! AMAZING! I can choose which book's annotation to export, unlike the method I mentioned earlier that the output file has everything. There is a space between each annotation, no annoying time. THIS IS EXACTLY WHAT I WANTED! If I have found this tool earlier, it could have saved me a few days being frustrated about extracting annotations. This tool should appear at the very top of Google search entry instead of that SQL base annoying method

Again, thank you so much for your help and I am really satisfied

[kwokhin]

Originally Posted by tshering

This is the correct link to the nice little tool. And here you can find a screenshot of it.

Thank you for taking your time and help me, tshering! This is EXACTLY WHAT I NEED AND WANT! It is so easy to use and just gets the job done!
Thank you!

[michailbachtin]

Hello, and thanks for your post.

Could you give me a screen shot of what the annotations look like once you export them? I assume that there is a separate file for each book.

Is there a character limit for highlighted text when exported? The sony PRS readers limit highlight-exports to only 200 characters, which is why I can't use the device. But the exported files looked great. You get a rough page number, the highlights, and next to them the annotations.

Here's what you get with the "PRST1AnnotationsExporter" available for free as a download:

[kndlewis]

With calibre built-in tool, the annotation is appended to the books comment section.
Below the double horizontal line.

[marp68]

I'm trying out the different ways of backup annotation, highligts etc.

Originally Posted by davidfor

On the device, you can add "ExportHighlights=true" into the FeatureSettings section of the config file. With this, the long press menu of the book will have an extra option, "Export highlights" (or something like this). Using this will prompt you for a file name and export the annotations to that file in the root directory of the device.

The Kobo ereader config file there is no FeatureSettings section. Only General, ApplicationPreferences, OneStoreServices, PowerOptions, Reading, ReadingLife and Services.

Where would I paste the "ExportHighlights=true"?

Thanks for great extra functions

[tshering]

Originally Posted by marp68

The Kobo ereader config file there is no FeatureSettings section. Only General, ApplicationPreferences, OneStoreServices, PowerOptions, Reading, ReadingLife and Services.

Where would I paste the "ExportHighlights=true"?

Thanks for great extra functions

You can add the section yourself.

[FeatureSettings]
ExportHighlights=true

[marp68]

Aha, great. Thanks

[marp68]

Originally Posted by davidfor

On the device, you can add "ExportHighlights=true" into the FeatureSettings section of the config file. With this, the long press menu of the book will have an extra option, "Export highlights" (or something like this). Using this will prompt you for a file name and export the annotations to that file in the root directory of the device.

If you are a calibre user, there is "Fetch annotations" (or something like that) on the device menu. This appends the annotations to the comments field of the metadata in the calibre library. It shows the highlighted text plus the annotation.

Again for calibre users, there is the Annotation plugin. Like the built-in calibre function, this fetches the annotations, but it can put them into a custom column. It is also handles them better if you use multiple devices or change them in the calibre library. This supports Kobo and Kindle devices plus some ereader apps. This plugin was created by someone else, but I am maintaining it.

As meera said, my Kobo Utilities plugin has a annotations function. It is basically the calibre "Fetch annotations" but written to a dialog that you can copy the text from. I wrote this before the Annotations plugin existed and do not recommend this. But, if I remove it, someone is sure to complain.

tshering wrote a nice little tool for Windows that extracts the annotations. I don't think it has been updated for a while, but the way annotations are stored hasn't changed, so it should still work.

Lastly, if the epub is on the main memory of the device, there is a file with the annotations in it. This is produced by the Adobe RMSDK side of the reader. It is in the directory "Digital Editions/Annotations". The rest of the path matches the path to the epub and the file has the extension "annot". This is an XML file that can easily be read to see the annotations. And with a little care, can be copied to the PC to use with ADE.

So, I've so far tried the Kobo Utility, Annotation to copy annotation/highlights to the computer. Also located the annotation files within the DE folder, which can be copied.

But since I copy the annotations for backup reasons in case of complete ereader failure or change of ereader, how do I restore the annotations back to either the factory reset eareader or a new ereader to which I copy all books from the calibre library?

And when making a calibre backup for the backup hardrive, does it include the books AND annotation files?

Sorry, about the questions, but when I was trying the kobo utilities to backup the device database it went fine yesterday. But when trying a lot with this annotation backup plugins, the database backup went wrong today, said it was corrupt. The reader worked okej though, so I did like a another member, just copied back the backup from yesterday. But I feel a bit scared to use plugins again. Just want to be able to make backups of my books and annotations from the device to the computer and from the computer to the external backup harddrive, for restoration of computer or reader.

[davidfor]

Originally Posted by marp68

So, I've so far tried the Kobo Utility, Annotation to copy annotation/highlights to the computer. Also located the annotation files within the DE folder, which can be copied.

But since I copy the annotations for backup reasons in case of complete ereader failure or change of ereader, how do I restore the annotations back to either the factory reset eareader or a new ereader to which I copy all books from the calibre library?

And when making a calibre backup for the backup hardrive, does it include the books AND annotation files?

There are no automatic backup methods of the Kobo devices that do everything. The backup in the utilities plugin is only the database and configuration files. If you know what books are on the device, this can be used to do a restore if the books are put back into the same place.

If the want to restore exactly to a prior state, you need to backup every file on the user partition of the device.

Sorry, about the questions, but when I was trying the kobo utilities to backup the device database it went fine yesterday. But when trying a lot with this annotation backup plugins, the database backup went wrong today, said it was corrupt. The reader worked okej though, so I did like a another member, just copied back the backup from yesterday. But I feel a bit scared to use plugins again. Just want to be able to make backups of my books and annotations from the device to the computer and from the computer to the external backup harddrive, for restoration of computer or reader.

The database backup us a simple copy of the file from the device followed by a check that it is OK and then compress it. If the database is corrupt when the check is done, it means either the database is corrupt on the device, or something else was using the database when the copy was taken. If the backup us done automatically, it is sequenced with the other connection jobs, so nothing else should be using it. If you do it manually, wait until the connection jobs have finished before doing it.

If the database is corrupt, the device can keep running. It reads the database into memory and works on that with writes at appropriate times. It is only when the database is read anew that the problem are noticed. The primary time for this is a restart. What you usually see is thing like the reading status reverting to an earlier state, or old books reappearing the home page.

If the utilities plugin reports a corrupt backup, you should check the database on the device. The utilities plugin has a function to do this. If it is OK, then have a look at what was happening during the backup. If it is corrupt, then you should restore a previous backup. You can also try compressing the database. Depending on the errors, especially if they are only in the indexes, this can fix the database.

[marp68]

Thanks for the explanation. I really appreciate you taking the time to explain. I want to build up knowledge and set up everything as good as possible from start, so I don't in the future have missed something when the device brakes down or need to be factory reset (which my experience with electronical things tells me the do sooner or later :-) ).

I do accept that there is no complete and easy way of doing it. Just want to be sure how to practically be able to restore a new/factory reset device.

Basically I only need one thing. How can I restore a new/factory reset device with markups/annotation made in another device. In this moment I never need to use any annotation/markup backup on the computer. And the books are always backuped on both computer and backup harddrive, so I would just copy the books back to a new/factory reset device.

So the question is which of the listed workarounds is best, if I later on need to copy the fetched/backuped markups/annotations back to a new/restored device again and being able to see them when opening the book aganin? And how do I practically do it?

For exampel, when using Annotation plugin and fetching markups/annotations into the Comment field or own created column, how do I later get them back to a new/restored device and get them to function with the book again when opened?

Haven't found any explanation about this so far, only the possibilities and processes of fetching/backup the annotations from the device to the computer.

I tried the "ExportHighlights=true" option, but in the exported file everything is mixed completely with no separation between the highlights. Did I missed something? And if I would have had annotations written to the markups, then it would be even more difficult. And is it possible to use the file when restoring a new/factory reset device?

[davidfor]

Sorry, but none of the annotation extraction methods support putting them back on the device. They are all fetching them to some other format.

If the purpose is to restore to a replacement device, the only real way to do is to completely backup the disk and restore that. This works, and I have done it a few times to recover after a factory reset.

You could just dump the Bookmarks table from the database. And then restore that to the new device. But, the books have to be restored to exactly the same path as before. Otherwise, things don't get connected properly. The full path is used as a key in the database on the device, so if this changes, the relationships in the database change. Of course, you could fix this, but, if there are a significant number of books, that will take a lot of time.

Someone could create an extract that kept the association with book by some other means. Then, restoring the bookmarks shouldn't be to hard. The obvious way to do this is as a calibre plugin and store the extract inside the library. That would maintain the link with the book for later restoring. All that wouldn't be hard, but personally, I've never had the need.

Another simpler backup of the annotations is the annotations files in the "Digital Editions" directory. That is a simple file copy each way. But, like the database, these use the file path of the books and they have to be restored to the same place.

Of course, there is one completely safe method; buy all your books from Kobo. Books bought from and downloaded to the device from the Kobo server will sync the bookmarks and reading positions to the server. These will then sync to any other device that uses the same Kobo userid.

Edit: For the "ExportHighlights=true" option, that's how it works. It is an unsupported feature of the firmware. It's handy for a quick-and-dirty way of saving some text and annotations. But, that's about all.

[marp68]

Thanks for the clarification. At least I now know there is no easy plug in or other program to use. I wish I could write programs and contribute my self

Ah, just backup the DE annotation files seems great. Putting it and the book back in the same place seems easy and seems like a nice backup solution for kobo readers anyway. Just backup the DE file after finishing the book. Would this mean that opening the book in the restored/new kobo reader, one would see the annotations/markups? As long as the book hasn't been chnaged in someway. And I doubt it would work with another reader that doesn't use DE files, right?

Also putting back the Bookmarks table wouldn't be hard. But exactly how do you dump the Bookmark tabel from the database? And does it not only contains bookmarks?

Backing up the whole disk is tempting, but wouldn't using the backup maybe be problematic replacing also the functioning software etc in the restored/new Kobo device. Btw, do you mean just making a normal image copy of the disk, firmware, patches, books, settings, etc?

Kind of strange that the manufactures of the ereaders haven't made it easy to backup the whole thing. I mean, if they advertise them with the possibility to store 2000 books with markups and annotations, they should understand that people want a secure and easy way to backup these things. Not only books bought from their own platform and not only to a cloud, but also to a computer or backup harddrive. The Tolino at least lets people synch all books, even those not bought at the the Tolino platform ebook sellers. That's a nice gesture.

Otherwise I'm so pleased with having started with ereaders. Very convenient. I borrow a lot of books from the public library. But no Spanish books though, since the public library says it's to expensive for them to buy Spanish ebooks. But I think I will be able to get a library card in Spain and hopefully being able to borrow books online, like I do here. Or maybe through my friend there.

[davidfor]

Originally Posted by marp68

Ah, just backup the DE annotation files seems great. Putting it and the book back in the same place seems easy and seems like a nice backup solution for kobo readers anyway. Just backup the DE file after finishing the book. Would this mean that opening the book in the restored/new kobo reader, one would see the annotations/markups?

Yes, that's the way it works. The Kobo device looks for the annotations file and loads it in place of whatever is in the database.

As long as the book hasn't been chnaged in someway.

That's a limitation for all the bookmark methods I know of. They contain references to the structure of the book and the code. If that changes, the bookmark becomes invalid.

And I doubt it would work with another reader that doesn't use DE files, right?

Maybe. Or maybe with a little work. I have transferred the annotations file from a Kobo to ADE on my PC, but I had to fiddle with it a little bit. From memory, there was an identifier in the file that was different. It's possible that if I had used ADE to send the book to the device, that it would have worked. I don't have another ereader that uses RMSDK to test with.

Also putting back the Bookmarks table wouldn't be hard. But exactly how do you dump the Bookmark tabel from the database? And does it not only contains bookmarks?

No, it isn't hard. The simplest thing is to backup the full database. Then when needed use an SQLite manager to dump the table from the backup to a file and then load that into the database on the device.

The Bookmarks table contains the details for all bookmarks, annotations and highlights on the device.

Backing up the whole disk is tempting, but wouldn't using the backup maybe be problematic replacing also the functioning software etc in the restored/new Kobo device. Btw, do you mean just making a normal image copy of the disk, firmware, patches, books, settings, etc?

I meant backing you the user visible partition. There are two partitions that do not get presented to the PC when you connect the device. You cant touch these partitions without removing the internal SD card, or doing a bit of work. You can wipe the user partition and the device will just recreate it the next time you restart it.

Kind of strange that the manufactures of the ereaders haven't made it easy to backup the whole thing. I mean, if they advertise them with the possibility to store 2000 books with markups and annotations, they should understand that people want a secure and easy way to backup these things. Not only books bought from their own platform and not only to a cloud, but also to a computer or backup harddrive. The Tolino at least lets people synch all books, even those not bought at the the Tolino platform ebook sellers. That's a nice gesture.

A secure way to backup this stuff means a lot of money spent developing it. That isn't really in their interest as the number of people who care is a lot lower than you think. Whether to syncing non-shop books is a decision they had to make. Again, there are development costs, but there are also legal considerations. Someone has said there is something in the Canadian privacy laws that made it a problem. Plus there is the simple issue of copyright. Can they do this legally? Amazon has decided they can, and apparently Tolino has. Maybe they have better lawyers the Kobo.

[marp68]

Yes, the laws may of course differ between countries. Here in Sweden the laws says you can make a copy or backup but only for private use. Don't know if it applies to ebooks though.

I think I will start out with the DE option. I also discovered the kobo utility option to delete empty DE annotation files. So the only ones that remains would be the one to backup.

Sorry, just to be clear about the practical stuff about the user partition backup and the database backup.

Will a normal complete folder copy of what's visible on the device select enough? And when copying back to device, do I just replace everything that is visible on the new/factory reset device? And then just run the device as normal?

Regarding database, I've set the kobo utilities to backup and compress the database automatically. Is that results a "full" database that I can use to dump the bookmark table to file with SQLite? And practically, how do you load the file back into the database on the device, with SQLite as well? Hmm... why not just use the the whole backuped database from the beginning then? I probably misunderstood something here. And does this means that the bookmark table contains the actual annotations etc?

Thanks for your patience. I like to learn and do things by myself. Slowly, step by step I get there... :-)

[davidfor]

Originally Posted by marp68

Yes, the laws may of course differ between countries. Here in Sweden the laws says you can make a copy or backup but only for private use. Don't know if it applies to ebooks though.

I think I will start out with the DE option. I also discovered the kobo utility option to delete empty DE annotation files. So the only ones that remains would be the one to backup.

Yes, that is needed as the firmware doesn't cleanup these files when deleting a book. It's caused a bit of confusion when people have deleted a book and then sent a new copy to the device.

Sorry, just to be clear about the practical stuff about the user partition backup and the database backup.

Will a normal complete folder copy of what's visible on the device select enough? And when copying back to device, do I just replace everything that is visible on the new/factory reset device? And then just run the device as normal?

Copy everything on the partition. If you are using a Mac or Linux, there are hidden directories. These must be copied. For the restore, I normally delete everything and then copy the backup back to the devvice.

Regarding database, I've set the kobo utilities to backup and compress the database automatically. Is that results a "full" database that I can use to dump the bookmark table to file with SQLite? And practically, how do you load the file back into the database on the device, with SQLite as well? Hmm... why not just use the the whole backuped database from the beginning then? I probably misunderstood something here. And does this means that the bookmark table contains the actual annotations etc?

The database backup is a copy of the file containing the database. So, yes it is a full backup. It's intended use is for when you realised the database on the device is corrupted. If there haven't been many changes, then simple replacing the database on the device and restarting it will get the you out of trouble. You will lose any changes made since the backup was made.

The longer the time since the backup and the more changes, the less worthwhile it is to restore the database. You will lose to many changes, any added books have to be processed and any removed books removed from the database. A factory reset is the ultimate in this. Unless the books are restored to exactly the same place, the contents of the database will effectively be cleared and rebuild.

To restore just one table is done by opening the backup database with an SQLite database manager and using its tools to export the table. Then opening the live database and importing the table.