Glo Kobo Annotation Extractor

[drjd]

I have been using the Tshering's Annotation Extractor tool for Kobo quite satisfactorily for some time. Recently I connected my device to extract some annotations from books stored on my storage card, and I could not see the contents of card while loading the database from device. I can see the contents on main memory on device, although. Restarting and reconnecting the kobo also does not work. I can however, still connect to Calibre with USB and can see the card's contents. I wonder what has happened. Any help will be highly appreciated. Thanks!

[tshering]

Originally Posted by drjd

Recently I connected my device to extract some annotations from books stored on my storage card, and I could not see the contents of card while loading the database from device. I can see the contents on main memory on device, although.

Do I understand correctly that

• you connect the device to your pc
• you can read the content of the internal and of the external sd cards with some file manager like the windows explorer
• you select (KT annotation extractor) "load database" > "from Kobo Reader"
• the file manager does not any longer see the content of the external sd, but still sees the content of the internal sd?

Or are you saying that from a certain point (namely since you tried to extract annotations from a book stored on the external sd), you cannot see the content of the external sd with any file manager or similar program, except if you have caliber running?

In both cases, I have no idea what could have happened. But I doubt that I understand correctly what you are saying.

[DNSB]

Originally Posted by drjd

I have been using the Tshering's Annotation Extractor tool for Kobo quite satisfactorily for some time. Recently I connected my device to extract some annotations from books stored on my storage card, and I could not see the contents of card while loading the database from device. I can see the contents on main memory on device, although. Restarting and reconnecting the kobo also does not work. I can however, still connect to Calibre with USB and can see the card's contents. I wonder what has happened. Any help will be highly appreciated. Thanks!

Hmmm... have you recently switched the external uSD card? That's about the only time short of a factory reset that I've lost the annotations for an ebook on my external uSD card. It doesn't seem to matter if you copy the contents between the cards as each card is given a different identifier in the database. That is, if I decide to move to the 32GB external uSD card from a 16GB, I remove the current card, copy all the files to the new card and then insert it into the Kobo, the card will be processed and all the ebooks on it treated as new.

[drjd]

Originally Posted by tshering

Do I understand correctly that

• you connect the device to your pc
• you can read the content of the internal and of the external sd cards with some file manager like the windows explorer
• you select (KT annotation extractor) "load database" > "from Kobo Reader"
• the file manager does not any longer see the content of the external sd, but still sees the content of the internal sd?

Or are you saying that from a certain point (namely since you tried to extract annotations from a book stored on the external sd), you cannot see the content of the external sd with any file manager or similar program, except if you have caliber running?

In both cases, I have no idea what could have happened. But I doubt that I understand correctly what you are saying.

Yes, the device connects properly to PC through USB.

1. I can see the contents of main memory and sd card in the windows explorer.

2. I can also connect to Calibre, which properly reads main/sd card, and can transfer files to any of them

3. When I use KT annotation extractor, I can load the database > from Kobo reader, but I can seen only few files stored on main memory, but it does not show files on the sd card, thus I can not extract annotations from the books stored on sd card.

My sd card has about 300 books on it, and most of them are annotated. Earlier I could see the list of annotated books from main memory as well as sd card.


This recent image above is showing only the books stored in main memory. No books from sd card.

PS: One more little thing, if it is relevant, I am getting this message while connecting the device to the PC, for a few seconds:


But after that, the windows explorer normally opens the sd card window.

Originally Posted by DNSB

Hmmm... have you recently switched the external uSD card? That's about the only time short of a factory reset that I've lost the annotations for an ebook on my external uSD card. It doesn't seem to matter if you copy the contents between the cards as each card is given a different identifier in the database. That is, if I decide to move to the 32GB external uSD card from a 16GB, I remove the current card, copy all the files to the new card and then insert it into the Kobo, the card will be processed and all the ebooks on it treated as new.

No, I have not changed the sd card. It is the same old 32 GB card. If I copy the contents of this card on my pc, then format the card and re-copy the contents on it and insert it in my device, will I lose all the annotations? In fact, I do not want to mess up with my annotations and highlights, which may be of importance to me.

[tshering]

Now I understand your problem a little better. Just for clarification, the annotation extractor reads only the database KoboReader.sqlite, it does not look for any other files on the external or internal sd. For instance, if you want to extract annotations of a book that is stored on the external sd, it does not matter whether the external sd is currently inserted into the device or not.

If you previously were able to extract annotations from books on the external sd, and you cannot do so any longer, it seems to indicate that something in the database has changed. Do you still see these annotations on the device? If this is the case, I guess there is no way of telling what is wrong without seeing the database.

I think it would be a good idea to make back up copies of your database regularly. I find it for instance very convenient, that I can still extract annotations that I made in 2012 of books that I have deleted from the device long ago.

[drjd]

Originally Posted by tshering

Do you still see these annotations on the device?

I think I've lost them!

While opening individual books on device already annotated and highlighted by me, I get a message "You don't have any annotations in this book."

I unfortunately didn't backup my database since long.

Can't help it now, I think. Even if I agree to forget the loss, what would you please suggest me now? Should I format the card and re-copy the books on it, so in future I can see them on Kobo annotation exporter, or should I factory reset the device, and reinstall the FW update? Kindly advise.

[JSWolf]

What is the URL for the Annotation Extractor? Thanks.

[drjd]

Originally Posted by JSWolf

What is the URL for the Annotation Extractor? Thanks.

Here it is :

https://www.mobileread.com/forums/att...3&d=1369686478

[davidfor]

Originally Posted by drjd

I think I've lost them!

While opening individual books on device already annotated and highlighted by me, I get a message "You don't have any annotations in this book."

I unfortunately didn't backup my database since long.

Can't help it now, I think. Even if I agree to forget the loss, what would you please suggest me now? Should I format the card and re-copy the books on it, so in future I can see them on Kobo annotation exporter, or should I factory reset the device, and reinstall the FW update? Kindly advise.

It doesn't matter. The annotations for all books are stored in the database. tshering's Annotation Extractor is reading this database to extract the annotations and doesn't make any reference to the actual books. As long as the annotations are there, the application should find them.

As you were expecting to find annotations for the books on the SD card, the question is where did they go. The most likely reason for the annotations being lost, is that the book was replaced. When the device is ejected from the PC or an SD card is inserted, the device look for changes. If the size of a book changes, it is removed from the database and processed as a new book. That means the reading status and annotations are all removed. If you edit the book and replace it, this will happen.

If you simply remove the external SD card, add a couple of books and insert it again, the books already on the card and in the database should be seen as the same. But, this has not always been the case. Older firmware handled this differently. And there was at least one that always reprocessed all the books. I haven't had an external SD card in one of in devices for a long time, but a quick test with firmware 3.19.5671 has it working as I expected. The annotations were kept when I ejected and reinserted the card.

Another reason to lose the annotations is if the external card is swapped. I can't check this at the moment, but the last time I tested, even if the exact same books were on the two cards in the same place, they would be treated as new books. The books on the first card would be removed from the database and the books on the new card would be added as new.

Some exceptions to all this:
- The driver in calibre will update the file size in the database when resending a book. This is to prevent the loss of reading status and annotations.
- For ePubs on the main memory, annotations are also stored in file. I find them more annoying than helpful.

[tshering]

@drjd

I do not think that you have to do anything. Under the assumption that you lost your annotations caused by one of the possibilities mentioned by Davidfor, all your books on the external sd are already registered in the database, the reading positions and so on however reset to start, and the annotations lost (that means you have exactly the situation that you would get by reformatting and so on).

You can simply start reading your books again and make new annotations. Do not forget to make backups.

@JSWolf
For those who prefer the link to the post providing the annotation extractor rather than the download link: here it is.

[drjd]

Originally Posted by davidfor

It doesn't matter. The annotations for all books are stored in the database. tshering's Annotation Extractor is reading this database to extract the annotations and doesn't make any reference to the actual books. As long as the annotations are there, the application should find them.

As you were expecting to find annotations for the books on the SD card, the question is where did they go. The most likely reason for the annotations being lost, is that the book was replaced. When the device is ejected from the PC or an SD card is inserted, the device look for changes. If the size of a book changes, it is removed from the database and processed as a new book. That means the reading status and annotations are all removed. If you edit the book and replace it, this will happen.

Thank you very much davidfor, for the detailed explanation. I had not made any changes in the books affecting their sizes. Yes, but I had ejected and reinserted the card just to insure that it is clean and dust free.

Originally Posted by davidfor

If you simply remove the external SD card, add a couple of books and insert it again, the books already on the card and in the database should be seen as the same. But, this has not always been the case. Older firmware handled this differently. And there was at least one that always reprocessed all the books. I haven't had an external SD card in one of in devices for a long time, but a quick test with firmware 3.19.5671 has it working as I expected. The annotations were kept when I ejected and reinserted the card.

I did not remove the card from device, but have added and/or removed some books through caliber or directly from windows explorer. I am sure I did not remove any book which contains annotations/highlights, because I had planned to extract them later on.

Originally Posted by davidfor

Another reason to lose the annotations is if the external card is swapped. I can't check this at the moment, but the last time I tested, even if the exact same books were on the two cards in the same place, they would be treated as new books. The books on the first card would be removed from the database and the books on the new card would be added as new.

Some exceptions to all this:
- The driver in calibre will update the file size in the database when resending a book. This is to prevent the loss of reading status and annotations.
- For ePubs on the main memory, annotations are also stored in file. I find them more annoying than helpful.

I have not swapped the cards in device. So this possibility can be ruled out. Also, I make it sure that books are in kepub format, because either these are purchased from Kobo, or sideloaded through caliber, duly converted from epubs.

PS: I also tried to extract annotations through your Kobo Utility Plugin installed in Calibre, but while selecting a book in sd card, the "copy annotation for selected book" is grayed out.

[drjd]

Originally Posted by tshering

@drjd

I do not think that you have to do anything. Under the assumption that you lost your annotations caused by one of the possibilities mentioned by Davidfor, all your books on the external sd are already registered in the database, the reading positions and so on however reset to start, and the annotations lost (that means you have exactly the situation that you would get by reformatting and so on).

You can simply start reading your books again and make new annotations. Do not forget to make backups.

You are right, tshering. I am not hopeful anymore to recover the lost annotations, but at least I have learned another lesson of taking backups periodically. (And I don't know how many times have I learned this lesson in life and forgot it. )