Help! I lost all my highlights after changing SD card

IseeDeadPeople · 10-29-2014, 02:05 AM

Hi,

I really need

I had a 32GB SD card in my Kobo Aura, with some generic books (ePubs) I found on the internet. I never bought any book on Kobo, and I never will. So all the books I have are on my external SD card. I read a couple of these books, and for each book I always like to highlight text, and add some annotations when needed.

Everything was working fine (I had access to my annotations/hightlights/etc) until I decided to swap my SD card. I'm prudent, so I decided to plug the old SD card into my computer to grab ALL content in there, and store it in my PC in a temporary folder, so that I can re-copy the exact same content onto the new SD card.

Then, I plugged the new SD card, and I put the data (previously stored in my PC) back in place. The same folders are on the new SD card.

YES, I have access to the exact same books as before. Only thing is, I lost all my highlights/annotations!!! Where are they? Another (major) problem, is that I already formatted the previous SD card, so I need to forget about this one.

My question is : Where are the highlights/annotations/bookmarks stored, when we read books that are stored (only) on the external SD card?

Note : I already looked in my temporary folder on my PC, for .kobo/KoboReader.sqlite file, and it seems like I have no annotations/highlights in there

davidfor · 10-29-2014, 02:50 AM

Yes, the annotations for books are stored in the database .kobo/KoboReader.sqlite. But, only the annotations for books the device knows about are there. That means all books on the internal memory and all books on the SD card that is currently inserted, or if no card is inserted, the last card that was inserted.

So, removing and reinserting the card, adding a book or two, should have no affect on the bookmarks. And removing the card, removing a book from the card and reinserting the card will mean that the device removes all information about the removed book when it reprocesses the card. If you remove the card, put a new copy of the book on the card and reinsert it, in most cases during processing the book will be removed from the database and the details added back in as new book. The only time this doesn't happen is if the size of the book didn't change.

For swapping cards, the device identifies them somehow. The database has the id for the card against all books on the card. This id is also in the config file. It is a six or eight character hex number and will have a value of "True". So, it look something like "123A56H8=True".

When you swap cards, the device removes all books that were on the old card from the database and processes the new card into the database. It doesn't care if the books are identical, just that the card was changed.

So, sorry, unless you have backup of the database, you have lost the annotations.

IseeDeadPeople · 10-29-2014, 11:38 PM

Hi davidform,

thanks for your precious informations...

I was also thinking that the Kobo may have some way of recognizing the SD card actually inserted, so that if we change card, the Kobo knows about it. And I think this is because when we format an SD card (or any other device), a unique "Volume ID" is given to it. Kobo may read this Volume ID and store it in the DB, as HEX numbers. Am I right to think like that?

So, I have a backup of my old "external" SD card on my computer, and also, a backup of the "internal" card. So, after opening the Kobo eReader.conf file, here are the only instances of HEX numbers in there :
[ApplicationPreferences]
B45F0B8C=true
BrightnessOpenCount=-1
C94709E6=true

I do have the KoboReader.sqlite file as well. I'm on Linux, so I open this file with "DBeaver", a Database Manager. I found out that in the table "content", I can see the last book title I was reading. When I scroll a bit on the right in the data, there's a field named "ExternalId", and the content is this : "C94709E6". This may be the HEX number we are looking for?

The problem is, the "Bookmark" table seems to be completely empty

Is this the table where I should find my annotations and highlights?

davidfor · 10-29-2014, 11:57 PM

Yes, all that is correct, or at least agrees with my investigations. The column ExternalId shows what card the books are on. And as far as I can tell, it only ever has the value for the last card that was inserted. The hex value in ExternalId is always in the config file as you see it. I don't know what the meaning of "true" is for that value and haven't tried changing it to "false". Maybe if I am bored over the weekend.

The Bookmark table holds the bookmarks and annotations for all books. For epubs and PDF in the main memory, there is also a file in the "Digital Editions" directory. But, it causes as many problems as it solves.

Your only way out at this point is if you had a backup of the database from before swapping the card.

IseeDeadPeople · 10-30-2014, 01:14 AM

Unfortunately, I think the backup of my internal SD card is more recent than the other one... And this may be why I don't see anything in the Bookmark table

It sucks, because when you realize that you lost all annotations/highlights/etc, it's just like losing a hard disk drive after a crash. I guess you've already lost at least 1 hdd in the past? It's really frustrating...

So, for now, I can just protect my ass for the future, so that this doesn't happen again. I read that I can sync my annotations and highlights with CALIBRE, at least many folks seem to like this platform.

Here's my plan :
1) Install firmware update on my Kobo Aura
2) Upgrade Android to latest version (if possible)
3) Install Calibre on my Synology Disk Station
4) Sync my Kobo Aura with Calibre, by Wi-Fi. (How?)

It seems like I can easily get info on this forum, for the 1st 3 steps. But I heard I can sync my Kobo with Calibre via Wi-Fi. How is this possible? Do I need to tweak my Kobo a bit? Where can I find more info about this?

Thanks for your help!

davidfor · 10-30-2014, 06:42 PM

Quote:

Originally Posted by IseeDeadPeople

Unfortunately, I think the backup of my internal SD card is more recent than the other one... And this may be why I don't see anything in the Bookmark table

It sucks, because when you realize that you lost all annotations/highlights/etc, it's just like losing a hard disk drive after a crash. I guess you've already lost at least 1 hdd in the past? It's really frustrating...

Yes, but my disk backups have always been good enough that I haven't lost any important data.

Quote:

So, for now, I can just protect my ass for the future, so that this doesn't happen again. I read that I can sync my annotations and highlights with CALIBRE, at least many folks seem to like this platform.

Here's my plan :
1) Install firmware update on my Kobo Aura
2) Upgrade Android to latest version (if possible)

The Kobo ereader devices do not run Android. It is Linux plus as custom application.

Quote:

3) Install Calibre on my Synology Disk Station
4) Sync my Kobo Aura with Calibre, by Wi-Fi. (How?)

It seems like I can easily get info on this forum, for the 1st 3 steps. But I heard I can sync my Kobo with Calibre via Wi-Fi. How is this possible? Do I need to tweak my Kobo a bit? Where can I find more info about this?

You heard wrong. The firmware in the Kobo ereaders will sync over WiFi to Kobo servers. But, they only sync the books bought from the Kobo shop. The sync does include the bookmarks and annotations made on these books.

Calibre doesn't backup annotations for any devices. For some, the Annotations plugin can be used to fetch them to a column in calibre. But, this is not reversible. It doesn't retrieve enough information to send them back.

You can backup the devices database using my Kobo Utilities plugin in calibre. It can automate this the first time you connect to calibre on any day.

IseeDeadPeople · 10-31-2014, 04:10 AM

Wow,

this clarifies a lot of things

If I understand well, it is possible to sync TO Kobo (via WiFi) highlights & annotations, but ONLY from books bought from the Kobo shop. Since I'm not buying books from the Kobo Shop, the SYNC function is almost useless for me. I guess...

So from what I'm reading now

, the easiest way to have a backup of my annotations, would be to use this setting in Kobo eReader.conf :

Code:

[FeatureSettings]
ExportHighlightsEnabled=true

With this, I will get a TXT file in the ROOT of my partition on my Kobo.

Is this function going to work with my Kobo Aura?

davidfor · 10-31-2014, 05:14 AM

Quote:

Originally Posted by IseeDeadPeople

Wow,

this clarifies a lot of things

If I understand well, it is possible to sync TO Kobo (via WiFi) highlights & annotations, but ONLY from books bought from the Kobo shop. Since I'm not buying books from the Kobo Shop, the SYNC function is almost useless for me. I guess...

Yes, that's right.

Quote:

So from what I'm reading now

, the easiest way to have a backup of my annotations, would be to use this setting in Kobo eReader.conf :

Code:

[FeatureSettings]
ExportHighlightsEnabled=true

With this, I will get a TXT file in the ROOT of my partition on my Kobo.

Is this function going to work with my Kobo Aura?

That will work with all Kobo devices with recent firmware. It's probably the easiest way, but only does one book at a time.

10-29-2014, 02:05 AM	#1
IseeDeadPeople Junior Member Posts: 4 Karma: 10 Join Date: Oct 2014 Device: Kobo Aura	Help! I lost all my highlights after changing SD card Hi, I really need I had a 32GB SD card in my Kobo Aura, with some generic books (ePubs) I found on the internet. I never bought any book on Kobo, and I never will. So all the books I have are on my external SD card. I read a couple of these books, and for each book I always like to highlight text, and add some annotations when needed. Everything was working fine (I had access to my annotations/hightlights/etc) until I decided to swap my SD card. I'm prudent, so I decided to plug the old SD card into my computer to grab ALL content in there, and store it in my PC in a temporary folder, so that I can re-copy the exact same content onto the new SD card. Then, I plugged the new SD card, and I put the data (previously stored in my PC) back in place. The same folders are on the new SD card. YES, I have access to the exact same books as before. Only thing is, I lost all my highlights/annotations!!! Where are they? Another (major) problem, is that I already formatted the previous SD card, so I need to forget about this one. My question is : Where are the highlights/annotations/bookmarks stored, when we read books that are stored (only) on the external SD card? Note : I already looked in my temporary folder on my PC, for .kobo/KoboReader.sqlite file, and it seems like I have no annotations/highlights in there

10-31-2014, 04:10 AM	#7
IseeDeadPeople Junior Member Posts: 4 Karma: 10 Join Date: Oct 2014 Device: Kobo Aura	Wow, this clarifies a lot of things If I understand well, it is possible to sync TO Kobo (via WiFi) highlights & annotations, but ONLY from books bought from the Kobo shop. Since I'm not buying books from the Kobo Shop, the SYNC function is almost useless for me. I guess... So from what I'm reading now , the easiest way to have a backup of my annotations, would be to use this setting in Kobo eReader.conf : Code: [FeatureSettings] ExportHighlightsEnabled=true With this, I will get a TXT file in the ROOT of my partition on my Kobo. Is this function going to work with my Kobo Aura?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Notes & Highlights for epubs stored on the sd card	einkuser	Barnes & Noble NOOK	0	07-19-2014 09:49 PM
PRS-650 bookmarks and highlights all lost?	catsknit	Sony Reader	4	09-18-2012 02:38 PM
The Lost Gate - Orson Scott Card	leebase	Reading Recommendations	0	03-01-2011 04:57 PM
Is it possible to flash or changing fonts of reader without card?	frodox	Sony Reader Dev Corner	5	04-21-2010 08:24 AM
Classic haw eney body tryd changing sim card?	Skaven	Barnes & Noble NOOK	0	12-20-2009 05:43 AM

10-29-2014, 02:50 AM	#2
davidfor Grand Sorcerer Posts: 24,907 Karma: 47303748 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	Yes, the annotations for books are stored in the database .kobo/KoboReader.sqlite. But, only the annotations for books the device knows about are there. That means all books on the internal memory and all books on the SD card that is currently inserted, or if no card is inserted, the last card that was inserted. So, removing and reinserting the card, adding a book or two, should have no affect on the bookmarks. And removing the card, removing a book from the card and reinserting the card will mean that the device removes all information about the removed book when it reprocesses the card. If you remove the card, put a new copy of the book on the card and reinsert it, in most cases during processing the book will be removed from the database and the details added back in as new book. The only time this doesn't happen is if the size of the book didn't change. For swapping cards, the device identifies them somehow. The database has the id for the card against all books on the card. This id is also in the config file. It is a six or eight character hex number and will have a value of "True". So, it look something like "123A56H8=True". When you swap cards, the device removes all books that were on the old card from the database and processes the new card into the database. It doesn't care if the books are identical, just that the card was changed. So, sorry, unless you have backup of the database, you have lost the annotations.

10-29-2014, 11:38 PM	#3
IseeDeadPeople Junior Member Posts: 4 Karma: 10 Join Date: Oct 2014 Device: Kobo Aura	Hi davidform, thanks for your precious informations... I was also thinking that the Kobo may have some way of recognizing the SD card actually inserted, so that if we change card, the Kobo knows about it. And I think this is because when we format an SD card (or any other device), a unique "Volume ID" is given to it. Kobo may read this Volume ID and store it in the DB, as HEX numbers. Am I right to think like that? So, I have a backup of my old "external" SD card on my computer, and also, a backup of the "internal" card. So, after opening the Kobo eReader.conf file, here are the only instances of HEX numbers in there : [ApplicationPreferences] B45F0B8C=true BrightnessOpenCount=-1 C94709E6=true I do have the KoboReader.sqlite file as well. I'm on Linux, so I open this file with "DBeaver", a Database Manager. I found out that in the table "content", I can see the last book title I was reading. When I scroll a bit on the right in the data, there's a field named "ExternalId", and the content is this : "C94709E6". This may be the HEX number we are looking for? The problem is, the "Bookmark" table seems to be completely empty Is this the table where I should find my annotations and highlights?

10-29-2014, 11:57 PM	#4
davidfor Grand Sorcerer Posts: 24,907 Karma: 47303748 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	Yes, all that is correct, or at least agrees with my investigations. The column ExternalId shows what card the books are on. And as far as I can tell, it only ever has the value for the last card that was inserted. The hex value in ExternalId is always in the config file as you see it. I don't know what the meaning of "true" is for that value and haven't tried changing it to "false". Maybe if I am bored over the weekend. The Bookmark table holds the bookmarks and annotations for all books. For epubs and PDF in the main memory, there is also a file in the "Digital Editions" directory. But, it causes as many problems as it solves. Your only way out at this point is if you had a backup of the database from before swapping the card.

10-30-2014, 01:14 AM	#5
IseeDeadPeople Junior Member Posts: 4 Karma: 10 Join Date: Oct 2014 Device: Kobo Aura	Unfortunately, I think the backup of my internal SD card is more recent than the other one... And this may be why I don't see anything in the Bookmark table It sucks, because when you realize that you lost all annotations/highlights/etc, it's just like losing a hard disk drive after a crash. I guess you've already lost at least 1 hdd in the past? It's really frustrating... So, for now, I can just protect my ass for the future, so that this doesn't happen again. I read that I can sync my annotations and highlights with CALIBRE, at least many folks seem to like this platform. Here's my plan : 1) Install firmware update on my Kobo Aura 2) Upgrade Android to latest version (if possible) 3) Install Calibre on my Synology Disk Station 4) Sync my Kobo Aura with Calibre, by Wi-Fi. (How?) It seems like I can easily get info on this forum, for the 1st 3 steps. But I heard I can sync my Kobo with Calibre via Wi-Fi. How is this possible? Do I need to tweak my Kobo a bit? Where can I find more info about this? Thanks for your help!