PRS-650 Restore annotations - how?

[elminster]

Problem
*Sigh*, the story... My 650 was hanging after start up and several resets didn't help. Well, being smart (well ) I backed up the "database\media" and the "Digital Editions\Annotations" folders. After doing a hard reset, I uploaded them again.

After the first start the reader obviously is doing some time consuming indexing. But for some reason the <annotation>-blocks in the ANNOT-files are deleted, only the <publication> block at the top is left unchanged.

Is there any other "master" file that keeps track of annotations? There must be a way to restore annotations, isn't there?

Solution
Assuming you have at least backed up the files in "Digital Editions\Annotations", it's pretty simple. For each annot-file:

Step 1: Open it in any text editor
Step 2: Remove any "sony:data" tag. Something like...

<sony:data xmlns:sony="http://www.sony.com/xmlns/product/prs/adeannotations">
<sony:item name="identifier" value="prsID.2.highlight:76-1-0-T0VCUFMvVGV4dC9TZWN0aW9uMDAwNF9zcGxpdF8wMDEueGh0bW wjcG9pbnQoLzEvNC8xMTQvMS8xOjE1ODEpAA==-T0VCUFMvVGV4dC9TZWN0aW9uMDAwNF9zcGxpdF8wMDEueGh0bW wjcG9pbnQoLzEvNC8xMTQvMS8xOjE3MDQpAA==" />
<sony:identifier>PLEASE IGNORE</sony:identifier>
</sony:data>

Step 3: Save and replace file on the reader

The reader will rebuild all your (edited) notes after the next restart. This may take some time.

If you've got a good text editor that can do regular expression replacement for multiple files at once, it's even easier and takes just a few seconds. The following perl regular expression worked for me in Ultraedit:

<sony:data(.+\n)*.+<\/sony:data>

[vxf]

Sorry, no good advice in regards - except to note that I have tried doing exactly what you mention - I was trying to figure out how to backup notes - and failed as well.

the only way I found to backup annotations was to copy the documents to my PC using the Reader Library software and then backing up the Reader Library directories. Those can be restored. But, for some reason, the annotation directories are structured differently on the device - and I was never able to restore those.

Hope someone with more technical knowledge can solve your problem.

[Bookworm_Girl]

Did you save the cacheEXT.xml file when you did the backup? You can open this file with a text editor and it tracks your annotations. Here is an old thread with some instructions:
https://www.mobileread.com/forums/sho....php?p=1368895

A few differences:
1) I don't modify the media.xml file as suggested.
2) I have found that you don't have to to paste everything between <text> and </text>. You will see for each book there is a tag called <markup> and </markup> where your annotations are stored. I copy & paste everything between those items from the back-up to my ereader.

[elminster]

Thanks for the link and information Bookworm_Girl! I didn't backup the cache folder, since I thought... well, it's a cache-folder; who'd put something important there?

Sony really should rethink this whole concept of linking information over multiple places without a decent way to backup everything safely.

Well, just 300+ notes gone. The most dumb thing is that the information obviously somehow IS available in the annot files themselves, but cannot easily be converted... :-/

[elminster]

Oh, just found out that the annotation are available, if I open the books in Adobe Digital Editions after copying the "Digital Editions\Annotations" folder to the device again. Well, in a way pretty obvious, but I never really used ADE much before. Anyway, they will be deleted at the next startup, but at least one could easily re-create the annotation step-by-step with the reader software.

Sure, there must be place for the annot-files on the harddisk, but anyway, there is no convenient way to get the notes back to the device itself. at least I think so...

[elminster]

Ok, found an elegant solution. I posted it in the first post, maybe it will help someone else.

[Bookworm_Girl]

Glad you found a solution! Thanks for posting the instructions.

[PRS-User]

Thanks very much Elminster, after weeks or even months of trying to find a solution, your method is the first one that has worked for me (I successfully restored the highlights for all 3 of the books/articles I tried), thereby in principle giving me access again to hundreds or thousands of highlights.

The only part that didnt work though is to do it through "find and replace" (hence, automatically all at once, rather than manually remove every instance) in one file, let alone in several files at once. You use

<sony:data(.+\n)*.+<\/sony:data>

but this didnt work for me. If anybody knows what line I can use to for this find-and-replace process in OpenOffice, or Word, or any other freely available text editor, I'd be very grateful. Having to manually remove thousands of those tags would be a huge hassle.

[PRS-User]

Also, the great thing about your solution is that now I can safely store annotations and books elsewhere (on computer, or SD-card for example, instead of on my reader) which greatly reduces the time it takes to create a highlight. When I only have a couple of dozen books/articles and their highlights on the main memory of my reader, it only takes one or two seconds for a highlight to save. When my reader was full with books/articles and their highlights it could take over 10 seconds.

[elminster]

Glad soembody finds it useful. I'm not a regular expression expert (it always takes me quite some time to get the right one, especially if something is to be matched over multiple lines), but since you asked for a freeware solution...

Programmer's Notepad is a freeware editor that supports regular expression. You can get it here: http://www.pnotepad.org/ (there is also a portable version, so you can use it from an usb stick for example).

The problem is that it doesn't support regular expressions fully yet. Anyway, this one works for me:

<sony:data(.+\n){1,10}.+</sony:data>

Usually the {1,10} part (meaning 1 to max 10 times) should be a *, but then the editor matches everything to the last and not the first </sony:data> closing tag. But I think (!) there are never more than 10 lines between the opening and closing tag, so this should work well.

The editor is also capable of replacing in multiple files.

Another freeware editor supporting regular expressions is PSPad (http://www.pspad.com/en/), but it doesn't support them for multiple files. Also I couldn't figure out the right syntax to match multiple lines. But that doesn't mean it is not possible

[PRS-User]

You are a God among men! This worked perfectly. As a first step, I restored the about 200 highlights in a big book, and it does it flawlessly. When I then disconnect the Reader from the computer it takes a couple of minutes for the Reader to restart. I'm assuming this is because it has to rebuild the highlights. Next up is trying to do it for multiple files at the same time.
Anyway, I'm very happy! Thanks again.

[PRS-User]

After spending a couple of minutes trying to figure out how to do it completely automatically and coming up short, I decided to do it semi-automatically. It took about 10 minutes to restore the notes of about 200 articles/books.

I opened all the .annot files at the same time in Programmer's Notepad(do 'open' and then go to the folder where you have all the .annot files and select them all. Earlier I had created a new folder with just the .annot files in it). Then it takes 10 - 20 seconds before Programmer's Notepad has opened all the about 200 files at the same time, in windows behind one another so that if you click one away you see the next one.

I then started with the first file, did 'find and replace' using the line you mentioned, then I closed the file, at which point the program asks whether I want to save the changes, I clicked okay, the file saves and closes closes and then the next file is automatically in front of me, and I do the same thing, until I've gone through all the files. Takes about 3 seconds per file.

[jbennion]

Just a note on elminster's technique for restoring notes, if you are ALSO using Calibre. When you reorganize book names, or author names, it changes the directory and filenames, so if you do that after you copy your books back, the filenames and directories would be different, and if you just copy the .annot file to the old location, it won't restore your notes. So here's how the file structure should look (my drive letter for my sony reader main memory is drive H).

If the book has this directory and filename:
H:\database\media\books\Baumeister, Roy F_\Willpower - Roy F. Baumeister_1690.epub

Then the .annot file should be in this directory, with this filename:
H:\Digital Editions\Annotations\database\media\books\Baumeist er, Roy F_\Willpower - Roy F. Baumeister_1690.epub.annot

[sun surfer]

elminster, thank you so much!

The Sony software somehow deleted lots of my books and notes, and I managed to find the .annot files still hidden on the reader, the same place you found them I think. But I couldn't get them to work again with the same re-downloaded books no matter what. I started a thread here but no help on that front and I scoured google with no help on .annot files either. Then as a last resort over a week later I thought of searching mobileread for .annot and see what comes up, and of only about a measly 20 results from the entire site, how lucky was I to find this one with the solution to the exact problem I'm having!

Though it took me awhile to get it right since what you were saying almost looked like a foreign language at first (I'm very non-tech), with patience I succeeded and I'm so appreciative that now all my hundreds/thousands of notes aren't gone with the wind. Thanks!