Annotations - any ideas of sensible approach?

[Mekk]

I am fairly heavily annotating books while reading (using my Sony PRS-600). Especially I mark a lot of quotes. And I dream about some solution which would let me to:

a) backup those annotations (preferably to calibre library, so they could be kept together with books)

b) restore them back to my reader (in case I delete the book from the reader, then put it back there)

and maybe

c) review those annotations on PC somehow (one of the ideas was to convert into format of jarnal or gournal, at least in case of pdf files).

d) convert those annotations to another reader format in case I buy one in the future.


I plan to hack some code (from the Sony side it does not seem too difficult, they use XML files with readable structure), but I am not yet sure which approach to undertake. In particular:

1) Are there any chances to define some "portable" annotations format (or database structure, or...) for Calibre? Sony has fairly rich set of possibilities (marked quotes, freeline drawings, text notes - all bound to specific locations in the book), I don't know about other readers but I feel the general model should be similar.

2) Does it seem sensible to define some "common" place in Calibre library to store those. Extra files in book directories? Custom database fields? Sth else?

3) In case there is a hope for 1) and 2), what is the best approach to merge such functionality with calibre?

[Mekk]

In case it could be of some use, here is what Sony saves:

1) for each annotated book there exists XML file in annotations directory, for example for the book

./database/media/books/Christian Crumlish/Designing Social Interfaces - Christian Crumlish & Erin Malone.epub

I have file

./Digital Editions/Annotations/database/media/books/Christian Crumlish/Designing Social Interfaces - Christian Crumlish & Erin Malone.epub.annot

containing all annotations.



2) The latter looks like:

<annotationSet xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns="http://ns.adobe.com/digitaleditions/annotations">
  <publication>
    <dc:title>Designing Social Interfaces</dc:title>
    <dc:creator>Christian Crumlish</dc:creator>
    <dc:publisher>O'Reilly Media</dc:publisher>
    <dc:identifier>urn:isbn:9780596806125</dc:identifier>
  </publication>
  <annotation>
    <dc:identifier>urn:uuid:6973626E3A39-3738-3035-393638303631</dc:identifier>
    <dc:date>2010-03-13T09:24:06Z</dc:date>
    <target>
      <fragment start="OEBPS/ch02.html#point(/1/2/1/10/7/2/1:185)" end="OEBPS/ch02.html#point(/1/2/1/10/7/3/1:13)"/>
    </target>
    <content>
      <dc:date>2010-03-13T09:24:06Z</dc:date>
      <text/>
    </content>
    <sony:data xmlns:sony="http://www.sony.com/xmlns/product/prs/adeannotations">
      <sony:item name="identifier" value="prsID.2.highlight:37-3-0-T0VCUFMvY2gwMi5odG1sI3BvaW50KC8xLzIvMS8xMC83LzIvMToxODUpAA==-T0VCUFMvY2gwMi5odG1sI3BvaW50KC8xLzIvMS8xMC83LzMvMToxMykA" />
      <sony:identifier>PLEASE IGNORE</sony:identifier>
    </sony:data>
  </annotation>

(and now much more similar <annotation> blocks, the one above points to marked quote)


3) freehand drawings are noted by

  <annotation>
    <dc:identifier>urn:uuid:742F3E000030-3954-3035-3A32303A3436</dc:identifier>
    <dc:date>2010-03-10T02:50:29Z</dc:date>
    <target>
      <fragment start="index_split_000.html#point(/1/4/90/1:272)" end="index_split_000.html#point(/1/4/90/1:272)"/>
    </target>
    <content>
      <dc:date>2010-03-10T02:51:29Z</dc:date>
      <text/>
    </content>
    <sony:data xmlns:sony="http://www.sony.com/xmlns/product/prs/adeannotations">
      <sony:item name="identifier" value="prsID.2.freehandMarkup:6-9-0-4" />
      <sony:identifier>PLEASE IGNORE</sony:identifier>
      <sony:item name="duplicate" value="false" />
    </sony:data>
  </annotation>

and accompanied by yet another directory (named ./database/markup/database/media/books/Christian Crumlish/Designing Social Interfaces - Christian Crumlish & Erin Malone.epub/ in this case) which contains .jpg and .svg for every drawing. Those files has numeric names (for example
1268189694120.249.jpg and 1268189694120.249.svg) and I am still to guess how to match those with the place in the book.

[kovidgoyal]

This is a several stage process:

1) Come up with a calibre annotations "standard"

2) Write code to map the SONY annotations to the standard defined in (1)

3) Write code to map "standard" annotations to SONY annotations

4) Port the calibre e-book viewer to use the standard

5) Figure out the best way to store annotations in the calibre library

5.5) If the best way involves storing annotations outside ebook files, modify the metadata writers for EPUB to insert the annotations into the file when saving/sending to disk

6) Modify the SONY driver to use 3) and send annotations to the SONY so that they can be read by the SONY

I am planning to work on 1) and 4) as part of the embedded web viewer in the calibre content server.

Presumably, once 1) and 4) are done, the rest can be tackled. If you (or anyone else) wants to contribute to 1) or 4) feel free, if there are enough contributors, I'll setup a wiki page defining the annotations standard.

To get you started on 1):

The standard will have two parts

a) Defining a "location" in the ebook

Schemes that are used today in various ebook readers

- jquery selectors
- XPath selectors
- Adobe's usual opaque nonsense
- rendered position percentage
- text strings
- Binary offsets (Amazon the great)

A location IMO will need at least three levels

Level One: XPath selector
Level Two: text string
Level Three: rendered percentage

The idea being that if level one is unavailable use level two if that is unavailable use level three and so on.

The hardest part will be writing code to migrate legacy selectors to the standard scheme. The only way to reliably migrate for example, the Adobe location selector you described previously is to open the epub and parse the html to find what the selector points to and then convert that to XPath. And of course, I doubt Amazon's scheme will ever be migrated.

An then there is the question of identifying the ebook itself in the location (probably use the EPUB unique identifier).

You can of course come up with a much simpler solution if all you want to do is support the SONY devices, but I am not interested in that.

[Mekk]

I feel somewhat afraid of shooting at 1) as I know very little about ebook specs and ebook readers different than mine. Still, friendly google took me to http://www.niso.org/workrooms/daisy/...2005.html#Bkmk - which, while intended for audio, solves similar problem and is very similar to the Sony approach - and I started to think that maybe instead of inventing new format we could just document and use Adobe's one (maybe introducing some restrictions). Here http://code.google.com/p/epub-revisi...pleAnnotations seems that some people could think similarly.

What do you think about such an idea? Or, if not Adobe's, maybe someone's else format could be used?

Are you (or anybody here) able to share annotation format used on other devices? What is Kindle saving?

[kovidgoyal]

I'm definitely not going to use adobe's format as a basis for annotations, it is too limited. It is based on hierarchical tag counts with no fallback, which means that if you edit the file even slightly, the annotation position could be lost. Plus the annotation are completely non-portable across formats. That can be remedied with a simple text string fallback, as I described earlier.

If you find 1) intimidating, you will just have to wait until I get around to defining the standard.

[Mekk]

Selectors

Maybe important thing to note: .epub is not the only format which can be annotated. Sony (the reader) nicely handles PDF annotation providing the same interaction (likely other formats too, but I haven't tried).

Here it looks so:

  <annotation>
    <dc:identifier>urn:uuid:3C64633A6964-656E-7469-666965722F3E</dc:identifier>
    <dc:date>2010-03-03T03:36:21Z</dc:date>
    <target>
      <fragment start="#pdfloc(9c75,44,78,0,9,0,0,1)" end="#pdfloc(9c75,44,89,0,10,0,1,1)"/>
    </target>
    <content>
      <dc:date>2010-03-03T03:36:21Z</dc:date>
      <text/>
    </content>
    <sony:data xmlns:sony="http://www.sony.com/xmlns/product/prs/adeannotations">
      <sony:item name="identifier" value="prsID.2.highlight:44-1-0-I3BkZmxvYyg5Yzc1LDQ0LDc4LDAsOSwwLDAsMSkA-I3BkZmxvYyg5Yzc1LDQ0LDg5LDAsMTAsMCwxLDEpAA==" />
      <sony:identifier>PLEASE IGNORE</sony:identifier>
      <sony:item name="duplicate" value="false" />
    </sony:data>
  </annotation>

In case of PDF we won't be able to use selectors, so the page/position-based method also may be needed (at least if the solution is to be able to accomodate different file formats).

Inside or in separate file

Considering readers keep annotations in separate file, different formats can be used and you plan to survive editing, I am not sure whether the idea of saving annotations inside .epub is good, I'd rather keep them separately. This may be also better from the copyright/editing rights/... point of view (leaving apart discussion whether I can edit bought epub, separate annotation file can be safely and easily send to friend without troubling anyone rights).

[Mekk]

OK, as Adobe's format is excluded, here is the very very initial idea sketch.

Annotation file(s)

The book annotations are kept in a dedicated file (say - annotations.xml in book directory).

Possible extension: keeping also device-specific file(s), at least until the format matures and is proven to handle conversion back properly.

Annotation types

The file is to accomodate (at least) the following annotation types:

- highlight/quote (effect of marking some text in the book - from, to, maybe excerpt)
- drawing (freehand image hooked to the specific position in the book - location, size, SVG file)
- text note (textual note hooked to the specific position in the book - location, note text)
- bookmark (pointer to specific page, possibly entitled or marked up with iconic symbol)

I am not yet sure whether those should be treated separately, or just
as single "annotation" item which possibly can have many attributes
(at the very least it seems sensible to handle things like bookmark
with name, highlight with additional text title, image with label).

Location problem

All anotations must specify their position (or two positions). Due to
already named problems with detection/conversion of those, it seems
best to accomodate multiple simultaneous variations (so the same
location can be specified by both logical and physical coordinates, or
one of them).

Calibre functions

Basic data exchange:

- allow user to "download annotations" (grab all annotations from the device, or just annotations to the selected files on the device, and save them to the library)

- automatically "upload annotations", if present, together with the book whenever it is uploaded, and/or upload annotations separately on demand

PC review/editing:

- viewing annotations in .epub viewer
- entering/modifying annotations in .epub viewer
- (?) supporting other formats (maybe things like export to notetaking apps)

Sync (more difficult and surely not for the first step):

- allow user to "sync annotations" (3-way merge of changes made on the device and on PC, with conflicts resolved in favor of keeping/duplicating notes)

[kovidgoyal]

well the annotations will definitely be stored separately, but they may in addition, be stored in the epub. I find sending two files to give someone annotations is unnecessary complicated. Annotations will be handled like the rest of the metadata in the calibre library (i.e. they will be updated in the file when saving to disk and sending to device).

[kovidgoyal]

Apart from sync, which will be difficult to implement, sounds about right, except that annotations will probably be stored in the sqlite database or in a separate file in each books directory in the calibre database.

And I'd prefer to use JSON rather than XML for the annotations.

[Mekk]

Originally Posted by kovidgoyal

well the annotations will definitely be stored separately, but they may in addition, be stored in the epub.

If thing is saved in two places, one goes out of date ;-)

I find sending two files to give someone annotations is unnecessary complicated.

But note also the case mentioned above: if the book annotated is commercial, I may not feel allowed to send the book itself. At the same time sending or even publishing on the web "my annotations to such and such book" is OK (and people who also bought this book would be able to review what I marked as important - for example).

Whatever - it need not be resolved straight away.

Annotations will be handled like the rest of the metadata in the calibre library (i.e. they will be updated in the file when saving to disk and sending to device).

I am not very happy with having to "save to disk" my book to preserve annotations. My book usually already is on the disk, annotations are just extras.

In fact, I'd love most the scenario in which whenever I connect my reader, I'd be able to just "sync annotations" without having to worry about which books I annotated since prev sync etc.

except that annotations will probably be stored in the sqlite database or in a separate file in each books directory in the calibre database.

Considering loose structure and possibly large size (I happened to make hundreds of notes in some book) I suggest a file in book directory.

Also, some idea of where to save graphical annotations (likely as .svg) is needed. Subdirectory below book dir? Paste them inside actual annotations file? Sth else?

And I'd prefer to use JSON rather than XML for the annotations.

I feel sick whenever I see XML namespaces so I like this idea, I offered XML as everything in .epub is <so>taggy</so>.

Another reason for XML is that it may be easier to embed data inside (like aforementioned graphics files). But, well...

I will try to draft some sketch of how could such .json possibly look like (but not today ;-)). Once it is defined I can also volunteer to work on converters to/from sony (=adobe) format.

[kovidgoyal]

Go for it, once you're done defining the spec, I'll add my input, we can finalize it, and then go from there.

[loyukfai]

Looks like EPUB3 may have annotation specified in the specs...?

http://code.google.com/p/epub-revisi...salAnnotations

And there seems to be a scholar-led effort on defining annotation...?

http://www.openannotation.org/

[loyukfai]

So, EPUB3 arrived without a standard for the annotations after all... There are some people working on it though, maybe in EPUB3.x or EPUB4...?

http://www.hkdlc.org/index.php?optio...ment&Itemid=39
http://code.google.com/p/epub-revision/wiki/Annotations
http://code.google.com/p/epub-revisi...salAnnotations

[Man Eating Duck]

Originally Posted by loyukfai

So, EPUB3 arrived without a standard for the annotations after all... There are some people working on it though, maybe in EPUB3.x or EPUB4...?

http://www.hkdlc.org/index.php?optio...ment&Itemid=39
http://code.google.com/p/epub-revision/wiki/Annotations
http://code.google.com/p/epub-revisi...salAnnotations

Ouch, that's too bad. Annotation support really needs to be a mandatory part of the core specs, otherwise I'm pretty sure that XKCD already nailed the upcoming situation.

We'll probably end up there anyway, manufacturers don't exactly adhere to the existing specs as it is, but having annotation support as part of the specs would probably be an improvement

[elminster]

I recently had a problem with my annotations being "lost". Solved it here https://www.mobileread.com/forums/sho...89#post1826989

The interesting thing is, that the Sony device(s) basically only needs those adobe annotations, yet builds its own index (in database\cache\cacheExt.xml); maybe really just for performance reasons like cache suggests.

So in respect to any annotations except drawings (I think)...
for a) You just have to backup the annot files
for b) Just remove the parts the sony reader adds and copy the annot file back, named like the ebook file (which is only slightly tricky, because of calibre adding the numbering automatically)
for c) You could review/edit the annotations only on PC with Adobe Digital Editions (well, sure here it gets a little bit more complicated in regard to a) and b) with paths and consistency)
for d) The interesting question is if (many) other readers use different ways to annotate. Just in case it should not be a Sony specific solution only...

Hope I didn't miss the topic too much (?)