[GUI Plugin] Annotations

[davidfor]

Originally Posted by Gergely

This is an old post from 2011. Is there a github page for this Annotations plugin for Calibre? Can I use this to import annotations from my Kobo Aura H20 ebook reader?

This thread is the active thread for discussing the Annotations plugin for calibre (and was started in 2014 after the previous developer of the plugin left).

There is a github page at https://github.com/davidfor/calibre-annotations, but support should be through this thread. You can get the source from there, or it is all in the plugin. You can download that from the first page of the thread, or by unpacking the plugin after installing it through calibre.

Yes, the plugin can fetch annotations from all Kobo devices. And states that in the first post. And there has been lots of discussion on this throughout the thread.

[Terisa de morgan]

Originally Posted by Gergely

This is an old post from 2011. Is there a github page for this Annotations plugin for Calibre? Can I use this to import annotations from my Kobo Aura H20 ebook reader?

ANSWER: https://github.com/davidfor/calibre-annotations

Yes, you can use for importing annotations from Kobo ereaders-

[Gergely]

Sorry, already asked.

[sweth]

Recently started getting the following error when trying to fetch annotations from my Kobo. Any thoughts on what might be going on?

calibre, version 3.26.1
Error fetching annotations: <p>Unable to fetch annotations from Kobo Aura H2O.</p>

Traceback (most recent call last):
File "calibre_plugins.annotations.action", line 301, in fetch_device_annotations
File "calibre_plugins.annotations.action", line 1024, in process_selected_books
File "calibre_plugins.annotations.action", line 158, in add_annotations_to_calibre
File "calibre_plugins.annotations.annotations", line 251, in merge_annotations
File "calibre_plugins.annotations.annotations_db", line 241, in capture_content
TypeError: 'NoneType' object has no attribute '__getitem__'

[davidfor]

Originally Posted by sweth

Recently started getting the following error when trying to fetch annotations from my Kobo. Any thoughts on what might be going on?

The error is when the plugin is parsing the existing annotations before merging them with the new annotations. It looks like the format of the stored annotation is wrong. If you edit the annotation, you need to be careful not to change the structure to allow later merges to work.

I'll add some code to catch the error. Strangely, this particular line is the only problem. Most of the parsing of the existing annotations catches when there is a missing attribute, just not this one.

[davidfor]

Attached is a beta that adds support for the tolino devices. The annotations are read from the "notes.txt" file in the root partition of the device. This is similar to the clippings file in Kindle devices.

This is all new work but seems to be OK with my tolino epos. It should be the same for the other tolino devices as they seem to use the same basic firmware.

Fetching the annotations works by parsing the notes.txt file. This includes words to indicate the types of annotations. I have put support in for German and Spanish. But, I haven't done much testing with either of these languages. For other languages, it is a largely matter of getting a sample notes.txt file with each type of annotation in it. From that, I can get the strings needed.

If anyone with a tolino can test this and tell me how it goes, it would be much appreciated.

Also, this beta has a fix for the problem reported by @sweth. The error will be handled, and the new annotations should be added.

[DNSB]

Tested on my Tolino/calibre 3.27.1 and imported the annotations without any issues that I noticed.

[sweth]

Originally Posted by davidfor

Also, this beta has a fix for the problem reported by @sweth. The error will be handled, and the new annotations should be added.

I don't remember intentionally changing any annotations, but the beta does resolve the error. Thanks for the prompt fix!

[BobC]

Imports my bookmarks from a Tolino Shine o.k.

Calibre 3.27.1 64Bit

BobC

[BeckyEbook]

Originally Posted by davidfor

If anyone with a tolino can test this and tell me how it goes, it would be much appreciated.

Calibre 3.28
Windows 7 Pro 64-bit
Tolino Vision 2
Annotations 1.8.1 beta (with tolino support)

Look at the attachment.
1. A normal title WORKS
2. If there is only a title (without the author in brackets) DOES NOT WORK
3. If there is a range of pages DOES NOT WORK


Edit:
Temporary solution (ignoring case 2 and case 3):

Spoiler:

ParseTolinoNotesTxt.py from line 251

            try:
                log('DEBUG', "match: ---%s---" % (match,))
            except:
                pass
            try:
                log('DEBUG', "match.groups(): ---%s---" % (match.groups(),))
            except:
                pass

[davidfor]

Originally Posted by BeckyEbook

Calibre 3.28
Windows 7 Pro 64-bit
Tolino Vision 2
Annotations 1.8.1 beta (with tolino support)

Look at the attachment.
1. A normal title WORKS
2. If there is only a title (without the author in brackets) DOES NOT WORK
3. If there is a range of pages DOES NOT WORK

Hmm, two situations that I didn't have, and didn't even know could happen. I have done the work and the attached beta should fix this. Plus, I realised I wasn't populating the notes correctly and there were a few other problems. And I've added the other languages.

So, this beta has:

• Fix: tolino: Handle annotation without an author.
• Fix: tolino: Handle highlight and note properly.
• Update: tolino: Add French, Italian and Dutch support.
• Fix: Populate the note and highlighted text for the annotations.
• Fix: Better handling of bad dates.
• Fix: Strip the opening and closing quote.

This should be pretty much complete. As usual, please tell me if you see any problems.

[BeckyEbook]

The plugin (1.8.3 beta) with tolino reader works great now. Thank you!

[Sam Sahara]

I use the annotations plugin to fetch annotations from a kobo A1.

There are two types of annotations:

• simple highlights: the text is underlined;
  
• highlights with annotations: the text is underlined and the user adds some personal chunk of text.

Well, the bug is: every time I fetch the annotations from the kobo to calibre, the simple highlights that are already fetched are not fetched again, while the highlights with annotations are newly added to notes.

As a clarification I add the code taken from the opf file of a my epub with some notes and highlights: as you can see every highlight with notes are repeated two times, whlie the simple highlights are not.

<meta
  name="calibre:user_metadata:#mm_annotations"
  content="{'label': 'mm_annotations','link_column': 'value','display': {},'is_editable': true,'is_custom': true,'table': 'custom_column_19','is_csp': false,'category_sort': 'value','name': 'Annotations','datatype': 'comments','#extra#': null,'rec_index': 27,'colnum': 19,
  '#value#':
    '<div class=\'user_annotations\' style=\'margin:0\'>
    <div class=\'annotation\' genre=\'\' hash=\'0b59471c538494c262593330dae33efc\' location_sort=\'011016\' reader=\'KoboTouch\' style=\'margin:0 0 0.5em 0\'>
    <table cellpadding=\'0\' width=\'100%\' style=\'background-color:LightGray;color:black;font-size:80%;font-weight:bold;margin:0;\' color=\'Gray\'>
      <tr>
        <td class=\'location\' style=\'text-align:left\'>II</td>
        <td class=\'timestamp\' uts=\'1528105650.0\' style=\'text-align:right\'>lun 04 giu 2018 11:47:30 </td>
      </tr>
    </table>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'>Vi assicuro con la massima serietà che molte volte ho desiderato di diventare un insetto.</p>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'></p>
    <p class=\'note\' style=\'background-color: rgb(255, 255, 77);font-size:90%;font-style:italic;margin:0;\'>Forse Gregor S.?</p>
    <p class=\'note\' style=\'background-color: rgb(255, 255, 77);font-size:90%;font-style:italic;margin:0;\'></p>
    </div>

    <!-- This div group repeats the preceding one -->
    <div class=\'annotation\' genre=\'\' hash=\'27f6e510b7334401b8bd86d7d377e932\' location_sort=\'011016\' reader=\'KoboTouch\' style=\'margin:0 0 0.5em 0\'>
    <table cellpadding=\'0\' width=\'100%\' style=\'background-color:LightGray;color:black;font-size:80%;font-weight:bold;margin:0;\' color=\'Gray\'>
      <tr>
        <td class=\'location\' style=\'text-align:left\'>II</td>
        <td class=\'timestamp\' uts=\'1528105650.0\' style=\'text-align:right\'>lun 04 giu 2018 11:47:30 </td>
      </tr>
    </table>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'>Vi assicuro con la massima serietà che molte volte ho desiderato di diventare un insetto.</p>
    <p class=\'note\' style=\'background-color: rgb(255, 255, 77);font-size:90%;font-style:italic;margin:0;\'>Forse Gregor S.?</p>
    </div>

    <div class=\'annotation\' genre=\'\' hash=\'39ec145f4967efbd85099567b490d72e\' location_sort=\'012085\' reader=\'KoboTouch\' style=\'margin:0 0 0.5em 0\'>
    <table cellpadding=\'0\' width=\'100%\' style=\'background-color:LightGray;color:black;font-size:80%;font-weight:bold;margin:0;\' color=\'Gray\'>
      <tr>
        <td class=\'location\' style=\'text-align:left\'>III</td>
        <td class=\'timestamp\' uts=\'1528182681.0\' style=\'text-align:right\'>mar 05 giu 2018 09:11:21 </td>
      </tr>
    </table>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'>Signore Iddio, ma a me che importa delle leggi della natura e dell’aritmetica se poi, chissà perché, queste leggi e questo due più due quattro non mi piacciono? S’intende che non cercherò di abbattere quel muro a testate se non avrò davvero la forza di abbatterlo, ma non mi concilierò con esso soltanto perché mi trovo davanti un muro di pietra e le mie forze non sono sufficienti.</p>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'></p>
    </div>

    <div class=\'annotation\' genre=\'\' hash=\'d41d8cd98f00b204e9800998ecf8427e\' location_sort=\'012100\' reader=\'KoboTouch\' style=\'margin:0 0 0.5em 0\'>
    <table cellpadding=\'0\' width=\'100%\' style=\'background-color:LightGray;color:black;font-size:80%;font-weight:bold;margin:0;\' color=\'Gray\'>
      <tr>
        <td class=\'location\' style=\'text-align:left\'>III</td>
        <td class=\'timestamp\' uts=\'1528182829.0\' style=\'text-align:right\'>mar 05 giu 2018 09:13:49 </td>
      </tr>
    </table>
    </div>

    <div class=\'annotation\' genre=\'\' hash=\'07d445f8e87b324e7e19f0f0d044ccee\' location_sort=\'017040\' reader=\'KoboTouch\' style=\'margin:0 0 0.5em 0\'>
    <table cellpadding=\'0\' width=\'100%\' style=\'background-color:LightGray;color:black;font-size:80%;font-weight:bold;margin:0;\' color=\'Gray\'>
      <tr>
        <td class=\'location\' style=\'text-align:left\'>VIII</td>
        <td class=\'timestamp\' uts=\'1528390811.0\' style=\'text-align:right\'>gio 07 giu 2018 19:00:11 </td>
      </tr>
    </table>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'>alle spalle ben quarant’anni di sottosuolo</p>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'></p>
    <p class=\'note\' style=\'background-color: rgb(255, 255, 77);font-size:90%;font-style:italic;margin:0;\'>Forse le «Statue d'acqua» della Jaeggy provengono da questo sottosuolo.</p>
    <p class=\'note\' style=\'background-color: rgb(255, 255, 77);font-size:90%;font-style:italic;margin:0;\'></p>
    </div>

    <!-- This div group repeats the preceding one -->
    <div class=\'annotation\' genre=\'\' hash=\'15ca4833fc0dfd428f700c3317e07eab\' location_sort=\'017040\' reader=\'KoboTouch\' style=\'margin:0 0 0.5em 0\'>
    <table cellpadding=\'0\' width=\'100%\' style=\'background-color:LightGray;color:black;font-size:80%;font-weight:bold;margin:0;\' color=\'Gray\'>
      <tr>
        <td class=\'location\' style=\'text-align:left\'>VIII</td>
        <td class=\'timestamp\' uts=\'1528390811.0\' style=\'text-align:right\'>gio 07 giu 2018 19:00:11 </td>
      </tr>
    </table>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'>alle spalle ben quarant’anni di sottosuolo</p>
    <p class=\'note\' style=\'background-color: rgb(255, 255, 77);font-size:90%;font-style:italic;margin:0;\'>Forse le «Statue d'acqua» della Jaeggy provengono da questo sottosuolo.</p>
    </div>

    <div class=\'annotation\' genre=\'\' hash=\'71b6772f37d63934682e5fedd6d599a0\' location_sort=\'019100\' reader=\'KoboTouch\' style=\'margin:0 0 0.5em 0\'>
    <table cellpadding=\'0\' width=\'100%\' style=\'background-color:LightGray;color:black;font-size:80%;font-weight:bold;margin:0;\' color=\'Gray\'>
      <tr>
        <td class=\'location\' style=\'text-align:left\'>X</td>
        <td class=\'timestamp\' uts=\'1528563367.0\' style=\'text-align:right\'>sab 09 giu 2018 18:56:07 </td>
      </tr>
    </table>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'>E del resto, sapete che vi dico? Io sono convinto che noialtri, uomini del sottosuolo, dobbiamo essere tenuti al guinzaglio. Siamo capaci di starcene magari per quarant’anni rinchiusi in silenzio nel sottosuolo, ma se una volta riusciamo a liberarci e a tornare alla luce, allora cominciamo a parlare, parlare, parlare…</p>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'></p>
    </div>

    <div class=\'annotation\' genre=\'\' hash=\'ebad3644be6b2e64423ace68c8c564df\' location_sort=\'020057\' reader=\'KoboTouch\' style=\'margin:0 0 0.5em 0\'>
    <table cellpadding=\'0\' width=\'100%\' style=\'background-color:LightGray;color:black;font-size:80%;font-weight:bold;margin:0;\' color=\'Gray\'>
      <tr>
        <td class=\'location\' style=\'text-align:left\'>XI</td>
        <td class=\'timestamp\' uts=\'1528713308.0\' style=\'text-align:right\'>lun 11 giu 2018 12:35:08 </td>
      </tr>
    </table>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'>voglio fare appunto questo esperimento: è possibile essere completamente franchi con se stessi e non aver paura di tutta la verità?</p>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'></p>
    </div>

    <div class=\'annotation\' genre=\'\' hash=\'34d32afcb950e3814671537952acb828\' location_sort=\'022009\' reader=\'KoboTouch\' style=\'margin:0 0 0.5em 0\'>
    <table cellpadding=\'0\' width=\'100%\' style=\'background-color:LightGray;color:black;font-size:80%;font-weight:bold;margin:0;\' color=\'Gray\'>
      <tr>
        <td class=\'location\' style=\'text-align:left\'>I</td>
        <td class=\'timestamp\' uts=\'1528714786.0\' style=\'text-align:right\'>lun 11 giu 2018 12:59:46 </td>
      </tr>
    </table>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'>Io, per esempio, odiavo la mia faccia, la trovavo ripugnante e sospettavo perfino che vi si leggesse una espressione di viltà, e pertanto ogni volta che mi recavo in ufficio mi sforzavo tormentosamente di comportarmi nella maniera più indipendente, affinché non mi si sospettasse di viltà, e di avere il viso atteggiato all’espressione più nobile. </p>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'></p>
    </div>

    <div class=\'annotation\' genre=\'\' hash=\'c26794f3d534bf9dfbc5f73af54a8f8a\' location_sort=\'022040\' reader=\'KoboTouch\' style=\'margin:0 0 0.5em 0\'>
    <table cellpadding=\'0\' width=\'100%\' style=\'background-color:LightGray;color:black;font-size:80%;font-weight:bold;margin:0;\' color=\'Gray\'>
      <tr>
        <td class=\'location\' style=\'text-align:left\'>I</td>
        <td class=\'timestamp\' uts=\'1528723160.0\' style=\'text-align:right\'>lun 11 giu 2018 15:19:20 </td>
      </tr>
    </table>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'>A casa mi dedicavo soprattutto alla lettura. Volevo soffocare con sensazioni esteriori tutto ciò che ribolliva continuamente dentro di me. E le sole sensazioni esteriori che potessi procurarmi provenivano dalla lettura. Naturalmente la lettura mi era di notevole aiuto: mi commuoveva, mi deliziava, mi torturava. Ma certe volte anche mi annoiava terribilmente. Allora mi prendeva la voglia di muovermi, e di tanto in tanto mi sprofondavo in una dissolutezza meschina, tetra, schifosa, sotterranea.</p>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'></p>
    </div>

    <div class=\'annotation\' genre=\'\' hash=\'fe90e63880f891a42ffe13f84f2830ff\' location_sort=\'023025\' reader=\'KoboTouch\' style=\'margin:0 0 0.5em 0\'>
    <table cellpadding=\'0\' width=\'100%\' style=\'background-color:LightGray;color:black;font-size:80%;font-weight:bold;margin:0;\' color=\'Gray\'>
      <tr>
        <td class=\'location\' style=\'text-align:left\'>II</td>
        <td class=\'timestamp\' uts=\'1528753026.0\' style=\'text-align:right\'>lun 11 giu 2018 23:37:06 </td>
      </tr>
    </table>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'> È proprio così: a quel tempo io credevo ciecamente che per qualche miracolo, grazie a qualche circostanza esteriore, qualcosa si sarebbe messo in movimento, l’orizzonte si sarebbe ampliato; d’un tratto mi si sarebbe aperto un campo di attività corrispondente alle mie aspirazioni, attività benefica, stupenda e soprattutto già pronta (di quale attività esattamente si trattasse non avrei mai saputo dirlo, ma l’essenziale era il fatto che fosse già pronta), ed ecco che avrei improvvisamente fatto la mia comparsa nel mondo, addirittura su un cavallo bianco e con una corona d’alloro in capo.</p>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'></p>
    </div>

    <div class=\'annotation\' genre=\'\' hash=\'1697e6f8645d9f7f57826a39786bbd0f\' location_sort=\'024017\' reader=\'KoboTouch\' style=\'margin:0 0 0.5em 0\'>
    <table cellpadding=\'0\' width=\'100%\' style=\'background-color:LightGray;color:black;font-size:80%;font-weight:bold;margin:0;\' color=\'Gray\'>
      <tr>
        <td class=\'location\' style=\'text-align:left\'>III</td>
        <td class=\'timestamp\' uts=\'1528830842.0\' style=\'text-align:right\'>mar 12 giu 2018 21:14:02 </td>
      </tr>
    </table>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'>odiavo il suo volto bello ma sciocco (con il quale, tuttavia, avrei volentieri cambiato il mio viso intelligente) </p>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'></p>
    </div>

    <div class=\'annotation\' genre=\'\' hash=\'f76e587f075a7fb031228c01d985cb42\' location_sort=\'027052\' reader=\'KoboTouch\' style=\'margin:0 0 0.5em 0\'>
    <table cellpadding=\'0\' width=\'100%\' style=\'background-color:LightGray;color:black;font-size:80%;font-weight:bold;margin:0;\' color=\'Gray\'>
      <tr>
        <td class=\'location\' style=\'text-align:left\'>VI</td>
        <td class=\'timestamp\' uts=\'1529526147.0\' style=\'text-align:right\'>mer 20 giu 2018 22:22:27 </td>
      </tr>
    </table>
    <p class=\'highlight\' style=\'margin:0;text-indent:0.5em;\'>, sì, una schiava! Tu dai tutto, sacrifichi tutta la tua libertà. E più tardi, anche se vorrai spezzare questa catena, non sarà più possibile: ti c’invischierai sempre più strettamente. È una catena maledetta. Io la conosco bene. Lasciamo stare il resto, perché tanto tu non lo capiresti, credo; ma dimmi solo questo: ti sei già indebitata con la padrona, vero? Ecco, lo vedi!” ripresi a dire, sebbene lei non mi avesse risposto; però mi ascoltava in silenzio, mi ascoltava con tutto il suo essere; “ecco la catena che ti stringe!</p>
    </div>

    </div>',
    'kind': 'field', 'column': 'value', 'is_category': false, 'search_terms': ['#mm_annotations'], 'is_multiple2': {}, 'is_multiple': null}"/>

I've added some html comments to mark the repeated chunks.

[davidfor]

Originally Posted by Sam Sahara

I use the annotations plugin to fetch annotations from a kobo A1.

There are two types of annotations:

• simple highlights: the text is underlined;
  
• highlights with annotations: the text is underlined and the user adds some personal chunk of text.

Well, the bug is: every time I fetch the annotations from the kobo to calibre, the simple highlights that are already fetched are not fetched again, while the highlights with annotations are newly added to notes.

As a clarification I add the code taken from the opf file of a my epub with some notes and highlights: as you can see every highlight with notes are repeated two times, whlie the simple highlights are not.


I've added some html comments to mark the repeated chunks.

I did test this when I was adding the tolino support to check how things worked. It seemed to work, but, I'm not sure if I had both types of annotations. I'll recheck tonight.

Also, what happens if you do the fetch a third time? Is a third copy added?

[Sam Sahara]

Originally Posted by davidfor

Also, what happens if you do the fetch a third time? Is a third copy added?

It seems not.