Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 08-09-2018, 03:58 PM   #541
Katja_hbg
Connoisseur
Katja_hbg 's ceiling is 100% spider-free.Katja_hbg 's ceiling is 100% spider-free.Katja_hbg 's ceiling is 100% spider-free.Katja_hbg 's ceiling is 100% spider-free.Katja_hbg 's ceiling is 100% spider-free.Katja_hbg 's ceiling is 100% spider-free.Katja_hbg 's ceiling is 100% spider-free.Katja_hbg 's ceiling is 100% spider-free.Katja_hbg 's ceiling is 100% spider-free.Katja_hbg 's ceiling is 100% spider-free.Katja_hbg 's ceiling is 100% spider-free.
 
Posts: 63
Karma: 118898
Join Date: Oct 2015
Device: Kobo Glo HD (landscape), Kobo Aura One
hi @davidfor,
in your help with my issue fetching annotation using kobo-utility, you mentioned this plugin. So I gave it a try. I did not finally decide as there are pros and cons on both.
The KU has still too many empty rows while this one has layout instead plain text.
With KU I have to do it one by one while this fetches in one go.
Lets see where I end with.

This text is to ask you to add something in the very first text. I searched whether and how to suppress the timestamp and found the answer in blog #48 page 4.
There is a section in blog #1 about modifying the timestamp. Perhaps you like to mention this hidden function ... To suppress write display=none in the timeout css field
Katja_hbg is offline   Reply With Quote
Old 08-10-2018, 08:40 AM   #542
loFreaQ
Junior Member
loFreaQ began at the beginning.
 
Posts: 1
Karma: 10
Join Date: Aug 2013
Device: Kindle Paperwhite
Hello!

I'm trying to use your plugin with my Paperwhite 1, but I have two problems that neither the latest beta resolves.

When I select the option to retrieve annotations from my kindle, the list of books installed on my Kindle appears correctly, but when I click the button to fetch the annotations, i get this error:
Code:
calibre, version 3.29.0
Errore recuperando le annotazioni: <p>Non sono in grado di recuperare annotazioni da Amazon Kindle.</p>

Traceback (most recent call last):
  File "calibre_plugins.annotations.action", line 301, in fetch_device_annotations
  File "calibre_plugins.annotations.action", line 1027, in process_selected_books
  File "calibre_plugins.annotations.action", line 135, in add_annotations_to_calibre
  File "calibre_plugins.annotations.annotations_db", line 180, in annotations_to_html
  File "calibre_plugins.annotations.annotations", line 170, in to_HTML
UnboundLocalError: local variable 'text_style' referenced before assignment
Also, if I try to modify the aspect by clicking the proper button in the plugin settings window, I get this error:
Code:
calibre, version 3.29.0
ERRORE: Eccezione non gestita: <b>KeyError</b>:u'Testo'

calibre 3.29 Portable embedded-python: True is64bit: False
Windows-10-10.0.17134-SP0 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.17134')
Python 2.7.12+
Windows: ('10', '10.0.17134', 'SP0', u'Multiprocessor Free')
Interface language: it
Successfully initialized third party plugins: DeDRM (6, 1, 0) && Kindle Collections (1, 7, 20) && DOC Input (1, 0, 1) && Quality Check (1, 9, 11) && Annotations (1, 8, 2) && Generate Cover (1, 5, 21) && Save To Format (0, 0, 6)
Traceback (most recent call last):
  File "calibre_plugins.annotations.config", line 325, in configure_appearance
  File "calibre_plugins.annotations.appearance", line 496, in __init__
  File "calibre_plugins.annotations.appearance", line 293, in initialize
  File "calibre_plugins.annotations.appearance", line 346, in populate_table
  File "calibre_plugins.annotations.appearance", line 353, in populate_table_row
KeyError: u'Testo'
Anything I can do to help you debug this?
Windows 10, Calibre 3.29 portable 64 bit.

Thanks!
loFreaQ is offline   Reply With Quote
Old 08-10-2018, 09:06 AM   #543
bopuc
Member
bopuc began at the beginning.
 
Posts: 12
Karma: 10
Join Date: Jun 2014
Device: KOA
Quote:
Originally Posted by davidfor View Post
@GoNico: I don't consider the issue with annotation sharing or retrieving a issue with copyrights. It is more an issue of standards. To the best of my knowledge, there is no real standard for how annotations are marked in ebooks or stored for sharing. Each ereading device and application basically does its own thing. Some have ways to export to things like EverNote, for others, it is completely contained.

Unfortunately, it doesn't look like something the big companies are interested in fixing, so I can't see anything is likely to change in the near future.
Not that this is the place to debate such things but just to point out (as someone who has experience with standards bodies, most recently the W3C's new Digital Publishing Working Group)… you are half way to understanding how this is fundamentally a Copyright issue.

Copyright law disallows copying of "too much text" (and a bunch of other stuff). Any move to make it easy for the (human) reader to manipulate too much text—nevermind to share or somehow republish!—takes the developer into dangerous waters. Bad enough for individual companies, from Amazon down, totally a no-go for any type of online, hosted services / platform, let alone a standardized, interoperable one. The legal, political will is not there, and not enough mass-market readers want or need annotation features, so… no one bothers. They do the bare minimum they can get away with, and sometimes we're lucky and they leave a raw text file lying around for plugins like this one to do something with them.

In other words Copyright law has a chilling effect on technology development and marketing for consumer manipulation of content. This has severe repercussions on human cultures, but that's another topic.

In any case, I am SO SO grateful to you do, @davidfor, for maintaining and advancing Annotations. Thank you.
bopuc is offline   Reply With Quote
Old 09-04-2018, 01:09 PM   #544
danhof
Junior Member
danhof began at the beginning.
 
Posts: 1
Karma: 10
Join Date: Sep 2018
Device: Tolino
Quote:
Originally Posted by davidfor View Post
Attached is a beta that adds support for the tolino devices. The annotations are read from the "notes.txt" file in the root partition of the device. This is similar to the clippings file in Kindle devices.

[...]

If anyone with a tolino can test this and tell me how it goes, it would be much appreciated.
Hello David,

Thank you so much for providing this plugin.

I have checked it with my Tolino Vision 4 HD and it works fine. However, I assume that there are different locations where Notes are stored on the device. There is the file notes.txt where notes are included as soon as they are created. But if you change or delete these notes later, these changes are not reflected in this file. Thererefore, the 'actual' notes must be saved at a different location.

Do you happen to know where these data are stored and do you see any chance to fetch these "original" notes as well?

Warm regards,
Daniel
danhof is offline   Reply With Quote
Old 09-04-2018, 09:06 PM   #545
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 15,464
Karma: 25374506
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo,Aura H2O,Glo HD,Aura ONE,Clara HD,Forma;tolino epos
Quote:
Originally Posted by danhof View Post
Hello David,

Thank you so much for providing this plugin.

I have checked it with my Tolino Vision 4 HD and it works fine. However, I assume that there are different locations where Notes are stored on the device. There is the file notes.txt where notes are included as soon as they are created. But if you change or delete these notes later, these changes are not reflected in this file. Thererefore, the 'actual' notes must be saved at a different location.

Do you happen to know where these data are stored and do you see any chance to fetch these "original" notes as well?
All that is correct. The "notes.txt" file is effectively a log of annotations. It isn't used when you look at the annotations while reading on the device. Those are somewhere else. I can't see any likely files when the device is connected, so I assume they are on a partition that isn't made available to the PC. That makes them inaccessible and impossible to use.

There are instructions for rooting the tolino devices. I assume that will make more of the filesystem available. Apart from a little curiosity about what is hidden, and how various pieces of information are stored, I'm not very interested in doing this.

At this point, this is the only way to get any sort of annotations from the device. It's better than nothing, but I'm not sure how much.
davidfor is offline   Reply With Quote
Old 09-14-2018, 03:11 AM   #546
TGSET
Junior Member
TGSET began at the beginning.
 
Posts: 7
Karma: 10
Join Date: Jan 2017
Device: kobo glo
On my Kobo I store fetched annotations in comments and want to change that behavior. Under customize the "custom column wizard" asked me if I want to move the existing annotations to the new column. Fabulous! but the task threw an error (which of course I didn't copy). The column was created but no notes appeared. I've tried several times since with various column names but it never asks if I want existing notes moved, nor has it had further errors.

It appears I can create and likely populate a new column with new notes but I'm unsure it will move existing notes as well since it no longer asks permission. I fear completing the new column add and having notes in two places. It appears to work but I'd like to be absolutely sure ALL notes will be in only one place. Any guidance or suggestions?

BTW, much thanks to all Calibre developers - it's the best FOSS out there!
TGSET is offline   Reply With Quote
Old 09-14-2018, 03:58 AM   #547
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 15,464
Karma: 25374506
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo,Aura H2O,Glo HD,Aura ONE,Clara HD,Forma;tolino epos
Quote:
Originally Posted by TGSET View Post
On my Kobo I store fetched annotations in comments and want to change that behavior. Under customize the "custom column wizard" asked me if I want to move the existing annotations to the new column. Fabulous! but the task threw an error (which of course I didn't copy).
Had you restarted calibre after creating the new column? That's the simplest reason for an error. Another is if the annotations in the comments field had been broken in some form. If it happens again, seeing the details will help.
Quote:
The column was created but no notes appeared. I've tried several times since with various column names but it never asks if I want existing notes moved, nor has it had further errors.

It appears I can create and likely populate a new column with new notes but I'm unsure it will move existing notes as well since it no longer asks permission. I fear completing the new column add and having notes in two places. It appears to work but I'd like to be absolutely sure ALL notes will be in only one place. Any guidance or suggestions?
From memory, if there are no annotations in the column, then it doesn't offer to move them. What you can do is change the annotations back to the comment column. That should work without any actual data changes. Then you can change it to the new column. And hopefully it will work without error. If there is an error, grab it and post it.

But, before doing that, do a backup of your calibre library. Simply copying the library directory will be enough. You can also, take a copy of the database. This is the file "metadata.db" inside your library. Take a copy of this, do the experiment and if it fails, restore it immediately.
davidfor is offline   Reply With Quote
Old 09-16-2018, 03:28 AM   #548
TGSET
Junior Member
TGSET began at the beginning.
 
Posts: 7
Karma: 10
Join Date: Jan 2017
Device: kobo glo
Well, that worked (Thank you) but I think I lost all my notes the first time through. I have a new column but all that it contains are bookmarks. I suppose I lost them somewhere in prior attempts. The comment field of my books that contained notes end with a separator, just FYI.

Backuos: Lesson learned (once again).
TGSET is offline   Reply With Quote
Old 09-27-2018, 01:55 AM   #549
ilovejedd
hopeless n00b
ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.
 
ilovejedd's Avatar
 
Posts: 3,022
Karma: 7343536
Join Date: Jan 2009
Location: in the middle of nowhere
Device: Paperwhite 3, iPad 9.7, iPad 12.9
Quote:
Originally Posted by davidfor View Post
Yes, I will consider adding that. I would need some sample files to look at. The simplest is probably the CSV, but the marvinz would be good to look at.

Having said that, the Marvin XD plugin will probably be revived. From it's thread, the changes in Marvin should mean external applications can reach the data files.
I saw mention of this plugin in the Understanding Metadata thread.

Given Apple closed access to iOS devices and I believe Marvin XD is effectively defunct, I wonder if you might revisit considering adding Marvin 3 to the plugin's list of supported apps?

The csv export is as follows:
Code:
ID,Type,Colour,Title,Author,Date,Created,Updated,Starred,Latitude,Longitude,Address,City,State,Country,HasPhoto,PhotoDate,PhotoLatitude,PhotoLongitude,EntryText,HighlightText
"0051DBCF-6143-4349-9FC7-ED18719666D6","2","0","A Song of Ice and Fire: A Game of Thrones, A Clash of Kings, A Storm of Swords, A Feast for Crows, and A Dance with Dragons","George R.R. Martin","2017-12-28T17:34:44Z","2017-12-28T17:34:44Z","2017-12-28T17:34:44Z","0","","","","","","","0","","","","","An oak may live three hundred years, a redwood tree three thousand. A weirwood will live forever if left undisturbed. To them seasons pass in the flutter of a moth’s wing, and past, present, and future are one. Nor will your sight be limited to your godswood. The singers carved eyes into their heart trees to awaken them, and those are the first eyes a new greenseer learns to use*…*but in time you will see well beyond the trees themselves."


The marvinz export is pretty much just a zip file containing a bunch of json files.

Each annotation is stored in a separate json file named <uuid>.entry.marvinj and a list of all these files is in manifest.json.

Sample manifest.json:
Code:
{
  "DeviceVersion" : "10.3.3",
  "Payload" : [
    {
      "filename" : "0051DBCF-6143-4349-9FC7-ED18719666D6.entry.marvinj"
    }
  ],
  "FileType" : "Marvin interchange file format",
  "ContentType" : "marvin\/annotations",
  "DateCreated" : 539657301.130649,
  "HardwareName" : "iPad6,11",
  "DeviceName" : "iPad",
  "MarvinBuild" : "34",
  "PayloadVersion" : 1,
  "MarvinVersion" : "3.1.2"
}
Sample entry.marvinj:
0051DBCF-6143-4349-9FC7-ED18719666D6.entry.marvinj
Code:
{
  "DeviceVersion" : "10.3.3",
  "Payload" : {
    "address" : null,
    "selectionSectionName" : "A Dance with Dragons, 34. Bran III",
    "country" : null,
    "photoBase64" : null,
    "tags" : [

    ],
    "dateUpdated" : 536175284.265789,
    "photoDate" : null,
    "uuid" : "0051DBCF-6143-4349-9FC7-ED18719666D6",
    "city" : null,
    "latitude" : null,
    "entryText" : "",
    "dateCreated" : 536175284.265787,
    "state" : null,
    "type" : 2,
    "photoLatitude" : null,
    "selectionSectionIndex" : 362,
    "starred" : false,
    "entryDate" : 536175284.265789,
    "longitude" : null,
    "selectionStart" : 27294,
    "photoLongitude" : null,
    "ID" : 459,
    "selectionEnd" : 27736,
    "selectionText" : "An oak may live three hundred years, a redwood tree three thousand. A weirwood will live forever if left undisturbed. To them seasons pass in the flutter of a moth’s wing, and past, present, and future are one. Nor will your sight be limited to your godswood. The singers carved eyes into their heart trees to awaken them, and those are the first eyes a new greenseer learns to use*…*but in time you will see well beyond the trees themselves.",
    "bookTitle" : "A Song of Ice and Fire: A Game of Thrones, A Clash of Kings, A Storm of Swords, A Feast for Crows, and A Dance with Dragons",
    "hasPhoto" : false,
    "colour" : 0,
    "bookAuthor" : "George R.R. Martin",
    "bookFileHash" : "a06fcbc831509ca9f5d1762585b28edb"
  },
  "FileType" : "Marvin interchange file format",
  "ContentType" : "marvin\/journalentry",
  "DateCreated" : 539657299.963053,
  "HardwareName" : "iPad6,11",
  "DeviceName" : "iPad",
  "MarvinBuild" : "34",
  "PayloadVersion" : 1,
  "MarvinVersion" : "3.1.2"
}
From my observations:
  • selectionSectionName - corresponds to the chapter title
  • selectionSectionIndex - index number of file based on epub's content.opf manifest?
  • selectionStart & selectionEnd - word count within the chapter/section's html file only?
  • ID - I believe Marvin stores all annotations in a single database and this ID corresponds to its index number in Marvin's database
  • bookFileHash - MD5 hash of the epub file

I've got a crap ton of highlights for this particular book (probably too much for fair use sharing) so I didn't attach the actual files but if you want, I can make annotation export files for a book in the public domain for your review.

Thanks!
ilovejedd is offline   Reply With Quote
Old 09-27-2018, 05:00 AM   #550
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 15,464
Karma: 25374506
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo,Aura H2O,Glo HD,Aura ONE,Clara HD,Forma;tolino epos
Quote:
Originally Posted by ilovejedd View Post
I saw mention of this plugin in the Understanding Metadata thread.
I knew I shouldn't have said that Both today or back then.
Quote:
Given Apple closed access to iOS devices and I believe Marvin XD is effectively defunct, I wonder if you might revisit considering adding Marvin 3 to the plugin's list of supported apps?

The csv export is as follows:
Code:
ID,Type,Colour,Title,Author,Date,Created,Updated,Starred,Latitude,Longitude,Address,City,State,Country,HasPhoto,PhotoDate,PhotoLatitude,PhotoLongitude,EntryText,HighlightText
"0051DBCF-6143-4349-9FC7-ED18719666D6","2","0","A Song of Ice and Fire: A Game of Thrones, A Clash of Kings, A Storm of Swords, A Feast for Crows, and A Dance with Dragons","George R.R. Martin","2017-12-28T17:34:44Z","2017-12-28T17:34:44Z","2017-12-28T17:34:44Z","0","","","","","","","0","","","","","An oak may live three hundred years, a redwood tree three thousand. A weirwood will live forever if left undisturbed. To them seasons pass in the flutter of a moth’s wing, and past, present, and future are one. Nor will your sight be limited to your godswood. The singers carved eyes into their heart trees to awaken them, and those are the first eyes a new greenseer learns to use*…*but in time you will see well beyond the trees themselves."


The marvinz export is pretty much just a zip file containing a bunch of json files.

Each annotation is stored in a separate json file named <uuid>.entry.marvinj and a list of all these files is in manifest.json.

Sample manifest.json:
Code:
{
  "DeviceVersion" : "10.3.3",
  "Payload" : [
    {
      "filename" : "0051DBCF-6143-4349-9FC7-ED18719666D6.entry.marvinj"
    }
  ],
  "FileType" : "Marvin interchange file format",
  "ContentType" : "marvin\/annotations",
  "DateCreated" : 539657301.130649,
  "HardwareName" : "iPad6,11",
  "DeviceName" : "iPad",
  "MarvinBuild" : "34",
  "PayloadVersion" : 1,
  "MarvinVersion" : "3.1.2"
}
Sample entry.marvinj:
0051DBCF-6143-4349-9FC7-ED18719666D6.entry.marvinj
Code:
{
  "DeviceVersion" : "10.3.3",
  "Payload" : {
    "address" : null,
    "selectionSectionName" : "A Dance with Dragons, 34. Bran III",
    "country" : null,
    "photoBase64" : null,
    "tags" : [

    ],
    "dateUpdated" : 536175284.265789,
    "photoDate" : null,
    "uuid" : "0051DBCF-6143-4349-9FC7-ED18719666D6",
    "city" : null,
    "latitude" : null,
    "entryText" : "",
    "dateCreated" : 536175284.265787,
    "state" : null,
    "type" : 2,
    "photoLatitude" : null,
    "selectionSectionIndex" : 362,
    "starred" : false,
    "entryDate" : 536175284.265789,
    "longitude" : null,
    "selectionStart" : 27294,
    "photoLongitude" : null,
    "ID" : 459,
    "selectionEnd" : 27736,
    "selectionText" : "An oak may live three hundred years, a redwood tree three thousand. A weirwood will live forever if left undisturbed. To them seasons pass in the flutter of a moth’s wing, and past, present, and future are one. Nor will your sight be limited to your godswood. The singers carved eyes into their heart trees to awaken them, and those are the first eyes a new greenseer learns to use*…*but in time you will see well beyond the trees themselves.",
    "bookTitle" : "A Song of Ice and Fire: A Game of Thrones, A Clash of Kings, A Storm of Swords, A Feast for Crows, and A Dance with Dragons",
    "hasPhoto" : false,
    "colour" : 0,
    "bookAuthor" : "George R.R. Martin",
    "bookFileHash" : "a06fcbc831509ca9f5d1762585b28edb"
  },
  "FileType" : "Marvin interchange file format",
  "ContentType" : "marvin\/journalentry",
  "DateCreated" : 539657299.963053,
  "HardwareName" : "iPad6,11",
  "DeviceName" : "iPad",
  "MarvinBuild" : "34",
  "PayloadVersion" : 1,
  "MarvinVersion" : "3.1.2"
}
From my observations:
  • selectionSectionName - corresponds to the chapter title
  • selectionSectionIndex - index number of file based on epub's content.opf manifest?
  • selectionStart & selectionEnd - word count within the chapter/section's html file only?
  • ID - I believe Marvin stores all annotations in a single database and this ID corresponds to its index number in Marvin's database
  • bookFileHash - MD5 hash of the epub file

I've got a crap ton of highlights for this particular book (probably too much for fair use sharing) so I didn't attach the actual files but if you want, I can make annotation export files for a book in the public domain for your review.
That doesn't look to bad. I'll have a look at it, but, I won't promise when or if I will do anything. But, if someone is interested in doing it, I will be happy to answer any questions and add it the plugin.
davidfor is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Open With kiwidude Plugins 332 08-31-2018 09:54 AM
[GUI Plugin] KindleUnpack - The Plugin DiapDealer Plugins 412 04-24-2018 07:07 PM
[GUI Plugin] Annotations (closed) GRiker Plugins 68 06-17-2014 07:11 AM
[GUI Plugin] KiNotes -axel- Plugins 0 07-14-2013 07:39 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 01:27 PM


All times are GMT -4. The time now is 10:34 PM.


MobileRead.com is a privately owned, operated and funded community.