Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Closed Thread
 
Thread Tools Search this Thread
Old 06-04-2013, 01:16 PM   #16
GRiker
Comparer of the Ephemeris
GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.
 
Posts: 1,496
Karma: 424697
Join Date: Mar 2009
Device: iPad
nameless66, I'd like to test with the .MRV file you're using. Please send me a pm, and I will reply with instructions on how to send it to me.

G
GRiker is offline  
Old 06-21-2013, 06:17 PM   #17
-axel-
Junior Member
-axel- could sell banana peel slippers to a Deveel.-axel- could sell banana peel slippers to a Deveel.-axel- could sell banana peel slippers to a Deveel.-axel- could sell banana peel slippers to a Deveel.-axel- could sell banana peel slippers to a Deveel.-axel- could sell banana peel slippers to a Deveel.-axel- could sell banana peel slippers to a Deveel.-axel- could sell banana peel slippers to a Deveel.-axel- could sell banana peel slippers to a Deveel.-axel- could sell banana peel slippers to a Deveel.-axel- could sell banana peel slippers to a Deveel.
 
Posts: 6
Karma: 3088
Join Date: Apr 2013
Device: Kindle Paperwhite
Kindle international

Quote:
Originally Posted by GRiker View Post
@-axel-,
I fixed the Position/Location error, version 1.1.2 should work properly for you.
G
Hi,

sorry, but it did not work for me. I've had a deeper look into the Kindle Clippings format and did learn some Python. I'll try to attach my version of Kindle.py, where the core parsing is done in a separate file ParseKindleMyClippingsTxt.py. (Actually I added a complete plugin zip file, so you can easily figure out the version I worked with and maybe it even runs out of the box.)

You are very welcome to use it and itegrate it into your plugin, but please resist the temptation to refactor ParseKindleMyClippingsTxt.py into your style and interface conventions. So in case I will use it for my project we both can benefit from improvements.

It turned out locale is worthless for a module parsing a multi-language file, because it is not thread-safe and not really portable among OS; Windows uses different locale names than Linux and Python does not supply an abstraction level for that. But the killer problem was thread-safety, which means that a module is not allowed to change locale, not even for a short time.

I've had a look into the Calibre code you were using for date parsing. Funny thing is, that module contains a helper function which replaces French and German month names by the English names. Inspired by this I did a similar thing and implemented a simple multilingual parsing which works for the format my Paperwhite generated, for some examples of My Clippings.txt content I googled (most of them here on the forum), and for a few format variations I considered worth covering without examples.

The multilingual parsing concept is a mix of table based and procedural encoding. It would hope that way it has the flexibility to be easily extended to cover future variations of Amazons My Clippings.txt format, including added languages.

Here are a few notes on limitations I observed for "My Clippings.txt":

It seems to be an append-only file. Changing Kindle language will not change the content of the file. Only new annotations are affected by the new language setting.

Also if a note, mark or bookmark is deleted, it will *not* be deleted from the txt file. Same holds for edited notes.

Using the timestamp as a unique ID does not always work, even if the date can be read perfectly. I had a case where two annotations, done a few seconds apart, did get the same timestamp.

The format of My Clippings.txt is ambiguous. E.g. I can add a note which looks like the separator line "==========". You may think that is a pretty academic case, but consider for example the My Clippings.txt is also a Kindle document. You can open it an highlight parts of it. With a bit of bad luck, the highlight will even be indistinguishable from a separate entry in My Clippings.txt. I did give up on the last case, but I tried to cover the simpler cases with my parser.

Not having such brain-twisting ambiguities in logfiles was one of the major reasons XML was invented. Which brings me to my last remark here: there is also an XML file system/userannotlog in the hidden but readable system folder of the Kindle, which contains a protocol of all annotation operations. From this file I can figure out deletions, edits, etc. Timestamps are in easy web format, no language translations involved. Locations are in full resolution, i.e. the identify start and end character of each highlight.

So that would be an interesting alternative, but not a perfect one. The log does not contain the highlighted text itself. And I have no information on how stable the format was or will be over the years. For example google found me a file userannotlog.0, which indicates there is or was some splitting strategy / naming convention to prevent the file to become too large. Does anyone know more?
Attached Files
File Type: zip Annotations.zip (598.9 KB, 433 views)
-axel- is offline  
Advert
Old 06-22-2013, 12:20 PM   #18
CtB
Member
CtB began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Jan 2013
Device: iPad
Re-Use of highlights

Greg,
Thanks for the Annotations plug-in for Marvin [the BEST reader I've seen by far].

Maybe confused, a question: Your annotation button works in Calibre. I export all bookmarks/highlights from Marvin and get a .mrv file. I then drag it into your plug in. When I look at the metadata of the book in Calibre, the highlights are there [as "comments"].

However, if I open the book in Calibre [in the native e-book browser] or put in a DropBox and reimport into Marvin, all the bookmarks/highlights cannot be seen. In fact in Marvin, the bookmarks and highlights dialog boxes are blank (as if the bookmarks no longer exist and are lost).

I remember that there was early discussion that the .mrv (annotations) file would allow users to export/save their highlights/bookmarks, then re-import/restore them later or e-mail/send to friends to import into their copy of the Marvin book on their end.
Ideally, I would like my highlights/bookmarks to show up "in" my book when I restore them, not be hidden in a comments file OR send them to a friend and he imports the .mrv file and has a complete pre-highlighted book with all highlights/bookmarks where I want them to be in his copy for him to skim the important parts of the book faster.

Am I not re-importing the .mrv file properly (through your plug in)? Do I need to do something different to restore the highlights back into the Marvin book so they are visible? How do I e-mail/share the .mrv file to another Marvin user so he can see the highlighted book just as I highlighted it (not in an invisible "comments" file)?

Thanks!!
CtB is offline  
Old 06-22-2013, 12:25 PM   #19
GRiker
Comparer of the Ephemeris
GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.
 
Posts: 1,496
Karma: 424697
Join Date: Mar 2009
Device: iPad
Quote:
Originally Posted by CtB View Post
Greg,
Thanks for the Annotations plug-in for Marvin [the BEST reader I've seen by far].

Maybe confused, a question: Your annotation button works in Calibre. I export all bookmarks/highlights from Marvin and get a .mrv file. I then drag it into your plug in. When I look at the metadata of the book in Calibre, the highlights are there [as "comments"].

However, if I open the book in Calibre [in the native e-book browser] or put in a DropBox and reimport into Marvin, all the bookmarks/highlights cannot be seen. In fact in Marvin, the bookmarks and highlights dialog boxes are blank (as if the bookmarks no longer exist and are lost).

I remember that there was early discussion that the .mrv (annotations) file would allow users to export/save their highlights/bookmarks, then re-import/restore them later or e-mail/send to friends to import into their copy of the Marvin book on their end.
Ideally, I would like my highlights/bookmarks to show up "in" my book when I restore them, not be hidden in a comments file OR send them to a friend and he imports the .mrv file and has a complete pre-highlighted book with all highlights/bookmarks where I want them to be in his copy for him to skim the important parts of the book faster.

Am I not re-importing the .mrv file properly (through your plug in)? Do I need to do something different to restore the highlights back into the Marvin book so they are visible? How do I e-mail/share the .mrv file to another Marvin user so he can see the highlighted book just as I highlighted it (not in an invisible "comments" file)?

Thanks!!
A couple of clarifications.
1) Annotations and highlights are never part of the book you're reading, they are created by Marvin externally to the book, and exported/extracted by the plugin. So you will never see the annotations/highlights in any reader other than Marvin. If that's a feature you're interested in, you should communicate that to Marvin's developer.
2) Sharing/reimporting your .MRV file to Marvin is also exclusively a Marvin feature that the plugin is not involved with. I would suggest you ask that question in this Marvin thread in the Apple forum.

Hope that helps.

G
GRiker is offline  
Old 06-22-2013, 12:57 PM   #20
CtB
Member
CtB began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Jan 2013
Device: iPad
Foolow up on Highlight/bookmarks portability back into Marvin

Greg,
Just saw the other blog on Marvin sight about "polishing" books.

I will try that to see if it works so that the highlights/bookmarks are restored into the book and visible in Marvin. I'll report back.

Thanks!
CtB is offline  
Advert
Old 06-22-2013, 12:59 PM   #21
GRiker
Comparer of the Ephemeris
GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.
 
Posts: 1,496
Karma: 424697
Join Date: Mar 2009
Device: iPad
Quote:
Originally Posted by CtB View Post
Greg,
Just saw the other blog on Marvin sight about "polishing" books.

I will try that to see if it works so that the highlights/bookmarks are restored into the book and visible in Marvin. I'll report back.

Thanks!
I assure you it will not add highlights to your books.

G
GRiker is offline  
Old 06-22-2013, 12:59 PM   #22
CtB
Member
CtB began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Jan 2013
Device: iPad
OK,

Just saw this too. I will contact Kristian.

Thanks!
CtB is offline  
Old 06-22-2013, 01:00 PM   #23
CtB
Member
CtB began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Jan 2013
Device: iPad
Thanks for quick reply here too; will save me time experimenting.

I think it's back to Kristian.
CtB is offline  
Old 07-26-2013, 06:36 AM   #24
Rob S
Junior Member
Rob S began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Jul 2013
Device: nook simple touch
Thank you GRiker.
The examples you include are with devices other then a nook. Does the plug in work with the nook simple touch?
Thanks again,
Rob
Rob S is offline  
Old 07-26-2013, 06:40 AM   #25
GRiker
Comparer of the Ephemeris
GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.
 
Posts: 1,496
Karma: 424697
Join Date: Mar 2009
Device: iPad
@Rob S:
The plugin does not currently support Nook, but it is open to supporting Nook if someone wants to write the code to support it.

G
GRiker is offline  
Old 09-14-2013, 01:25 AM   #26
Wyndham
Connoisseur
Wyndham has a complete set of Star Wars action figures.Wyndham has a complete set of Star Wars action figures.Wyndham has a complete set of Star Wars action figures.
 
Wyndham's Avatar
 
Posts: 96
Karma: 250
Join Date: Mar 2011
Location: Cricklewood (London)
Device: iPad Air 2 (iOS 9), iPhone 6 (iOS 9)
Hey there.

Just wanted to say thank you again for the connector.

I downloaded Marvin for iPhone last night and loading the library was a pleasure it was so easy.

Loaded 35 books initially
Then changed a collection tag in calibre
Loaded 200 books, including the first 35 which updated nicely
Then loaded the balance in that library of about 1100 books
Switched to another library
Loaded 1900 plus books

All ran smoothly apart from the occasional message about unsuitable source for converting or corrupt ePub - all valid messages.

Worked really well. Brilliant

Thanks again for a superb program in calibre, and of course now this connector.

For the record, Marvin upgraded the library in my iPad smoothly and I have encountered no problems at all.
Wyndham is offline  
Old 09-14-2013, 08:11 AM   #27
GRiker
Comparer of the Ephemeris
GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.
 
Posts: 1,496
Karma: 424697
Join Date: Mar 2009
Device: iPad
@Wyndham, thanks for the feedback, I'm glad it's working well for you. You might consider posting a similar message in the Marvin subforum so other users may benefit from your experiences.

G
GRiker is offline  
Old 09-14-2013, 10:53 AM   #28
Sabardeyn
Guru
Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.
 
Sabardeyn's Avatar
 
Posts: 644
Karma: 1242364
Join Date: May 2009
Location: The Right Coast
Device: PC (Calibre), Nexus 7 2013 (Moon+ Pro), HTC HD2/Leo (Freda)
Just wanted to note that Message #1 should include a clear list of supported ereader hardware and software.
Sabardeyn is offline  
Old 09-14-2013, 11:38 AM   #29
GRiker
Comparer of the Ephemeris
GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.
 
Posts: 1,496
Karma: 424697
Join Date: Mar 2009
Device: iPad
@Saberdeyn, I modified the opening sentence in the first post to call more attention to the table immediately below that lists the ereader hardware and software supported by the plugin.

G
GRiker is offline  
Old 12-01-2013, 03:16 PM   #30
spitfire_ch
Enthusiast
spitfire_ch began at the beginning.
 
Posts: 28
Karma: 10
Join Date: May 2012
Device: Sony PRS-T1
Sony PRS-T1

Your plugin looks exactly like what I have been looking for a long time: getting all my annotations off my Sony PRS-T1 into another format which I can consume on other devices (should the PRS-T1 ever break).

Well, almost, unfortunately it does not support the PRS-T1. I have been dealing with the SQLite DBs Sony is using on the device, and I might be able to extract the highlights from the DBs (the highlighted text is easy, the tricky part is to figure our how the position is saved). My questions are the following:

1) How does your plugin integrate the annotations in Calibre? What format do the annotation files from kindle and Marvin have? Maybe I could try to convert the Sony extracts into the correct format and then import them using your Plugin.

2) When you convert an epub with imported annotations to a pdf, will the annotations also be exported? That would be paradise!

Thanks for your feedback and best regards
- spitfire_ch
spitfire_ch is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Annotations davidfor Plugins 1152 03-22-2024 09:38 PM
[GUI Plugin] KindleUnpack - The Plugin DiapDealer Plugins 492 10-25-2022 08:13 AM
[GUI Plugin] Marvin XD (closed) GRiker Plugins 141 06-15-2014 05:36 PM
Plugin thread closed? BookJunkieLI Calibre 3 10-05-2013 02:56 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 01:34 PM.


MobileRead.com is a privately owned, operated and funded community.