Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Closed Thread
 
Thread Tools Search this Thread
Old 12-19-2013, 01:44 AM   #46
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,908
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
I have been remiss...

The new version of the plugin is working well. And I have a working class for the Kobo devices. It just needs a little more testing. If there is a willing guinea pig out there, please contact me.
davidfor is offline  
Old 12-19-2013, 05:08 AM   #47
Terisa de morgan
Grand Sorcerer
Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.
 
Terisa de morgan's Avatar
 
Posts: 6,211
Karma: 11766195
Join Date: Jun 2009
Location: Madrid, Spain
Device: Kobo Clara/Aura One/Forma,XiaoMI 5, iPad, Huawei MediaPad, YotaPhone 2
If you want I can add some annotations and test it.
Terisa de morgan is offline  
Advert
Old 12-19-2013, 09:23 PM   #48
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,908
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Thanks Terisa. I have sent you code and some instructions.
davidfor is offline  
Old 12-26-2013, 04:17 AM   #49
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,908
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Bug or feature?

Thanks to Terisa and gwalden who are testing the annotations for the Kobo devices, it is very close to being releasable. But, both noticed something I didn't and now I can't work out if it is a bug in my code, the plugin or whether "it's not a bug, it's feature" applies.

The problem is when an annotation is changed on the device after they have been fetched. When the annotations are fetched again, there are two entries for the annotation. The first has the original text and the second has the new text. When I do another fetch, the original version of the annotation is removed.

So, I can't decide if this is a bug or not. I can see that what I describe might be deliberate. I don't think it is my code, as I am simply building the current list of annotations and passing it on to update the metadata.

I did have a quick look at the code that rebuilds the annotation text, but it didn't jump out at me as to whether what is happening is deliberate or not.
davidfor is offline  
Old 12-26-2013, 05:32 AM   #50
GRiker
Comparer of the Ephemeris
GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.
 
Posts: 1,496
Karma: 424697
Join Date: Mar 2009
Device: iPad
The intent is to merge already-stored annotations with new annotations. Merging is done in annotations:merge_annotations() around line #229. Replication of existing annotations is avoided by comparing hashes. I'm not sure what would cause the behavior you're describing.

I am traveling with infrequent internet access until early January. When I get a chance, I will do some testing to see if I can replicate the behavior you're describing. In the meantime, if you identify an issue with the current code, please create a new issue at GitHub.

G
GRiker is offline  
Advert
Old 01-02-2014, 11:36 AM   #51
spitfire_ch
Enthusiast
spitfire_ch began at the beginning.
 
Posts: 28
Karma: 10
Join Date: May 2012
Device: Sony PRS-T1
Quote:
Originally Posted by davidfor View Post
I have been remiss...

The new version of the plugin is working well. And I have a working class for the Kobo devices. It just needs a little more testing. If there is a willing guinea pig out there, please contact me.
I don't have a kobo - hence I don't qualify as guinea pig. However, I am highly interested in your class for the Kobo devices. I am still trying to wrap my head around Sony annotations, and from what I have seen somewhere else, Sony and Kobo might use the same or at least a similar system to store annotations. Sony uses something similar to a canonical fragment identifier:
text/part0000_split_005.html#point(/1/4/9:2) (one position before the beginning of the highlight)
text/part0000_split_005.html#point(/1/4/10/1:1) (end of highlight)

Nook must use something alike:
https://www.mobileread.com/forums/showthread.php?t=99000
Quote:
Originally Posted by Reader2
Most of these are about 6/700 bytes but annotated are must bigger and contain things like:
#point(/1/4/150/5:3)
Looks the same to me

I already partially figured out how this works:
/1 indicates the root element (<html...)
even numbers navigate through the elements. So 2 is the first child of the root (<head...), 4 is the second child of the root (<body...), /1/4/10/ is the 5th child of the body (usually the 5th paragraph or header).
: idicates the offset within the element.

However, I am not quite sure what odd numbers mean. They usually occur if the highlight starts at the beginning of the paragraph. So the position before the highlight is "between the paragraphs". But in case of /1/4/9:2 --> what's the second character of "between paragraph 4 and 5? Do you happen to know that, by chance?

So, I am almost there, but not quite. I was hoping that looking at your code might help me to figure out the remaining gaps - and also give me an idea to write code to navigate using this kind of identifiers.

Thank you very much for any ideas - or the code ...
... and of course all the very best in 2014!!

- spitfire
spitfire_ch is offline  
Old 01-02-2014, 08:33 PM   #52
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,908
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Quote:
Originally Posted by spitfire_ch View Post
I don't have a kobo - hence I don't qualify as guinea pig. However, I am highly interested in your class for the Kobo devices. I am still trying to wrap my head around Sony annotations, and from what I have seen somewhere else, Sony and Kobo might use the same or at least a similar system to store annotations. Sony uses something similar to a canonical fragment identifier:
text/part0000_split_005.html#point(/1/4/9:2) (one position before the beginning of the highlight)
text/part0000_split_005.html#point(/1/4/10/1:1) (end of highlight)

Nook must use something alike:
https://www.mobileread.com/forums/showthread.php?t=99000

Looks the same to me
Yes, they are pretty much the same. And the reason is that they are the format ADE uses. Kobo and Sony uses a version of the Adobe RMSDK for reading epubs. In fact, as well as storing the annotations in a database, the Kobo writes them to a file. This file is identical to what ADE uses for annotations on Windows. I have copied the file between the two with some massaging.

I should add, the Kobo devices use two different readers. The other is for their own modified kepubs. The other reader uses a different bookmarking method.
Quote:
I already partially figured out how this works:
/1 indicates the root element (<html...)
even numbers navigate through the elements. So 2 is the first child of the root (<head...), 4 is the second child of the root (<body...), /1/4/10/ is the 5th child of the body (usually the 5th paragraph or header).
: idicates the offset within the element.
Yes, that is my understanding as well.
Quote:
However, I am not quite sure what odd numbers mean. They usually occur if the highlight starts at the beginning of the paragraph. So the position before the highlight is "between the paragraphs". But in case of /1/4/9:2 --> what's the second character of "between paragraph 4 and 5? Do you happen to know that, by chance?
It has been a while since I looked at this. I think that the count is for opening and closing tags at the same level and starts from zero. The count seems to be ignoring self closed tags, but I'd need to do some more testing to prove that. Each number definitely represents a level of nesting. I have a blockquote with several paragraphs in it. The paragraphs are counted within that.

I might have a bit of a play later. It will give me something to do while hiding from the in-laws.
Quote:
So, I am almost there, but not quite. I was hoping that looking at your code might help me to figure out the remaining gaps - and also give me an idea to write code to navigate using this kind of identifiers.
Unfortunately, it won't help. I don't touch the locations. I am only pulling out the highlighted text, the note, a timestamp and enough information to give the order within the list of annotations.

And my Kobo Utilities plugin won't help. It will store the current reading position, but it is the full string. I have considered trying to translate this to the method that calibre uses. That way I could update the position for the calibre viewer, but I haven't seen the need yet. I would actually be interested to know if the reading position on the Sony devices are stored somewhere accessible and in what format. Because of the common Adobe RMSDK, I have been wondering of the Kobo position could be copied to the Sony devices.
Quote:
Thank you very much for any ideas - or the code ...
If you want the code, I can PM you a link.
Quote:
... and of course all the very best in 2014!!
Yes, Happy New Year to everyone.
davidfor is offline  
Old 01-03-2014, 03:11 PM   #53
spitfire_ch
Enthusiast
spitfire_ch began at the beginning.
 
Posts: 28
Karma: 10
Join Date: May 2012
Device: Sony PRS-T1
Thank you very much for your fast, friendly and helpful reply! That helps a great deal, I did not know it was the same system ADE uses. That opens new sources of information And, maybe even the possibility to port the annotations to ADE? I'll have to look into that. My goal is basically to translate the Sony annotations to some other system, so that I can finally export the annotations together with the book to some other reader app. I don't like depending on one physical device in order to make use of my annotations. Understanding the system Sony uses is the first step to that.

Quote:
I would actually be interested to know if the reading position on the Sony devices are stored somewhere accessible and in what format. Because of the common Adobe RMSDK, I have been wondering of the Kobo position could be copied to the Sony devices.
It's done in the exact same way the annotations are stored, just in a different table. The Sony reader contains a couple of SQLite DBs, the most important one being the books.db. Within the books.db, you'll find a table named "current_position". It has the structure seen in the attachment "current_position.png". This table contains a blob column called "mark", which encodes the location in the familiar way:
OEBPS/9781118087459c09.xhtml#point(/1/4/2/276/1:453)
If you are interested, you can pm me and I will send you the entire books.db, so that you can have a look at it first hand.

Best regards, spitfire
spitfire_ch is offline  
Old 01-03-2014, 08:05 PM   #54
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,908
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Quote:
Originally Posted by spitfire_ch View Post
Thank you very much for your fast, friendly and helpful reply! That helps a great deal, I did not know it was the same system ADE uses. That opens new sources of information And, maybe even the possibility to port the annotations to ADE? I'll have to look into that. My goal is basically to translate the Sony annotations to some other system, so that I can finally export the annotations together with the book to some other reader app. I don't like depending on one physical device in order to make use of my annotations. Understanding the system Sony uses is the first step to that.
I can understand that. When I started looking at annotations and reading positions, I considered getting them into the same format the calibre viewer uses. But, it doesn't support annotations. And I haven't broken down and started looking at it myself. But, it might be possible to generate the ADE annotations file. That could be useful. But, there is an id in it that I think needs to be updated for the different readers.

The Annotations plugin might be right for you, so long as you don't want to send them back to a device. The fetching and display works well. I had a misunderstanding about an intention of how part of this works. My usual use for annotations is temporary notes for things I need to fix in the book. GRiker intends it a more permanent notes. Once he explained that, I can see the reason for some of the behaviour that had baffled me.
Quote:
It's done in the exact same way the annotations are stored, just in a different table. The Sony reader contains a couple of SQLite DBs, the most important one being the books.db. Within the books.db, you'll find a table named "current_position". It has the structure seen in the attachment "current_position.png". This table contains a blob column called "mark", which encodes the location in the familiar way:
OEBPS/9781118087459c09.xhtml#point(/1/4/2/276/1:453)
If you are interested, you can pm me and I will send you the entire books.db, so that you can have a look at it first hand.
Firstly, the attachment isn't displaying. You might need a few more posts for that to work. Or did you forget to attach it?

This is what I store for the reading position:
Code:
(4)OEBPS/Text/119579.xhtml|@ @|OEBPS/Text/119579.xhtml#point(/1/4/260/1:61)
That comes from two columns on different rows in the Kobo database. The "|@ @|" is a separator I added so I only needed one column. And I could probably get rid of the first part with a bit of thought, but it was easier to do. If the Sony database works like you are describing, it should be easy* to transfer the reading between Sony and Kobo device. I almost wish I had a Sony device to try it on.

I would like to see the full database to confirm this. I'll send you a PM.

* "easy" - Term used by programmers to describe a programming task they haven't really looked at but will probably take months of work.
davidfor is offline  
Old 01-03-2014, 08:19 PM   #55
PeterT
Grand Sorcerer
PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.
 
PeterT's Avatar
 
Posts: 12,119
Karma: 73448614
Join Date: Nov 2007
Location: Toronto
Device: Nexus 7, Clara, Touch, Tolino EPOS
Quote:
Originally Posted by davidfor View Post
* "easy" - Term used by programmers to describe a programming task they haven't really looked at but will probably take months of work.
Isn't actually a term used by management to describe a task that will require a total re architecturing of the project to implement a feature that no user will ever utilize?
PeterT is offline  
Old 01-03-2014, 09:39 PM   #56
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,908
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Quote:
Originally Posted by PeterT View Post
Isn't actually a term used by management to describe a task that will require a total re architecturing of the project to implement a feature that no user will ever utilize?
It's all a matter of your point of view, but it sounds like you've me the managers on my last big project.
davidfor is offline  
Old 01-04-2014, 02:07 AM   #57
Terisa de morgan
Grand Sorcerer
Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.Terisa de morgan ought to be getting tired of karma fortunes by now.
 
Terisa de morgan's Avatar
 
Posts: 6,211
Karma: 11766195
Join Date: Jun 2009
Location: Madrid, Spain
Device: Kobo Clara/Aura One/Forma,XiaoMI 5, iPad, Huawei MediaPad, YotaPhone 2
Well, you can think about a customer asking for a major feature they want for free, too.
Terisa de morgan is offline  
Old 02-02-2014, 01:38 PM   #58
astrodad
Guru
astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.
 
astrodad's Avatar
 
Posts: 962
Karma: 568242
Join Date: Dec 2007
Device: Kindle Oasis 3, Kindle Paperwhite 1, iPad, iPhone
Quick question. I see a class called _iosKindle.py in Github but it's not part of the annotation plugins downloaded through Calibre. I went through the steps of adding the file to the additional_readers config in the annotations.json file but I don't see the iOS kindle device in the menu. Is there something I am missing?
astrodad is offline  
Old 02-02-2014, 02:36 PM   #59
GRiker
Comparer of the Ephemeris
GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.GRiker ought to be getting tired of karma fortunes by now.
 
Posts: 1,496
Karma: 424697
Join Date: Mar 2009
Device: iPad
The classes preceded with a leading underscore are sample code or incomplete implementations. Kindle for iOS is incomplete because I couldn't figure out how to extract the highlight color from the database. You're welcome to work with it. If you have any implementation questions please post them in the GitHub project.

G
GRiker is offline  
Old 02-03-2014, 09:52 AM   #60
astrodad
Guru
astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.astrodad ought to be getting tired of karma fortunes by now.
 
astrodad's Avatar
 
Posts: 962
Karma: 568242
Join Date: Dec 2007
Device: Kindle Oasis 3, Kindle Paperwhite 1, iPad, iPhone
I am not a good Python developer but I might be able to hack it, having written a little calibre code in the past. I guess I'm just trying to figure out what's missing since it looks like the highlighter code is just assuming it's yellow. I'll post over at the GitHub project.
astrodad is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Annotations davidfor Plugins 1152 03-22-2024 09:38 PM
[GUI Plugin] KindleUnpack - The Plugin DiapDealer Plugins 492 10-25-2022 08:13 AM
[GUI Plugin] Marvin XD (closed) GRiker Plugins 141 06-15-2014 05:36 PM
Plugin thread closed? BookJunkieLI Calibre 3 10-05-2013 02:56 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 01:35 AM.


MobileRead.com is a privately owned, operated and funded community.