Would a Calibre plugin to extract reading progress from the kindle be useful?

tpb · 11-23-2024, 05:14 PM

I've built some scripts to automatically extract metadata from the Kindle and sync it to user defined custom columns in Calibre. This lets me sort my book collection by read/unread, date last read, number of reads, etc.

The data is provided by KRDS:info like position, completion status, hours read, etc.

If there is interest, I could clean this up and publish it as a calibre plugin. Years ago I saw quite a few threads about this, but I'm not sure if amazon has ever added this ability to newer kindles as mine is quite old now.

For me, my kindle only shows 0-100%, and that percent is reset if you ever open the book to a different page so your progress is lost. There's also no way to see if you read a book more than once, or sort by progress/last read.

This could also be easy to expand to other devices if extractors like KRDS exist on those platforms, but I haven't looked into that yet.

Defenderland · 11-25-2024, 01:04 PM

This sounds fantastic.
I personally would be most interested (besides the hours spent on a book being displayed in a column) in the calculation of the reading speed (WPM - words per minute).

Reading without any thoughts on measurement and afterwards checking it, could give interesting insights and comparisons (progress when learning a language, light literature vs heavy, speed variying with author, genre, ...).

JSWolf · 11-25-2024, 01:17 PM

Can the reading position be put back on the Kindle?

I don't know why there is an option for a different device as Kobo Utilities can already get reading positions and put them on the Kobo. This is a plugin for calibre. So we already have a different device able to do this.

And @tpb, if you want this to be a plugin, just make it so. That's how most plugins came to be. Someone wanted it and was able to make it.

tomsem · 11-26-2024, 02:14 PM

Quote:

Originally Posted by tpb

For me, my kindle only shows 0-100%, and that percent is reset if you ever open the book to a different page so your progress is lost. There's also no way to see if you read a book more than once, or sort by progress/last read.

(for current Kindle) Percent reflects reading location in the book, if you change the location, it changes the percent complete. Amazon also tracks 'furthest page read' FWIW, I don't know if that's available for extraction (probably not).

Of course end of book may be well before the last location in the book, if there are footnotes and other end matter that are there just for reference purposes. There is metadata (ToC entry) that defines End of reading as set by publisher.

Newer Kindles (2024 models and Scribe) use MTP so if you want to read it directly you'll need to figure out how to use calibre's MTP device interface, probably would need to extend it.

I personally don't have any interest in tracking reading at such a granular level but didn't want to vote 'No' as I would not want to discourage the effort to create a plugin that could do these things.

j.p.s · 12-05-2024, 05:17 PM

Quote:

Originally Posted by tomsem

(for current Kindle) Percent reflects reading location in the book, if you change the location, it changes the percent complete. Amazon also tracks 'furthest page read' FWIW, I don't know if that's available for extraction (probably not).

azw3f files in the .sdr directory for KF8/azw3 books have "fpr" and "lpr" fields each of which has a "position" subfield, where the numerical position is the byte number of that location in the uncompressed marked up book text. The values stored in those fields can be extracted with the (partially yours) utility krds.py
https://www.mobileread.com/forums/sh...d.php?t=322172
https://github.com/K-R-D-S/KRDS

I think that "fpr" is furthest position read and "lpr" is last position read, at least the numbers make sense for the books that I spot checked. (Well, I guess the "p" in fpr and lpr could still be "page" since the subfield is "position".)

KFX has a more complicated position structure, but it probably has a similar file amenable extraction by krds.py

tomsem · 12-05-2024, 06:13 PM

Good to know.

I suppose we need to check to see if any more changes are needed to krds.py (e.g. for active canvas location, underline type, etc.). I'll try to get to this.

Drybonz · 12-06-2024, 01:07 PM

I had actually just come to the forum to find out if there was a way to do this... so, yes!

John F · 12-06-2024, 04:03 PM

Quote:

Originally Posted by JSWolf

...

I don't know why there is an option for a different device as Kobo Utilities can already get reading positions and put them on the Kobo. This is a plugin for calibre. So we already have a different device able to do this.

There are other devices besides Kindle and Kobo? Or am I missing something?

tpb · 12-09-2024, 05:21 PM

Quote:

Originally Posted by tomsem

(for current Kindle) Percent reflects reading location in the book, if you change the location, it changes the percent complete. Amazon also tracks 'furthest page read' FWIW, I don't know if that's available for extraction (probably not).
...
Newer Kindles (2024 models and Scribe) use MTP so if you want to read it directly you'll need to figure out how to use calibre's MTP device interface, probably would need to extend it.

Thanks for this info. So it seems Amazon has still not included the ability to preserve reading progress/read indicators if you ever re-open the book to a different page, which was my main motivation behind this.

I'm really not sure about the MTP issue as a don't have a newer device to test on. Currently I'm just walking through the device path Calibre provides and using KRDS to read contents of the .sdr dirs.

Quote:

Originally Posted by Defenderland

This sounds fantastic.
I personally would be most interested (besides the hours spent on a book being displayed in a column) in the calculation of the reading speed (WPM - words per minute).
.

Good idea. I've added this already and it will be in the initial release.

Quote:

Originally Posted by JSWolf

Can the reading position be put back on the Kindle?.

Yes, I tested this with my kindle using .mobi files and writing the reading position seems to work. Replacing the first two numbers in the lpr struct in the .mbs file makes the kindle open to that new position on the device.

I do not know if this works on other kindles/firmware versions, and I also don't know what risks there may be if the .mbs file is corrupted (e.g. writing a value that is greater than the total positions in the book.). This could be really useful if people have multiple devices though, so I'm open to looking into it.

Defenderland · 12-09-2024, 10:17 PM

Great news. Amazing.

rszasz · 12-21-2024, 11:14 AM

That would be great, its a feature that's been missing since the first PWs changed the metadata formats. (read position and finished status are useful for me in organizing my library)

Defenderland · 03-29-2025, 11:55 AM

Quote:

Originally Posted by tpb

Good idea. I've added this already and it will be in the initial release.

How's the development going?
When will you release it?

View Poll Results: Do you want a plugin to import reading progress from your kindle?
Yes	30	78.95%
No	7	18.42%
Yes, but for a different device.	1	2.63%
Voters: 38. You may not vote on this poll

11-23-2024, 05:14 PM	#1
tpb Junior Member Posts: 2 Karma: 170 Join Date: Nov 2024 Device: Kindle Paperwhite	Would a Calibre plugin to extract reading progress from the kindle be useful? I've built some scripts to automatically extract metadata from the Kindle and sync it to user defined custom columns in Calibre. This lets me sort my book collection by read/unread, date last read, number of reads, etc. The data is provided by KRDS:info like position, completion status, hours read, etc. If there is interest, I could clean this up and publish it as a calibre plugin. Years ago I saw quite a few threads about this, but I'm not sure if amazon has ever added this ability to newer kindles as mine is quite old now. For me, my kindle only shows 0-100%, and that percent is reset if you ever open the book to a different page so your progress is lost. There's also no way to see if you read a book more than once, or sort by progress/last read. This could also be easy to expand to other devices if extractors like KRDS exist on those platforms, but I haven't looked into that yet.

11-25-2024, 01:17 PM	#3
JSWolf Resident Curmudgeon Posts: 79,756 Karma: 145864619 Join Date: Nov 2006 Location: Roslindale, Massachusetts Device: Kobo Libra 2, Kobo Aura H2O, PRS-650, PRS-T1, nook STR, PW3	Can the reading position be put back on the Kindle? I don't know why there is an option for a different device as Kobo Utilities can already get reading positions and put them on the Kobo. This is a plugin for calibre. So we already have a different device able to do this. And @tpb, if you want this to be a plugin, just make it so. That's how most plugins came to be. Someone wanted it and was able to make it. Last edited by JSWolf; 11-25-2024 at 01:20 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Syncing Reading Metrics/Progress in Between Kobo and Calibre	Deobulakenyo	Library Management	6	11-07-2021 10:57 PM
Is there any way to backup/restore reading progress to Calibre?	OrangeFlavored	KOReader	10	05-07-2020 11:33 PM
[Kobo] Reading progress not updated in calibre anymore	69carat	Devices	2	01-16-2020 11:01 PM
Calibre annotations and highlight extract plugin for the Nook Simple Touch?	einkuser	Plugins	15	01-24-2015 08:13 PM
Kindle Reading Progress	the_callant	Devices	0	10-06-2013 04:32 AM

11-25-2024, 01:04 PM	#2
Defenderland Groupie Posts: 181 Karma: 208 Join Date: Feb 2012 Device: Colorsoft, PW	This sounds fantastic. I personally would be most interested (besides the hours spent on a book being displayed in a column) in the calculation of the reading speed (WPM - words per minute). Reading without any thoughts on measurement and afterwards checking it, could give interesting insights and comparisons (progress when learning a language, light literature vs heavy, speed variying with author, genre, ...).

12-05-2024, 06:13 PM	#6
tomsem Grand Sorcerer Posts: 6,939 Karma: 27060151 Join Date: Apr 2009 Location: USA Device: iPhone 15PM, Kindle Scribe, iPad mini 6, PocketBook InkPad Color 3	Good to know. I suppose we need to check to see if any more changes are needed to krds.py (e.g. for active canvas location, underline type, etc.). I'll try to get to this.

12-06-2024, 01:07 PM	#7
Drybonz Connoisseur Posts: 93 Karma: 10 Join Date: Apr 2010 Device: iPad, Kindle Scribe 2024	I had actually just come to the forum to find out if there was a way to do this... so, yes!

12-09-2024, 10:17 PM	#10
Defenderland Groupie Posts: 181 Karma: 208 Join Date: Feb 2012 Device: Colorsoft, PW	Great news. Amazing.

12-21-2024, 11:14 AM	#11
rszasz Member Posts: 14 Karma: 10 Join Date: Jun 2010 Device: Kindle 4thGen	That would be great, its a feature that's been missing since the first PWs changed the metadata formats. (read position and finished status are useful for me in organizing my library)

Advert

Advert