Kindle Page Numbers

[DoctorOhh]

Originally Posted by user_none

Nope and nope. EPUB is not my area of expertise. I wasn't able to find a spec for computing page numbers so I went from memory about how I thought it worked.

I'm using 1024 blocks of uncompressed text. Since it's an arbitrary calculation it can always be tweaked. I'll look over the ebook docs on Adobe's website and see about getting it working using the same calculation.

I just posted on Adobe's page numbering here with links to appropriate pages.

[chyron8472]

I wish this was available for K2. =\

[Bratzzo]

Hi,

Originally Posted by user_none

I've reverse engineered the APNX file format. Full specs are in format_docs/pdb/apnx.txt of the calibre source. I've added support to the Kindle interface to write it when sending MOBI files to the Kindle. Pages are mapped the same way EPUB page numbers are using 1024 character blocks. If all goes well Kovid will merge the change and it will be in the next (0.7.45) release.

Thanks to you and Kovid for the fast implementation.

But I have one question:
Is it possible not to send the apnx file automaticly to the Kindle?
Because I figured out, that all pagenumbers from my books wich I have tested are way to high!

For example:
print version: 416 pages
ADE: 282 pages
Kindle: 813 pages

Or an other book:
print version: 864 pages
ADE: 646 pages
Kindle: 2193 pages

In this case, I prefer the locatin numbers only.
Therefore I would like that the apnx file will not be transferred automaticly.
OK, I can delete the file by myself. But I would prefer, if there is an option to choose between to send or not to send.



Bratzzo

[DoctorOhh]

Originally Posted by Bratzzo

Is it possible not to send the apnx file automaticly to the Kindle? Because I figured out, that all pagenumbers from my books wich I have tested are way to high!

It's possible, but why? It makes much more sense to just adjust the calculation in calibre to give a more realistic number.

Originally Posted by Bratzzo

For example:
print version: 416 pages
ADE: 282 pages
Kindle: 813 pages

Or an other book:
print version: 864 pages
ADE: 646 pages
Kindle: 2193 pages

Good info this will give the developers something to go on.

[goaspy]

Originally Posted by Bratzzo

Because I figured out, that all pagenumbers from my books wich I have tested are way to high!

For example:
print version: 416 pages
ADE: 282 pages
Kindle: 813 pages

Or an other book:
print version: 864 pages
ADE: 646 pages
Kindle: 2193 pages

You can use ApnxGen in meantime, and override the apnx file generated by Calibre with your custom file.

[user_none]

The location number is still shown in addition to the page number from the apnx file. Having the apnx doesn't prevent you from seeing the location number.

[lyric]

Hi all,

I have also noticed that the current page counting algorithm overestimates the number of pages. In the example I have checked a 464-page book is coming out as 980, so it seems that we are around a factor of 2 out.

Cheers,

Dan

[user_none]

I've tweaked the character count for estimating a page. It about doubles the number of characters per page. I've changed it from 1024 to 2300. This takes into account the average number of characters in a page in my test book and gives some overhead for markup.

Please remember this is never going to be 1 to 1 in a paper back book. In some cases it will be more and some cases it will be less. We are estimating page length based on the number of characters because we cannot take the time to decompress and parse the text.

The disadvantage of using a character count is pages with short (dialog) will create longer pages then long paragraphs. Making the page count shorter than the paper back.

A better but much more resource intensive and slower method to calculate the page length would be to parse the uncompressed text. For each paragraph we would want to find how many lines it would occupy in a paper back book. 70 characters per line and 32 lines per page. So divide the number of characters (minus markup) in each paragraph by 70. If there are less than 70 characters in the paragraph then it is 1 line. Then, count every 32 lines and mark that location as a page.

However, I'm not going to use the better method because it will take a lot of resources and too much time processing for such a feature. Sending to device should take seconds not minutes. If someone (goaspy) wants to make an external tool that will generate pages based on the above method, that would be your best bet for getting almost 1 to 1 page numbers.

[Perkin]

I had an idea, don't know if it's feasible or not...

What if the source has <a id="calibrekindlepage###" />, that the creator can insert for the 'pages', could calibre identify them an use them rather than calculating the positions.

Would allow for exact 1:1 match to paper book.
Might take too long to loop through getting positions.

Just an idea.

[user_none]

Originally Posted by Perkin

What if the source has <a id="calibrekindlepage###" />, that the creator can insert for the 'pages', could calibre identify them an use them rather than calculating the positions.

The biggest issue with this is still decompression and parsing which I would like to avoid. I would also like to avoid adding calibre specific markup. It would be better to have a user generated mapping using a standalone tool for cases where the user wants to manually create a 1 to 1 mapping.

I'm going to run some tests with decompression and parsing. If the added time it takes to do this isn't obscene I'll look into making it an option. Fast or accurate pseudo page numbers for instance.

[GRiker]

Originally Posted by user_none

I'm going to run some tests with decompression and parsing. If the added time it takes to do this isn't obscene I'll look into making it an option. Fast or accurate pseudo page numbers for instance.

If the decompression test works out favorably, you might look at this enhancement request at the same time.

G

[kovidgoyal]

@user_none: You can have the MOBI output plugin do the calculation for you and embed the result in some unused MOBI EXTH header for the driver to use. For non calibre generated MOBI files, it can fall back to a default, fast algorithm.

[user_none]

Here are my results:

Upload time

Accurate

14 books - 29 sec
5 books - 8 sec
3 books - 5 sec


Fast

14 books - 13 sec
5 books - 6 sec
3 books - 3 sec

Looking at set of 14 books we have approximately double the time to transfer. This is due to decompression and parsing the text. My parser only looks at the amount of visible text. Each page is comprised of 32 lines and each line can have up to 70 characters. A paragraph always starts a new line. The accuracy can be increased by adding support for handling <div class="mbp_pagebreak" /> and <br> tags.

One of my test books at page 105 mapped 1 to 1 to the print version. I did not do extensive testing of the other pages. Other books mapped very closely to the print edition.

Over all the more accurate APNX generator gives much closer results. With handling for the two additional elements OCRed texts and ones that use the same type setting as their print counter parts should give nearly 1 to 1 page mappings.

Now for the big question. Is doubling the time it takes to transfer the book to the device worth a more accurate mapping? The mapping will be thrown off if the print book physical dimensions are different than the average paper back size I'm using. So a hard cover, larger or smaller paper back will cause the mapping to be off.

Also, if it is worth the extra time for using the accurate parser would it be worth increasing the time even more by changing the parser to accommodate the two previously mentioned elements and make it even more accurate?

[kovidgoyal]

I suggest making it an option in the kindle driver and/or putting it into the MOBI output code.

[user_none]

Originally Posted by kovidgoyal

I suggest making it an option in the kindle driver and/or putting it into the MOBI output code.

I'll go with the option that way it works with user's existing MOBI format books. If the user selects accurate I'll have it fall back to fast if there is DRM on the book.