Kindle Page Numbers

[goaspy]

The new Kindle Firmware 3.1 is able to display page numbers.

Will it be possible to set page numbers from Calibre by converting to MOBI? Maybe similar to the way epub calculates pages. Or import page numbers from epub if this is possible.

[HarryT]

If you can explain to us what the technical specification of the new page number mechanism is, someone can implement it. Just asking a question such as "is it possible" without specifying what needs to be done isn't going to achieve very much .

[DoctorOhh]

Originally Posted by goaspy

Will it be possible to set page numbers from Calibre by converting to MOBI? Maybe similar to the way epub calculates pages. Or import page numbers from epub if this is possible.

First off it hasn't been released yet so this really isn't a question but a request to predict the future.

My prediction, Since their numbering equates to specific print version the odds that this will be implemented in calibre conversions to mobi is very remote. I don't see Amazon having any reason to supply this data via a API like their metadata. Of course anything is possible.

[goaspy]

I will investigate as much as I can, Amazon already sells books with page numbers.

[sam.doran]

From reading on Amazon's site, it looks like the page numbers are linked to an ISBN. Since calibre already has ISBN info for most books (or can acquire it automagically), I imagine it should be fairly simple to implement. I'm far from a programmer, but it looks as though the pieces are all there, it's just a matter of putting an ISBN into the .mobi file.

[goaspy]

The ISBN doesn't give you the pages, it is mentioned just so you know that page X from your kindle book is the same with page X from that book.

[DMSmillie]

Originally Posted by sam.doran

From reading on Amazon's site, it looks like the page numbers are linked to an ISBN. Since calibre already has ISBN info for most books (or can acquire it automagically), I imagine it should be fairly simple to implement. I'm far from a programmer, but it looks as though the pieces are all there, it's just a matter of putting an ISBN into the .mobi file.

How would the bibliographic data associated with an ISBN create links between pages in the print version and the points in the Kindle version where the page number should be incremented? The ISBN simply identifies the print version that the page numbering is based on.

One possibility is that Amazon might have introduced an implementation of some form of the NCX <pagelist> coding. But at the moment, that's just a guess.

[Manichean]

It could be just that they use the fact that the number of screen pages in an ebook and the number of print pages in a paper book scale mostly linearly. Thus, if you know the total number of print pages, and the total number and current number of screen pages, you can easily calculate the current print page. This wouldn't necessarily be 100% exact, though.

[goaspy]

This looks like a dead end (or not).

The page numbers are not in the book itself, they are in the 'apnx' file. The popular highligts file is the 'phl', and the 'ea' file I don't know what it's for.
If you delete these files from the kindle, the page numbers disappear, along with popular highlights.

The interesting part is that this apnx file works with a certain mobi file.

Update: Even better, the same file works with other mobi files as well, I can jump to page X, but there's nothing special in the mobile file, it's just a converted epub (created manually).
Because this book is smaller than the first one, if I jump to the last page, it displays 100%, Page 318 of 380 (fake numbers but similar).
There are about 18.7 locations per page, confirmed by the difference above.

[Piper_]

Originally Posted by goaspy

The interesting part is that this apnx file works with a certain mobi file.

Update: Even better, the same file works with other mobi files as well, I can jump to page X, but there's nothing special in the mobile file, it's just a converted epub (created manually).

That's great news. Maybe we could have an apnx file repository.

It would be nice if we could grab the .apnx from Amazon by forging the asin, but that so far has only allowed synchronizing.

Heck, I couldn't even get the .apnx of some of my older .azw books until I deregistered and registered back into that old account.

So it looks like they only send you page files for the books you have in your archive.

[user_none]

It sounds like the page number file is a simple mapping of stream location to page number.

Any chance you can send me the mobi and its apnx file so I can get a look at it?

If we can determine the structure of the file we can add support for writing it to the Kindle when using send to device. The only issue is the calculation of what a page is. My first idea would be to use the EPUB character count calculation to provide pseudo page numbers. I dont see how we could provide numbers that correspond to a print book like ons purchased due to lack of proper mapping info.

[dwig]

Originally Posted by dwanthny

...My prediction, Since their numbering equates to specific print version the odds that this will be implemented in calibre conversions to mobi is very remote. I don't see Amazon having any reason to supply this data via a API like their metadata. Of course anything is possible.

True. Without a database that correlates some print version's pages numbers with a particular text string, or some other text attribute, there would be no way for a converter to create the new page numbers, whatever underlying mechanism Amazon is planning to use.

The only dream feature I can think of that might work would be for a super intelligent PDF converter to not only strip headers and footer automagically, but to also read the page numbers in either the header or footer and generate Amazon's new page number tags in the appropriate locations (or build the location to page number data table if that is the mechanism).

[user_none]

goaspy has sent me a the book and apnx file that he was looking at.

Here's what I've figured out so far:

The file starts with

{"contentGuid":"8d3d16e0","asin":"B002RHGYOA","cdeType":"EBOK","fileRevisionId":"1296868639127"} ) = {"asin":"1906694184","pageMap":"(4,a,1)"}

• contentGuid is most likely just a guid used to differentiate the file from others.
• The first asin is the amazon asin for the Kindle version of the book.
• cdeType is a Mobi document type specified as ebook. I haven't played with it much but I'm going to assume this is important. The 3.1 update looks to have a new navigation system for periodicals. Most likely the apnx will have a different meaning (but same format) depending on the cdeType.
• fileRevisionId should be obvious.
• The second asin is the isbn for the paper back version of the book.
• Don't know what pageMap does yet.

Following this header is a list of 4 byte sequences of big endian ints. They are in an increasing order. The total number of the 4 byte sequences is 573. The first 3 ints are all 0 which leads me to believe they are padding. The total number of pages within the book are 570 (as shown on the Kindle itself). I need to do some testing but it looks like my assumption is correct and the apnx file is a list o file locations where each is the beginning of a new page.

[user_none]

I've reverse engineered the APNX file format. Full specs are in format_docs/pdb/apnx.txt of the calibre source. I've added support to the Kindle interface to write it when sending MOBI files to the Kindle. Pages are mapped the same way EPUB page numbers are using 1024 character blocks. If all goes well Kovid will merge the change and it will be in the next (0.7.45) release.

Thank you to goaspy for helping with with finding test files for me to look at. Your input was helpful too and make the whole process quicker than it would have been.

[Piper_]

Originally Posted by user_none

I've reverse engineered the APNX file format. Full specs are in format_docs/pdb/apnx.txt of the calibre source. I've added support to the Kindle interface to write it when sending MOBI files to the Kindle. Pages are mapped the same way EPUB page numbers are using 1024 character blocks. If all goes well Kovid will merge the change and it will be in the next (0.7.45) release.

Thank you to goaspy for helping with with finding test files for me to look at. Your input was helpful too and make the whole process quicker than it would have been.

Wow, that is just awesome! And so fast. I was afraid to hope. lol

Karma on its way. And a little something else for