The evasive apnx

[Iznogood]

Hi everyone. I just received my PW2 (that's supposed to be able to show the page numbers that corresponds to the printed edition), and wanted to test out this feature. I ran one of my favorite books through finereader, saved the page as one HTML file per book page and made a script to merge these files together adding

<a class="pagebreak" id="pageNNN"></a>

at the beginning of every page and repaired the broken paragraphs.

I now have a book with a lot of markup that shows where the printed pagebreaks were. I wrote a script that auto-generated a PageList to the ncx file and a separate page-map.xml. So now I have an (almost) valid epub that shows the page numbers.

Is there a way I can convert this epub to azw3 with page numbers? I have tried kindlegen to convert it to something amazon-compatible but it didn't generate any apnx-file. I tried converting it with calibre, but calibre generated its own page numbers and removed my own. Kovids documentation of the APNX specifications are not something I want to dive into if I can void it; it contains both asin numbers (something which this book doesn't have) and other stuff regarding amazon-specific knowledge about the amazon-formats that I know next to nothing about.

Is there a tool I can use to generate the apnx-file for me? Perhaps a tool that I can incorporate into my linux bash-scripts? Once I have a correct apnx file I can transfer it to the kindle myself. Can I use calibre to keep the apnx together with the azw3 file, or do I have to handle the apnx file manually? Any hints on how to fix this will be greatly appreciated

[kovidgoyal]

Creating an apnx file is pretty trivial, just look at apnx.py in calibre's source code. calibre will automatically create apnx files when sending books to kindle devices (e-ink kindle devices, not tablets using kindle apps). However the page numbers in those are automatically generated.

[Iznogood]

Thanks for pointing me in the direction of an already existing script. I shall see what I can do to modify it; it's been years since I last wrote code for python. Do you think that generating an apnx file can be done in bash or is python the easiest language to do this in?

[kovidgoyal]

I never program in bash, of the languages I do program in, (C/C++/Coffeescript/python), python is far and away the best.

[DaleDe]

Originally Posted by kovidgoyal

I never program in bash, of the languages I do program in, (C/C++/Coffeescript/python), python is far and away the best.

Certainly Bash is not the correct tool. It is a shell using shell scripting. It would not even begin without a lot of Unix tools like grep, awk, sed, etc. Python is certainly capable and easier that C and its variants.

Dale

[Iznogood]

I do a lot of my work in linux and am used to the linux terminal, but not so familiar with python and its way of invocation, that's why I wondered whether this was possible from within bash.

But OK, it looks like I will have to do this with python, and it will be a good exercise for me. I haven't used python for years, and never needed to invoke a python script and passing a file as a reference to it. I therefore put forward a few more questions before I dive into this.

1) I will re-use apnx.py from Calibre:
http://bazaar.launchpad.net/~kovid/c...kindle/apnx.py
How would I invoke this python script from the command line and passing the path of the mobi file for which I would like to generate an apnx file? The script provides these methods and classes:
class APNXBuilder(object)
def write_apnx(self, mobi_file_path, apnx_path, accurate=True, page_count=0)
def generate_apnx(self, pages, apnx_meta)
def get_pages_exact(self, mobi_file_path, page_count)
def get_pages_fast(self, mobi_file_path)
def get_pages_accurate(self, mobi_file_path)
but it has no body for importing and reading the file. I guess I shall have to create my own script for reading the path argument and pass that to the function

write_apnx

in one way or another. Am I on the right track?

2) Is apnx only for mobi files, or for azw3 files as well? I see calibre generates them for mobi files, but cannot find azw3 files in here?

3) Does calibre always create its own apnx file when sinding to device, even when there is an apnx present in the folder together with the mobi file?

[Doitsu]

I looked into the whole APNX generation a long time ago, and, AFAIK, apnx.py can only generate fake page numbers. The best it can do is take the total number of pages of the print edition and divide the file accordingly.

I.e. no matter what you do, it probably won't work for you, because the script apparently cannot identify the actual pageMap targets in the .mobi file and use them to write a proper .apnx file with the pre-defined page numbers.

Have you looked at John's APNX file format description?

[Iznogood]

Originally Posted by Doitsu

Have you looked at John's APNX file format description?

Yes, I have. And creating a file according to the specification shouldn't be much of a problem when I use the apnx generator from calibre as a template.

I don't want to use the pagination information from within the mobi file. My text contains anchors

<a class="pagebreak" id="pageNNN"></a>

for every pagebreak, and these anchors should survive the epub to mobi conversion. So I need to search for the position of these anchors, and add that instead of the fake positions generated by calibre, and that should suffice *knock on wood*

[Doitsu]

Originally Posted by Iznogood

I don't want to use the pagination information from within the mobi file. My text contains anchors

<a class="pagebreak" id="pageNNN"></a>

for every pagebreak, and these anchors should survive the epub to mobi conversion.

When I referred to "pagination information from within the mobi" I actually meant the page number anchors. If the KindleUnpack output is anything to go by, the anchors in the KF8 part survive the compilation while the page anchors in the Mobi7 part are lumped together with other anchors as <a id="fileposXXXX" /> anchors. (KindleUnpack apparently also cannot restore NCX pageLists.)

AFAIK, KindleUnpack doesn't distinguish between pageNNN anchors and other anchors in the mobi7 part, which is not really surprising since the tool predates the introduction of "real page numbers."

Originally Posted by Iznogood

So I need to search for the position of these anchors, and add that instead of the fake positions generated by calibre, and that should suffice *knock on wood*

That seems to be the sticking point. If finding page number targets was that easy, the extremely talented guys who reverse-engineered the Mobi7/KF8 file format for Calibre probably would've already added code that detects pageNNN anchors in Mobi7 files and used them for generating .apnx files with real page numbers in them.

IMHO, it might be helpful to cross-post your question to the KindleUnpack thread, since the KindleUnpack developers are intimately familiar with the file format and might be able to give you some pointers. Maybe one of them might even like the challenge to be the first one to develop code that'll allow to generate an .apnx file with real page numbers, if at all possible.