Amazon KF8 format for periodicals

[nickredding]

I've been banging my head against the Amazon KF8 format as it might apply to periodicals (magazines and newspapers) and I've come to the following conclusions.

1) Kindlegen v2 generates combined Mobi6/Mobi8 files but it looks like it is only intended for books.

Previous versions of Kindlegen would correctly generate magazines and newspapers with the requisite secondary index (used for the sections/articles list displayed you get when you tap the menu for the table of contents) as long as the NCX file supplied to Kindlegen had the three-level periodical/section/article structure.

Kindlegen v2 generates the secondary index as well, but the Kindlegen v2 file lacks the meta 501 = NWPR (or MAGZ) field and the mobi type is set to book. The Kindle Fire therefore opens it as a book and displays the KF8 content. If I patch in meta 501=NWPR and set the mobi type to newspaper, the Fire still opens it as a book. If I then patch in a meta 113 = some ASIN (Amazon identification number) the Fire trys to open it as a newspaper, but crashes trying to display the article list. Oddly, if I corrupt the secondary index (e.g., by removing the mastheadImage secondary index entry) the Fire is able to display the sections/articles list but crashes trying to display any of the articles.

2) If I separate the Mobi6 and Mobi8 parts of the KF8 file using mobi_unpack the Mobi6 piece works perfectly on the Fire (as a Mobi6-formatted periodical). The Mobi8 part exhibits the identical crashing behavior as the combined Mobi6/Mobi8 file.

3) Making various patches to the Mobi8 periodical to align its flags and meta data with the Mobi6 version doesn't help--result is the same.

4) I conclude that the Fire cannot properly handle KF8 files with secondary index records (i.e., periodicals).

5) I obtained an issue of Architectural Digest which is (so far) the only periodical I can find that says its for "Kindle Fire only" implying it uses the KF8 format. Surpise! The Mobi header says 6, but the issue certainly doesn't look like a Mobi6 document. It actually looks like page images of the print issue. The file is encrypted but that only applies to the HTML, so I was able to inspect the structure and this is what I found:

--The file doesn't contain any of the new record types that Kindlegen v2 introduces: No FDST, RESC, SKEL, OTHER records, etc. are present. However, there are some new meta data items (405, 406, 407) and I'm guessing the Fire is using those to recognize the document as a Fire-formatted magazine (unlike with Kindles up to V3, Amazon seems to be using the meta data to determine how latest generation Kindles handle content).

--The articles list includes a horizontal scrolling display of page thumbnails which you can use to get to a specific page.

--The images section includes complete page thumbnail (192 x 250 pixel) and full-resolution (1000 x 1299 pixel) page images. However, the images section also contains individual images.

I figured out that the Fire is displaying the full-resolution images when I look at the issue (it's not using HTML). If I patch the meta 501 field from MAGZ to NWPR I get the newspaper TOC but the articles display as Mobi6 using the individual images. So it's not really true that Architectural Digest is for "Kindle Fire only" since it contains a Mobi6 version of the periodical!

Needless to say the inconsistency in how Kindle Fire handles books/magazines/newspapers and the general opaqueness of the formatting details is going to make it very difficult to deal with. My interest has been in periodical formatting and I'm giving up on that for now. It looks like Amazon isn't using the HTML5/CSS3 capabilities of KF8 for its own periodicals and that is probably why the Fire is choking on Kindlegen v2-generated periodicals.

So there, for the benefit of anyone who is interested in pursuing this, is everything I have learned so far.

[KevinH]

Hi Nick,

Interesting. Will you run the DumpMobiHeader.py program on all variations of these periodicals and pm them to me. Also look at the exth_flags and see if there is any info there. The DumpMobiHeader.py program should work on even encrypted ebooks since it only looks at the non-encrypted parts.

It can be found here:

https://www.mobileread.com/forums/sho...63&postcount=8

Thanks.

Kevin