K4iOS, AZK, The Latest

[mattmc]

Hooray, solved!

Sorry, it took so long to get back and post here; I actually solved this fairly quickly, but I had several other things I had to handle.

The root of the problem was in my OPF file. The <dc:rights> tag contained a copyright that was several paragraphs (a tad unnecessary, but there you are). InDesign had inserted "Line Separator" and "Paragraph Separator" characters at the end of each line, which are invisible in my chosen text editor, Sublime. (They don't even act like newlines, so the text ran together, which could have been an indicator that there was something wrong.)

These two Unicode characters, U+2028 and U+2029, are somewhat infamous in the Javascript/JSON realm. Googling them reveals this post as one of the top hits, which says:

Basically if my JSON strings contains a U+2028 character (Unicode line separator) or U+2029 character (Unicode paragraph separator) then this is perfectly valid JSON. However when using JSONP the JSON gets executed as JavaScript and no string in JavaScript can contain a literal U+2028 or a U+2029 as it will break the JavaScript.
...
Essentially if these characters were inside strings in my JSONP data being sent to the client this would throw a line or paragraph break into the string which would break the JavaScript and stop it executing.

So when azkcreator converts the KF8 file into the AZK (which contains blobs of JSON) you are fine, but when K4iOS uses Javascript to convert the JSON blobs into a KCR, things break down. Specifically, in my case, the OPF file did not process/transform correctly, and the book would not open.

I added a Regex Replace function to my processing scripts that looks for those two characters and removes them from any files. After this change, I was able to create the AZK, load it into K4iOS, and the book opened fine.

Otherwise:

Originally Posted by Hitch

You all know I'm not coder, just a poor bookmaker. But in terms of troubleshooting, using fonts, for the purposes of testing an AZK, is worthless, as K4iOS is a KF7 reader.

Hm, are you sure that K4iOS is a KF7 reader? I mean, I'm sure that you'd know better than I, but when I side-load my book via AZK, the fonts show up just fine. Can you go into detail?

Originally Posted by Hitch

Speaking of which, for you, however: have you tried loading the same source file at the KDP, and seeing if the post-build MOBI (Step 7, I understand it is now) works on your iPad? Just as a comparative test?

I actually did load the book up through Amazon, purchased it, and downloaded it to my device--and it worked fine. I suppose that in the cloud, Amazon is not using Javascript to do their conversions. But at least now I can sideload to do quality checks, though.

(Did I understand your question correctly? I'm not precisely sure what the "post-build MOBI" is, or what Step 7 is. I didn't personally load the file up onto Amazon, so if you're referring to the KDP loading process, that would be why I'm drawing a blank.)

Anyway, my final question would be on your statement that K4iOS is a KF7 reader. If it's a KF7 reader, why does my sideloaded book (via AZK, which Amazon seems to think is the right way) show my embedded fonts? Do I need to investigate more on any differences between sideloading AZK and going via Amazon's cloud?

[Hitch]

To further the discussion, can you provide some screenshots of your eBook, with embedded fonts, from the K4iOS reader? From the sideloaded AZK?

Unless something has changed very, very recently--which it certainly could have, without us noticing--the K4iOS reader has been a KF7 reader since nearly the beginning. It even has some KF7-KF8 crossover glitches, like displaying things that have KF7 fallbacks, as well as the KF7 coding (for example, text boxes). I freely admit I don't test every book on iOS, for myriad reasons (mostly, the ridiculously time-consuming AZK, sideload, yadda, nonsense. I checked a book with fonts today (again, post-upload at the KDP), and nary a font was to be seen.

And are you still seeing these fonts, after you load the book at the KDP, and then download the post-conversion MOBI, from Step 7? (Download Preview File). I presume you keep a test account, for these purposes, so you can test books post-upload at Step 6? (There's a lot that happens at Step 6, which can alter the outcome and appearance of the book. We test our books at the KDP, and the file that you download at Step 7 is indicative of what you'll get, rather than what you see from Desktop Previewer).

If you haven't, I would recommend loading the mobi at KDP, and downloading the Step-7 mobi, and then viewing that on the iPad, to see what you're getting. Just a suggestion.


Hitch

[DaleDe]

Originally Posted by Hitch

To further the discussion, can you provide some screenshots of your eBook, with embedded fonts, from the K4iOS reader? From the sideloaded AZK?

Unless something has changed very, very recently--which it certainly could have, without us noticing--the K4iOS reader has been a KF7 reader since nearly the beginning. It even has some KF7-KF8 crossover glitches, like displaying things that have KF7 fallbacks, as well as the KF7 coding (for example, text boxes). I freely admit I don't test every book on iOS, for myriad reasons (mostly, the ridiculously time-consuming AZK, sideload, yadda, nonsense. I checked a book with fonts today (again, post-upload at the KDP), and nary a font was to be seen.

And are you still seeing these fonts, after you load the book at the KDP, and then download the post-conversion MOBI, from Step 7? (Download Preview File). I presume you keep a test account, for these purposes, so you can test books post-upload at Step 6? (There's a lot that happens at Step 6, which can alter the outcome and appearance of the book. We test our books at the KDP, and the file that you download at Step 7 is indicative of what you'll get, rather than what you see from Desktop Previewer).

If you haven't, I would recommend loading the mobi at KDP, and downloading the Step-7 mobi, and then viewing that on the iPad, to see what you're getting. Just a suggestion.


Hitch

At one time the Apple viewer was KF7 but when AZK was released it became like the Amazon tablets, a dual reader except instead of KF7 fallback from KF8 the Apple Kindle reader is AZK with a KF7 fallback. It does not read KF8 at all. It is not clear why Amazon developed two formats but perhaps they are testing an intend to adopt only one of them in the future.

Dale

[Hitch]

Originally Posted by DaleDe

At one time the Apple viewer was KF7 but when AZK was released it became like the Amazon tablets, a dual reader except instead of KF7 fallback from KF8 the Apple Kindle reader is AZK with a KF7 fallback. It does not read KF8 at all. It is not clear why Amazon developed two formats but perhaps they are testing an intend to adopt only one of them in the future.

Dale

DaleDe:

Yes--but in short, no fonts, as far as I know. AZK can't be uploaded at the KDP for sale; thus, it's a moot point. AZK's aren't generated for use, when the source (ePUB, MOBI, zipped HTML, whatever) at the KDP. As it only has KF7, for all intents and purposes, for a "for-sale" book, there are no fonts, as KF7 can't display them.

Not once the book is loaded for sale. And honestly, I don't recall seeing any fonts in any AZK that I've sideloaded, either--at least, not embedded font. (And for that matter, not even the fairly primitive, "sans serif here and serif there," from the firmware, either, IIRC.) OTOH, I freely admit that, given the limitations of the reader, I don't bother to test on it, unless we have an element (like the aforementioned text-boxes) that may present problems on the K4iOS reader.

Hitch

[KevinH]

Hi,
FYI, KF8's internally are almost a one for one mapping to the AZK format, with the same skeleton, and fragment sections. The only real differences are that azks are just encoded by a special dictionary based word compression scheme and encoded into json.

To test this I converted the same epub into kf8 andazk using kindlegen and studied the differences which were quite minimal. Both are set up to allow "fragments" of up to 8000 chars or so to be loaded upon demand inside a page "skeleton". KindleUnpack simply builds up the original pages from the fragments and skeleton info provided in the KF8 (and then does a lot of link fixups, css link fixups, and etc) to create an epub-like structure.

This is perfect format for limited memory phones and tablets with their own webkits and web-based technologies.

I could never fully figure out the encoding/encryption used for the dictionary based word compression scheme of the azk otherwise I would have modified KindleUnpack to unpack them as well since much of the code would be re-usable from the kf8 code.

Given how much morewe know based on this thread, I may decide to revisit it.

Hope this helps,

KevinH

Originally Posted by DaleDe

At one time the Apple viewer was KF7 but when AZK was released it became like the Amazon tablets, a dual reader except instead of KF7 fallback from KF8 the Apple Kindle reader is AZK with a KF7 fallback. It does not read KF8 at all. It is not clear why Amazon developed two formats but perhaps they are testing an intend to adopt only one of them in the future.

Dale

[mattmc]

Originally Posted by KevinH

Hi,
FYI, KF8's internally are almost a one for one mapping to the AZK format, with the same skeleton, and fragment sections. The only real differences are that azks are just encoded by a special dictionary based word compression scheme and encoded into json.

To test this I converted the same epub into kf8 andazk using kindlegen and studied the differences which were quite minimal. Both are set up to allow "fragments" of up to 8000 chars or so to be loaded upon demand inside a page "skeleton". KindleUnpack simply builds up the original pages from the fragments and skeleton info provided in the KF8 (and then does a lot of link fixups, css link fixups, and etc) to create an epub-like structure.

This is perfect format for limited memory phones and tablets with their own webkits and web-based technologies.

I could never fully figure out the encoding/encryption used for the dictionary based word compression scheme of the azk otherwise I would have modified KindleUnpack to unpack them as well since much of the code would be re-usable from the kf8 code.

Given how much morewe know based on this thread, I may decide to revisit it.

Hope this helps,

KevinH

Very interesting, Kevin. So KF8 ~= AZK?

However, given that AZK as a file format is not actually displayed anywhere, in any reader that I'm aware of, but is simply a bridge between KF8 and KCR for K4iOS, I would be even more interested to hear any comparisons you can do between KCR and KF8. Or a detailed analysis of the KCR file format (as used in K4iOS)? (Maybe there is one somewhere, I wouldn't mind being linked there.)



EDIT:

Originally Posted by Hitch

To further the discussion, can you provide some screenshots of your eBook, with embedded fonts, from the K4iOS reader? From the sideloaded AZK?

Sure! Attached.

And yet, in the other thread...

Originally Posted by Hitch

My request, just to clarify, is for screenshots displaying embedded fonts carrying through on a PUBLISHED book, on K4iOS. Not a side-loaded book or a previewed book, an Indy-pubbed (so, uploaded via the KDP) published book.

Updated to add: the lastest KPG's (Kindle Publishing Guidelines) did confirm info on using media queries, generally, for the iOS, which is GREAT.

Updated to add, 2: we're looking at some recently-pubbed books from some clients of ours, that have embedded fonts. Will update if needed.

ETA, 3: we don't know when this started, but we just downloaded a book, to a different iPad, that we'd downloaded when this thread started, and now, it has the embedded sans font in the pullquotes. BUT, this is new. I know for a fact that this wasn't working just a few months ago.

Hitch

Aha. So, no need for me to go via KDP. Although I probably should start doing that more...