Register Guidelines E-Books Today's Posts Search

Go Back   MobileRead Forums > E-Book Formats > Kindle Formats

Notices

Reply
 
Thread Tools Search this Thread
Old 05-04-2015, 03:13 PM   #1
mattmc
Connoisseur
mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.
 
Posts: 89
Karma: 185923
Join Date: May 2015
Device: iPad 1/2/Air, K3/PW2/Fire1, Kobo Touch, Samsung Tab, Nook Color/Touch
Question K4iOS, AZK, The Latest

Hey all!

TLDR; Does K4iOS still use .azk?

Introduction
I'm doing a fairly meticulous eBook creation process, which involves careful proofing of the book across the full range of popular devices. Obviously, with Amazon's share of the eBook market, this includes all things Kindle.

Making the AZK
Based on my internet research, I came to the conclusion that Amazon delivers the special ".azk" file type to your Kindle for iOS app. I managed to download Kindle Previewer on my OSX Yosemite machine, and work black magic to get it to run without crashing, at which point I was able to generate the .azk file. Hooray!

Loading the AZK
Now, I want to sideload my generated .azk onto my iPad. So I follow the simple procedure, going through iTunes -> iPad -> Apps -> Kindle -> Documents, drag it in there. After a short pause, the .azk does appear in my K4iOS library, with the correct cover image.

However, when I click on it to open it, I get an error: "The book could not be opened. Please remove the book from your device and redownload it."

Removing and re-sideloading the book doesn't help. Neither did un-registering and re-registering. I've tried this on an iPad 1, running Kindle for iPad v3.9.2, as well as an iPad 2, running Kindle for iPad v4.6.

Other AZK Files
I wondered if maybe there was something wrong with my .azk, or if I had some content in my .mobi file that became messed up in the conversion, or was incompatible with the .azk format. So I set out to get an .azk file by some other means.

First, I tried to use the Send to Kindle Mac App to simply send my .mobi file to my iPad, hoping that Amazon's delivery system would convert it to .azk as part of delivering it. It did show up on my iPad, and I was able to open the file. However, when I used iFunbox to examine the book on my iPad, it was an awz file (no encryption). So I failed to make an .azk file that way. (Note that I was only able to check this on my iPad 1 running iOS 5; my iPad 2 has iOS 8.1 and the file structure is different, I believe I'd need to jailbreak to examine the files.)

I became curious--if Send to Kindle delivered an awz file (which I believe is just .mobi with different compression & encrpytion, no?) to my K4iOS app when I send a .mobi, under what circumstances does it deliver an .azk file?

I realized I had several eBooks, purchased from Amazon, on my Kindle account, so I downloaded them to my iPad from within the K4iOS app. When I inspected them with iFunbox (iOS 5), I found that they were either azw files, or a folder that contained a book.kcr file and another folder called "resources", with a toc.ncx and a bunch of files which no extension. (Not totally sure what this other type of file is, but "kcr" seems like "kindle cloud reader".)

So basically, none of the books that I synced down from Amazon onto my K4iOS app seemed to be of the .azk extension.

The Questions
  • ...Does Amazon even use .azk anymore?
  • Does anyone have any proof of this?
  • Does anyone have an .azk file that they have side-loaded onto their K4iOS App, that I can get and look at/try out on my own iPad?
  • Anyone have a clue of what I could be doing wrong?
  • Should I stop banging my head against the wall, trying to get a working .azk file? (I do care about proofing the book exactly how it's going to look on the device.)

Thanks!

====================
Some relevant posts & articles:
http://pressbooks.com/blog/testing-p...kf8-azk-gahhh/
http://mandatedmemoranda.com/tag/azk-file/
https://wiki.mobileread.com/wiki/AZK
https://kdp.amazon.com/community/mes...ssageID=640003
http://www.newselfpublishing.com/ProofingKindle.html
mattmc is offline   Reply With Quote
Old 05-04-2015, 05:50 PM   #2
Doitsu
Grand Sorcerer
Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.
 
Doitsu's Avatar
 
Posts: 5,583
Karma: 22735033
Join Date: Dec 2010
Device: Kindle PW2
Quote:
Originally Posted by mattmc View Post
TLDR; Does K4iOS still use .azk?
It definitely still supports it.

There are several possibilities why your .azk version didn't work:
  1. The source files were invalid. You can usually test this by clicking the Compilation Details link in the Compiling Book dialog box and checking the build log.
  2. Your iTunes/Kindle Previewer/KindleGen version is outdated.
  3. The .azk file got corrupted
I've attached a sample .azk file that I successfully sideloaded and opened with Kindle for iOS. Note that I had to rename it to pseudo_firstletter.azk.mobi, because MR doesn't allow .azk attachments. You'll have to rename it back to pseudo_firstletter.azk before sideloading it.

(The source file is attached to this post.)
Attached Files
File Type: mobi pseudo_firstletter.azk.mobi (24.8 KB, 227 views)
Doitsu is offline   Reply With Quote
Old 05-04-2015, 08:04 PM   #3
mattmc
Connoisseur
mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.
 
Posts: 89
Karma: 185923
Join Date: May 2015
Device: iPad 1/2/Air, K3/PW2/Fire1, Kobo Touch, Samsung Tab, Nook Color/Touch
Quote:
Originally Posted by Doitsu View Post
It definitely still supports it.

There are several possibilities why your .azk version didn't work:
  1. The source files were invalid. You can usually test this by clicking the Compilation Details link in the Compiling Book dialog box and checking the build log.
  2. Your iTunes/Kindle Previewer/KindleGen version is outdated.
  3. The .azk file got corrupted
I've attached a sample .azk file that I successfully sideloaded and opened with Kindle for iOS.
Off that bat, thanks very much for your help Doitsu; I appreciate it.

Here's what I did:
  1. I started by downloading your .azk, removing the .mobi extension, and sideloading it into my iPad...It worked fine and I was able to view the book in K4iOS as normal!
  2. I checked the versions of my stuff. iTunes is v12.1.2.27 (almost latest), Kindle Previewer is v2.94 (freshly downloaded), the KindleGen I was using was v2.9, which matches what is bundled inside the latest Kindle Previewer. So I don't think that anything is outdated.
  3. I tried different routes of creating the .azk, mainly running it through KindleGen myself and then through azkcreator inside Kindle Previewer, and also taking it from .epub to .azk all with Previewer. Both KindleGen logs reported no warnings, finished with "Mobi file built successfully".

The end of my KindleGen log, in case it is of interest:
Quote:
Info(prcgen):I2001: Final stats - TEXT: 001465109 bytes 000000856 records
Info(prcgen):I2003: Final stats - INDEXES: 000018920 bytes 000000008 records
Info(prcgen):I2004: Final stats - TRAILING: 000005006 bytes 000000856 records
Info(prcgen):I2005: Final stats - FRAMES: 000000124 bytes 000000003 records
Info(prcgen):I2006: Final stats - OTHER: 000009925 bytes 000000003 records
Info(prcgen):I2007: Final stats - TOTAL: 001499084 bytes 000000870 records
Info(prcgen):I1039: Final stats - text compressed to (in % of original size): 41.79%
Info(prcgen):I1041: The file format version is V8
Info(prcgen):I15000: Approximate Standard Mobi Deliverable file size : 0003290KB
Info(prcgen):I15001: Approximate KF8 Deliverable file size : 0003641KB
Info(prcgen):I1036: Mobi file built successfully
So it seems that I have up-to-date software for everything, and there were no sorts of warnings or errors during the process. My guess from there would be that there's something off with my file?

My .epub file is created with Adobe Indesign CC, latest version. It has custom fonts, images, some special styles, hyperlink footnotes, etc. I don't know that there's anything in it that is abnormal or would compile to .mobi strangely (and KindleGen doesn't find anything wrong with it either).

Besides mucking around with my .epub (cutting out parts of it, perhaps, and seeing what part of it might be killing my .azk), any other ideas? Any way to analyze the integrity of my .azk, or retrieve error logs from the K4iOS app? (Hah.)

Additionally, and importantly, does Amazon actually still deliver .azk files to K4iOS apps? The app seem to read .mobi files just fine, and the Kindle books that I downloaded from Amazon onto my iPad were either .awz or this weird format with the .kcr manifest. I can be certain now that K4iOS supports .azk, but is there any evidence that Amazon is still using this format?

Again, thanks very much for your help with this esoteric corner of the eBook world!

Last edited by mattmc; 05-04-2015 at 08:07 PM.
mattmc is offline   Reply With Quote
Old 05-05-2015, 02:29 AM   #4
Doitsu
Grand Sorcerer
Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.
 
Doitsu's Avatar
 
Posts: 5,583
Karma: 22735033
Join Date: Dec 2010
Device: Kindle PW2
Since you've excluded all other possibilities, it seems that the OSX version of azkcreator has a bug. (I used the Windows version.) Maybe you could ask another Mac user to install Kindle Previewer and convert your book with it.
(I could also create an .azk file for you if you pm me the file.)
Since there weren't any warnings or errors at the end of the build log and .azk files support most .azw3 features, your .azk file will probably render exactly as the corresponding .azw3 file.

Quote:
Originally Posted by mattmc View Post
Additionally, and importantly, does Amazon actually still deliver .azk files to K4iOS apps?
AFAIK, Amazon never delivered .azk files to iOS Kindle apps. They're only for testing purposes. Sideloading .azk files apparently forces the iOS Kindle app to generate the same file that readers will get, if they download a book for which a KindleGen generated .mobi file exists (which contains the old mobi7 and new mobi8/KF8/AZW3 formats) and their app supports the newer AZK/AZW3 format.
Doitsu is offline   Reply With Quote
Old 05-05-2015, 08:33 PM   #5
mattmc
Connoisseur
mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.
 
Posts: 89
Karma: 185923
Join Date: May 2015
Device: iPad 1/2/Air, K3/PW2/Fire1, Kobo Touch, Samsung Tab, Nook Color/Touch
Quote:
Originally Posted by Doitsu View Post
Since you've excluded all other possibilities, it seems that the OSX version of azkcreator has a bug. (I used the Windows version.) Maybe you could ask another Mac user to install Kindle Previewer and convert your book with it.
(I could also create an .azk file for you if you pm me the file.)
Huh, interesting point. I'll just download Previewer onto a Windows I have and give it a whirl--but thanks for the offer to generate it for me

Quote:
Originally Posted by Doitsu View Post
Since there weren't any warnings or errors at the end of the build log and .azk files support most .azw3 features, your .azk file will probably render exactly as the corresponding .azw3 file.
Hm, hopefully, but can't take chances...

Quote:
Originally Posted by Doitsu View Post
AFAIK, Amazon never delivered .azk files to iOS Kindle apps. They're only for testing purposes. Sideloading .azk files apparently forces the iOS Kindle app to generate the same file that readers will get, if they download a book for which a KindleGen generated .mobi file exists (which contains the old mobi7 and new mobi8/KF8/AZW3 formats) and their app supports the newer AZK/AZW3 format.
Wow! Interesting. Yeah, I inspected your .azk file that I sideloaded successfully, and once it's pulled into Kindle it's become one of those folders with the .kcr manifest or whatever. Do we know what those are? They're not an azw, right? Some other kind of file format?

Otherwise, I'll report back when I've tried the Windows version of azkcreator.
mattmc is offline   Reply With Quote
Old 05-06-2015, 05:30 AM   #6
Notjohn
mostly an observer
Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.Notjohn ought to be getting tired of karma fortunes by now.
 
Posts: 1,515
Karma: 987654
Join Date: Dec 2012
Device: Kindle
I think you're expected to use iTunes to get the file onto the device, not sideload it yourself.

(I have a Kindle app on my iPhone and have never had any problem with how my books display. I don't use an iPad, except occasionally to check a book on a friend's tablet, and I've never had occasion to load a book onto it other than by downloading from the Amazon store. So I am not au courant on this.)
Notjohn is offline   Reply With Quote
Old 05-06-2015, 01:25 PM   #7
DaleDe
Grand Sorcerer
DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.
 
DaleDe's Avatar
 
Posts: 11,470
Karma: 13095790
Join Date: Aug 2007
Location: Grass Valley, CA
Device: EB 1150, EZ Reader, Literati, iPad 2 & Air 2, iPhone 7
Quote:
Originally Posted by mattmc View Post
Huh, interesting point. I'll just download Previewer onto a Windows I have and give it a whirl--but thanks for the offer to generate it for me


Hm, hopefully, but can't take chances...


Wow! Interesting. Yeah, I inspected your .azk file that I sideloaded successfully, and once it's pulled into Kindle it's become one of those folders with the .kcr manifest or whatever. Do we know what those are? They're not an azw, right? Some other kind of file format?

Otherwise, I'll report back when I've tried the Windows version of azkcreator.
Check AZK in our wiki. An AZK is a zipped archive. It is likely that they unzipped it.

Dale
DaleDe is offline   Reply With Quote
Old 05-07-2015, 06:36 PM   #8
mattmc
Connoisseur
mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.
 
Posts: 89
Karma: 185923
Join Date: May 2015
Device: iPad 1/2/Air, K3/PW2/Fire1, Kobo Touch, Samsung Tab, Nook Color/Touch
Quote:
Originally Posted by Notjohn View Post
I think you're expected to use iTunes to get the file onto the device, not sideload it yourself.
Ah, sorry that I wasn't clear: that's what I meant by "sideloading". With an iOS device, I suppose "true" sideloading would involve direct file system manipulation, and possibly jailbreaking.

Quote:
Originally Posted by DaleDe View Post
Check AZK in our wiki. An AZK is a zipped archive. It is likely that they unzipped it.
I did see that wiki entry--it's actually the most thorough analysis of the .azk format anywhere on the internet, and was very helpful!

However, what I see on the iPad is not simply an unzipping of the .azk archive. Here's what the contents of my .azk look like if I open it up:
(Note that I trimmed the quantity of files or the post would be 3 pages long.)
Code:
└── book
    ├── frags
    │** ├── gz_frag0.jsonp
    │** ├── gz_frag1.jsonp
    │** ├── gz_frag10.jsonp
    │** ├── gz_frag100.jsonp
    │** ├── gz_frag101.jsonp
    │** ├── gz_frag102.jsonp
    │** ├── gz_frag103.jsonp
    │** ├── gz_frag104.jsonp
    │** ├── gz_frag105.jsonp
    │** ├── gz_frag106.jsonp
    │** ├── gz_fragmap.jsonp
    │** ├── gz_skeleton0.jsonp
    │** ├── gz_skeleton1.jsonp
    │** ├── gz_skeleton10.jsonp
    │** ├── gz_skeleton11.jsonp
    │** ├── gz_skeleton12.jsonp
    │** └── gz_skeleton13.jsonp
    ├── kcrManifest.jsonp
    ├── metadata.jsonp
    ├── mimetype.json
    ├── mobilocations
    │** ├── gz_mobilocations0.jsonp
    │** ├── gz_mobilocations1.jsonp
    │** ├── gz_mobilocations10.jsonp
    │** └── gz_mobilocations11.jsonp
    ├── output.json
    ├── rawManifest.jsonp
    └── resources
        ├── resource0
        ├── resource1
        ├── resource10
        ├── resource100
        ├── resource101
        └── resource102
So I take the .azk and load it onto my iPad, via iTunes. Because it's iOS 5, I can use iFunbox to look at the file structure and see what my book looks like now that K4iOS has processed it. This is what I see:

Code:
└── my_book_title
    ├── book.kcr
    ├── gz_fragment555.jsonp
    ├── gz_fragment556.jsonp
    ├── gz_fragment557.jsonp
    ├── gz_fragment558.jsonp
    ├── gz_fragment559.jsonp
    ├── gz_fragment560.jsonp
    ├── gz_fragment561.jsonp
    ├── gz_fragment562.jsonp
    ├── gz_fragment563.jsonp
    ├── gz_fragment564.jsonp
    ├── gz_fragment565.jsonp
    ├── gz_fragment566.jsonp
    ├── gz_fragment567.jsonp
    └── resources
        ├── 1004544089
        ├── 621764591
        ├── 624248723
        ├── 689149105
        ├── 716323204
        ├── 760635517
        ├── 768981550
        ├── 769084276
        ├── 874920377
        ├── 884243033
        ├── 885818391
        ├── 966245230
        ├── 975879832
        └── 98527426
(Again, I removed some of the files because there were a lot.)

So, it's more than a simple unzip, there's some sort of processing that's done on these files.

On a related note, this is the directory structure of one of the books that I synced down straight from the Amazon cloud:

Code:
└── B00AFR4CK2_EBOK
    ├── book.kcr
    └── resources
        ├── 1198181274
        ├── 1213596095
        ├── 1381148118
        ├── 1508313587
        ├── 15464168
        ├── 1716487958
        ├── 1866276677
        ├── 1983956603
        ├── 2059389305
        ├── 242373662
        ├── 308183104
        ├── 517793479
        ├── 582929403
        ├── 657402086
        ├── 808580617
        └── toc.ncx
Similar, but not quite the same as what came out of K4iOS's processing of the .azk file. Does anyone know what that file type is? Other books that I synced down from Amazon came as straight-up .azw files...what is this directory with the .kcr manifest thing? Is this documented anywhere?

All that said, I did download the latest Kindle Previewer for Windows and gave that a spin. I loaded in the generated .azk via iTunes, it processed correctly (and shows the cover image in the library, again) but when I tried to open it I get the "Please delete and re-download this book" error. So, same thing as Previewer for OSX. If it's an azkcreator bug, it's common across the platforms.

Any other ideas? I may start hacking apart my file to see if something in it is foiling azkcreator or the K4iOS .azk conversion process...
mattmc is offline   Reply With Quote
Old 05-08-2015, 02:05 AM   #9
Doitsu
Grand Sorcerer
Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.
 
Doitsu's Avatar
 
Posts: 5,583
Karma: 22735033
Join Date: Dec 2010
Device: Kindle PW2
Quote:
Originally Posted by mattmc View Post
Any other ideas?
If you haven't already done so, check the ePub source file with epubcheck online.
Also try to sideload the hybrid .mobi file that Kindle Previewer creates to find out if the mobi7 file has problems, too.
Doitsu is offline   Reply With Quote
Old 05-08-2015, 09:05 PM   #10
DaleDe
Grand Sorcerer
DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.
 
DaleDe's Avatar
 
Posts: 11,470
Karma: 13095790
Join Date: Aug 2007
Location: Grass Valley, CA
Device: EB 1150, EZ Reader, Literati, iPad 2 & Air 2, iPhone 7
I think that you may have to just pull some of those files into a text editor and look at them. That is what I did to decode the AZK in the first place and then poked around a bit. The file command in Unix can sometimes magic a value. The AZK I took apart was one assembled as part of the previewer. It is likely that Amazon would massage that file before shipping it as a real file. In the very least they would remove the redundancy of file data in source formats. I suspect it was a single file download that was unpacked for use and there may have been some massaging but whether at the source or on the unit is anybody's guess.

Dale
DaleDe is offline   Reply With Quote
Old 05-12-2015, 03:46 PM   #11
mattmc
Connoisseur
mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.
 
Posts: 89
Karma: 185923
Join Date: May 2015
Device: iPad 1/2/Air, K3/PW2/Fire1, Kobo Touch, Samsung Tab, Nook Color/Touch
Okay, back for more!

Quote:
Originally Posted by Doitsu View Post
If you haven't already done so, check the ePub source file with epubcheck online.
Also try to sideload the hybrid .mobi file that Kindle Previewer creates to find out if the mobi7 file has problems, too.
Good suggestions. From EPUB-Checker, the OSX wrapper for epubcheck:

Quote:
Validating against EPUB version 2.0
(http://code.google.com/p/epubcheck/)
May 12, 2015 10:41:23 AM PDT
---------------------------------------------------
No errors or warnings detected
I also sideloaded the hybrid .mobi from Kindlegen, and it reads fine on K4iOS. It has some formatting issues compared to reading it on an eInk Kindle, but no errors or anything.

Quote:
Originally Posted by DaleDe View Post
I think that you may have to just pull some of those files into a text editor and look at them. That is what I did to decode the AZK in the first place and then poked around a bit. The file command in Unix can sometimes magic a value. The AZK I took apart was one assembled as part of the previewer. It is likely that Amazon would massage that file before shipping it as a real file. In the very least they would remove the redundancy of file data in source formats. I suspect it was a single file download that was unpacked for use and there may have been some massaging but whether at the source or on the unit is anybody's guess.
The book.kcr is a sqlite3 database. Hitting it with the sqlite command-line tool and executing the ".schema" command produces this (I omitted the indexes and built-in tables):

Code:
sqlite> .schema
CREATE TABLE ZBOOKPIECE ( Z_PK INTEGER PRIMARY KEY, Z_ENT INTEGER, Z_OPT INTEGER, ZPIECEID INTEGER, ZTYPE INTEGER, ZENCRYPTEDJSON BLOB );
CREATE TABLE ZLOCATIONMAP ( Z_PK INTEGER PRIMARY KEY, Z_ENT INTEGER, Z_OPT INTEGER, ZLOCATION INTEGER, ZPOSITION INTEGER );
In my case, book.kcr is about 8MB, although my book is quite large. The files in the "resources" folder are anywhere from 150KB to 80B. So it looks like the book itself is stored in the ZBOOKPIECE, in the ZENCRYPTEDJSON column. The table, in my case, has 737 rows. The last row in it looks like this (spaces & newlines removed for brevity):

Code:
737|1|1|43|2|I���F�9�f�{��1�g�P�����ʤg��F�F=�3�q�^�ߔ��X~�W�Z���X���x>�W�����\�ZQ�j�ƾ*�aAd�¹�e^���2`7�`�:O$�D�_�x�z��Xݴ�'�D�ߥ],����f�K�~tA�����.���V�ǕuA=�s�OT$1#���<#g�@�n��4��� hƳ��!��+M�`{V�ڡ3V�j�}��+�w�SdTۚ!(Ef�=_��j�� ���N��"g|°�*"��ޙ����Ȩ�8��y�H��hd��@�<Ȍvpr�{s��A��½��r�J�9&�'���x���+}���#����@:�q��Z��:ϥ&���WUt������q�
So there you go, encrypted json.

The files in "resources" seem to be straight-up binary, so they may be encrypted in some fashion.

-------------------------------------------

Okay, further down the rabbit hole!

I pulled the Kindle.app file off of my iPad with iFunbox (only possible with iOS 5). Examining the innards, I found this:

Code:
KcrBookData.momd/
├── KcrBookData.mom
├── KcrBookData.omo
└── VersionInfo.plist
These are Core Data files. Core Data is the iOS framework that can save/load information from a sqlite database. When I decompiled the MOM file with momdec, I get an XCDATAMODEL file. Opened in Xcode, there's a "BookPiece" entity that has the fields "encryptedJson", "pieceId" and "type". There's also a "LocationMap" entity. So K4iOS is using CoreData with the book.kcr file, clearly.

There's also this:

Code:
KindleCloudReader/
├── KindleCloudReader.html
├── bridge.min.js
├── preamble.min.js
└── renderer.min.js
The KindleCloudReader.html is a skeleton that loads in jQuery and the 3 other JS files. It looks like the HTML file would be displayed in a UIWebView, and the <body> of that HTML file would be populated with data that is dynamically loaded.

The preamble.js file contains some simple functions for determining what iOS platform and OS is in use (iPad, iPhone, iOS 4/5).

The renderer.js file is massive, and contains stuff for rendering glyphs, page turns, displaying images, zooming, building DOM, topaz stuff, default CSS styles, lists of colors, font loading, the whole nine yards. It's 8600 lines of code. Whoever made this thing is definitely a pro.

The bridge.js file is where things get interesting. It has some sort of bridging code (go figure) between the WebView Javascript and the iOS app. First, there's an object called "KCRiOSBridge". It has a method called "openBook". In "openBook", there's a number of sub-commands such as "loadMetadata", "loadFragmap", "loadManifest", "loadSkeleton".

Why is this interesting? In the unprocessed AZK file, we have a "kcrManifest.jsonp". It looks like a javascript file with a function call, "loadManifest", using a JSON blob as an argument.

Code:
loadManifest({"json":"somejson"});
We also have "metadata.jsonp", which has the function "loadMetadata", and "rawManiest.jsonp", which calls "loadManifest" as well. All of the "resource" files begin with the function call "loadResource#", where # is the number in the resource file name. Same with the files in "frags": they have a function call "loadFragment#", and the files in "mobilocations" begin with a function call "loadMobiLocationMap#".

Well...mostly. I noticed that one of my resources is one of my original CSS files, untouched, while another resource appears to be some sort of processed version of the same CSS file. Some of them are fonts (you can see the license & trademark information as cleartext in the middle of the binary font data) and some are straight-up binary, which are probably images.

Anyway, I'm guessing that these various files are executed as javascript snippets by the KCRiOSBridge, inside a Webview. The bridging object communicates with the iOS app via requests with special protocols, such as "command" and "kcr". So you get a URL like "command://KCRPostCall". The iOS app sees that it's not a normal HTTP request like "http://google.com", then intercepts it and uses it to perform some sort of operation.

(This is a standard method of communicating between a WebView--miniature web browser embedded in an iOS app--and the app itself.)

Anyway, I might be getting over some peoples' heads at this point, but my overall guess is that the contents of the AZK file are a series of Javascript snippets, which have the book encoded as JSON. The Javascript snippets are executed by the KCRiOSBridge, which loads the JSON data, and as bridge gets that data it is passed up to the iOS app itself. The app would then create a new "book.kcr" sqlite file, encrypt the JSON and save it into the database with Core Data. When this process is fully complete, K4iOS would destroy the AZK file and simply use the newly created KCR. This is, from what I can see, the process of converting AZK into KCR.

-----------------------------------------

Now, that doesn't really solve my problem. I may have a grasp of the process being used, but the resultant KCR that K4iOS is making is still erroring. The process is too opaque to me to divine what could be going wrong, so I'm left with three options:
  1. See if I can get a more useful error out of K4iOS, rather than "delete and re-download this book". This would involve jailbreaking and attaching a debugger to the app, which is something I've never done before and would be time-consuming, but enlightening.
  2. Compare my AZK to another and see if I can determine any differences that could cause the conversion to "succeed" and yet produce an unusable KCR.
  3. Hack apart my EPUB/AZK and see if I can excise a portion of it that is causing this problem.

So, that's where I stand. Any thoughts? Avenues I may have missed?
mattmc is offline   Reply With Quote
Old 05-12-2015, 04:16 PM   #12
DaleDe
Grand Sorcerer
DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.DaleDe ought to be getting tired of karma fortunes by now.
 
DaleDe's Avatar
 
Posts: 11,470
Karma: 13095790
Join Date: Aug 2007
Location: Grass Valley, CA
Device: EB 1150, EZ Reader, Literati, iPad 2 & Air 2, iPhone 7
Wow, great effort at decoding this mysterious format. It looks like a very complete analysis. Unfortunately I don't have any idea why you are getting failures. Possibly simplifying your source and getting something to work and then adding back pieces seems like it would help.

Dale
DaleDe is offline   Reply With Quote
Old 05-12-2015, 04:53 PM   #13
Doitsu
Grand Sorcerer
Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.
 
Doitsu's Avatar
 
Posts: 5,583
Karma: 22735033
Join Date: Dec 2010
Device: Kindle PW2
Quote:
Originally Posted by mattmc View Post
Hack apart my EPUB/AZK and see if I can excise a portion of it that is causing this problem.
IMHO, that's probably the only approach that'll work.

Try the following:
  • Build an .azk file without embedded fonts.
  • Build an .azk file without images and embedded fonts.
  • Successively delete 50% of the .html files and test the resulting .azk file.
Doitsu is offline   Reply With Quote
Old 05-15-2015, 10:38 PM   #14
mattmc
Connoisseur
mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.mattmc can program the VCR without an owner's manual.
 
Posts: 89
Karma: 185923
Join Date: May 2015
Device: iPad 1/2/Air, K3/PW2/Fire1, Kobo Touch, Samsung Tab, Nook Color/Touch
Quote:
Originally Posted by DaleDe View Post
Wow, great effort at decoding this mysterious format. It looks like a very complete analysis. Unfortunately I don't have any idea why you are getting failures. Possibly simplifying your source and getting something to work and then adding back pieces seems like it would help.
Thanks

Quote:
Originally Posted by Doitsu View Post
Try the following:
  • Build an .azk file without embedded fonts.
  • Build an .azk file without images and embedded fonts.
  • Successively delete 50% of the .html files and test the resulting .azk file.
Yeah, I think I'll take roughly that approach.

Before I dive into the carve-try-repeat process, I wanted to see if I could get some more data from log files. With iFunbox, I was able to find some text logs in Kindle.app/Library/Caches/logs.

This is roughly the log of what occurs when I try to open the broken book:

Code:
I KindleDocViewerFactory:+[KindleDocViewerFactory openHtmlBook:]:Thread(707)::Open Mobi8 Book with ASIN KPR2015-MAY-16_01_57_53.789863:  (info | KindleDocViewerFactory.mm:201)
I OCIPHAnimatedBookView:-[OCIPHAnimatedBookView openBookWithReaderItem:periodicalsModel:readerStyle:inReplicaMode:error:]:Thread(707)::Open book successfully with ASIN: KPR2015-MAY-16_01_57_53.789863  (info | OCIPHAnimatedBookView.mm:230)
I KcrDocWebViewBridge:-[KcrDocWebViewBridge openBook:rendererSize:fontName:fontSize:colors:margins:lineSpacing:numColumns:startPosition:leftJustification:]:Thread(707)::Open KCR book with ASIN KPR2015-MAY-16_01_57_53.789863: , fontName: Georgia ,fontSize: 20.900000 ,renderer's width: 628.000000 ,and renderer's height: 899.000000  (info | KcrDocWebViewBridge.mm:257)
E EANData:-[EANData initWithSidecarData:bookData:onBookOpen:]:Thread(707)::End-Actions: Unable to successfully load sidecar  (error | EANData.m:56)
E KcrDocWebViewBridge:-[KcrDocWebViewBridge pageRenderError:]:Thread(707)::Renderer error: No Message  (error | KcrDocWebViewBridge.mm:763)
Omitted are the logs of all of the "metrics" functions--the Kindle app has very thorough metrics recorded of every event. I also omitted all of the sync-related calls; the app attempts to verify that the book you've just opened is the correct version and syncs down the correct one if it's outdated. Finally, I also carved off the timestamps. As curiosa, each log entry is prefixed with "Lassen: ", the meaning of which I can only guess at--perhaps the German word, which translates to "leave"?

The character all the way to the left of those lines would be the type of the log message: I'm guessing "I" for info and "E" for error. The EANData error can be ruled out, I think, because EAN is essentially a product code, like the ISBN. I loaded up Doitsu's AZK that he gave me, and it also had that EANData error, but it displays fine. So the "pageRenderError" from "KcrDocWebViewBridge" is the killer.

So then, what logs do I have when I load a book in?

Code:
I LibraryManager:-[LibraryManager deleteBooksAtPaths:]:Thread(707)::Removed 0 files in library. 0 books archived, 0 books deleted.  (info | LibraryManager.mm:649)
I LibraryFileManager:-[LibraryFileManager scanDirectoryAtPath:]:Thread(5093)::Library scanning completed for path '/var/mobile/Applications/0931E3A7-F562-4BE9-BB60-ACF67D118443/Documents'. 1 files in path, 0 books added, 1 M8 books sideloaded, 0 books archived, 0 books deleted from core data, 0 files deleted  (info | LibraryFileManager.mm:770)
I KindleApplicationDelegate:-[KindleApplicationDelegate applicationDidReceiveMemoryWarning:]:Thread(707)::Memory warning received.  (info | KindleApplicationDelegate.mm:549)
I LibraryManager:-[LibraryManager deleteBooksAtPaths:]:Thread(707)::Removed 0 files in library. 0 books archived, 0 books deleted.  (info | LibraryManager.mm:649)
I LibraryFileManager:-[LibraryFileManager scanDirectoryAtPath:]:Thread(3c7b)::Library scanning completed for path '/var/mobile/Applications/0931E3A7-F562-4BE9-BB60-ACF67D118443/Library/eBooks'. 9 files in path, 1 books added, 0 M8 books sideloaded, 0 books archived, 0 books deleted from core data, 1 files deleted  (info | LibraryFileManager.mm:770)
I LibraryFileManager:-[LibraryFileManager scanDirectoryAtPath:]:Thread(3c7b)::Library scanning completed for path '/var/mobile/Applications/0931E3A7-F562-4BE9-BB60-ACF67D118443/Documents'. 0 files in path, 0 books added, 0 M8 books sideloaded, 0 books archived, 0 books deleted from core data, 0 files deleted  (info | LibraryFileManager.mm:770)
I LibraryManager:-[LibraryManager deleteBooksAtPaths:]:Thread(707)::Removed 0 files in library. 0 books archived, 0 books deleted.  (info | LibraryManager.mm:649)
Well, all that really tells me is that the "Library Manager" scanned the Documents folder, and found a sideloaded M8 book (Mobi v8?). 30 seconds later, according to the timestamps, it had added 1 book to the library and deleted the document. So there are logs of the conversion process happening, but no errors or anything.

Alright, time to start slicing and dicing...
mattmc is offline   Reply With Quote
Old 05-17-2015, 10:32 PM   #15
Hitch
Bookmaker & Cat Slave
Hitch ought to be getting tired of karma fortunes by now.Hitch ought to be getting tired of karma fortunes by now.Hitch ought to be getting tired of karma fortunes by now.Hitch ought to be getting tired of karma fortunes by now.Hitch ought to be getting tired of karma fortunes by now.Hitch ought to be getting tired of karma fortunes by now.Hitch ought to be getting tired of karma fortunes by now.Hitch ought to be getting tired of karma fortunes by now.Hitch ought to be getting tired of karma fortunes by now.Hitch ought to be getting tired of karma fortunes by now.Hitch ought to be getting tired of karma fortunes by now.
 
Hitch's Avatar
 
Posts: 11,460
Karma: 158448243
Join Date: Apr 2010
Location: Phoenix, AZ
Device: K2, iPad, KFire, PPW, Voyage, NookColor. 2 Droid, Oasis, Boox Note2
Quote:
Originally Posted by Doitsu View Post
IMHO, that's probably the only approach that'll work.

Try the following:
  • Build an .azk file without embedded fonts.
  • Build an .azk file without images and embedded fonts.
  • Successively delete 50% of the .html files and test the resulting .azk file.
FWIW:

You all know I'm not coder, just a poor bookmaker. But in terms of troubleshooting, using fonts, for the purposes of testing an AZK, is worthless, as K4iOS is a KF7 reader. That might at least shorten your troubleshooting efforts. The analysis is truly great, but as you noted--doesn't help you much with the "why" of why your file is failing. Did you take up Doits, I think it was, on his generous offer to try to build the file for you and see if it would then work on your device(s)? Or did I miss that (entirely possible)?

In terms of troubleshooting Amazon build issues--something I do have vast experience in--I highly recommend simple bracket navigation. Start out with something like, rip everything out of your ePUB except, say, Chapter 1. (Or even just the frontmatter, one page/section/file at a time). Build, sideload, and see if that works. If it does, add Chapter 2. It's tedious as crap, but it's the only way I've ever seen really work in narrowing down the problems. I usually manage (except with font issues, which can still remain mysterious, no matter HOW much troubleshooting you do) to find the single item that's screwing up the file that way. Horrifically time-consuming, but at least it's (somewhat) faster than having to load builds from the KDP. (I'm not really posting this for you--you obviously don't need help in troubleshooting. I'm posting this for other, far less experienced folks who come along later and read this thread.)

Speaking of which, for you, however: have you tried loading the same source file at the KDP, and seeing if the post-build MOBI (Step 7, I understand it is now) works on your iPad? Just as a comparative test?

Hitch
Hitch is offline   Reply With Quote
Reply

Tags
kindle azk mobi


Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Converting to azk through KindlePreviewer AThirstyMind Kindle Formats 36 01-29-2015 02:26 AM
AZK issues gdgibson Kindle Formats 3 02-04-2014 05:24 PM
Request support for Amazon AZK format HarryT Calibre 1 01-14-2014 06:28 AM
azk files exaltedwombat Workshop 1 10-21-2013 11:38 AM


All times are GMT -4. The time now is 11:19 AM.


MobileRead.com is a privately owned, operated and funded community.