05-04-2015, 03:13 PM | #1 |
Connoisseur
Posts: 89
Karma: 185923
Join Date: May 2015
Device: iPad 1/2/Air, K3/PW2/Fire1, Kobo Touch, Samsung Tab, Nook Color/Touch
|
K4iOS, AZK, The Latest
Hey all!
TLDR; Does K4iOS still use .azk? Introduction I'm doing a fairly meticulous eBook creation process, which involves careful proofing of the book across the full range of popular devices. Obviously, with Amazon's share of the eBook market, this includes all things Kindle. Making the AZK Based on my internet research, I came to the conclusion that Amazon delivers the special ".azk" file type to your Kindle for iOS app. I managed to download Kindle Previewer on my OSX Yosemite machine, and work black magic to get it to run without crashing, at which point I was able to generate the .azk file. Hooray! Loading the AZK Now, I want to sideload my generated .azk onto my iPad. So I follow the simple procedure, going through iTunes -> iPad -> Apps -> Kindle -> Documents, drag it in there. After a short pause, the .azk does appear in my K4iOS library, with the correct cover image. However, when I click on it to open it, I get an error: "The book could not be opened. Please remove the book from your device and redownload it." Removing and re-sideloading the book doesn't help. Neither did un-registering and re-registering. I've tried this on an iPad 1, running Kindle for iPad v3.9.2, as well as an iPad 2, running Kindle for iPad v4.6. Other AZK Files I wondered if maybe there was something wrong with my .azk, or if I had some content in my .mobi file that became messed up in the conversion, or was incompatible with the .azk format. So I set out to get an .azk file by some other means. First, I tried to use the Send to Kindle Mac App to simply send my .mobi file to my iPad, hoping that Amazon's delivery system would convert it to .azk as part of delivering it. It did show up on my iPad, and I was able to open the file. However, when I used iFunbox to examine the book on my iPad, it was an awz file (no encryption). So I failed to make an .azk file that way. (Note that I was only able to check this on my iPad 1 running iOS 5; my iPad 2 has iOS 8.1 and the file structure is different, I believe I'd need to jailbreak to examine the files.) I became curious--if Send to Kindle delivered an awz file (which I believe is just .mobi with different compression & encrpytion, no?) to my K4iOS app when I send a .mobi, under what circumstances does it deliver an .azk file? I realized I had several eBooks, purchased from Amazon, on my Kindle account, so I downloaded them to my iPad from within the K4iOS app. When I inspected them with iFunbox (iOS 5), I found that they were either azw files, or a folder that contained a book.kcr file and another folder called "resources", with a toc.ncx and a bunch of files which no extension. (Not totally sure what this other type of file is, but "kcr" seems like "kindle cloud reader".) So basically, none of the books that I synced down from Amazon onto my K4iOS app seemed to be of the .azk extension. The Questions
Thanks! ==================== Some relevant posts & articles: http://pressbooks.com/blog/testing-p...kf8-azk-gahhh/ http://mandatedmemoranda.com/tag/azk-file/ https://wiki.mobileread.com/wiki/AZK https://kdp.amazon.com/community/mes...ssageID=640003 http://www.newselfpublishing.com/ProofingKindle.html |
05-04-2015, 05:50 PM | #2 |
Grand Sorcerer
Posts: 5,635
Karma: 23191067
Join Date: Dec 2010
Device: Kindle PW2
|
It definitely still supports it.
There are several possibilities why your .azk version didn't work:
(The source file is attached to this post.) |
Advert | |
|
05-04-2015, 08:04 PM | #3 | ||
Connoisseur
Posts: 89
Karma: 185923
Join Date: May 2015
Device: iPad 1/2/Air, K3/PW2/Fire1, Kobo Touch, Samsung Tab, Nook Color/Touch
|
Quote:
Here's what I did:
The end of my KindleGen log, in case it is of interest: Quote:
My .epub file is created with Adobe Indesign CC, latest version. It has custom fonts, images, some special styles, hyperlink footnotes, etc. I don't know that there's anything in it that is abnormal or would compile to .mobi strangely (and KindleGen doesn't find anything wrong with it either). Besides mucking around with my .epub (cutting out parts of it, perhaps, and seeing what part of it might be killing my .azk), any other ideas? Any way to analyze the integrity of my .azk, or retrieve error logs from the K4iOS app? (Hah.) Additionally, and importantly, does Amazon actually still deliver .azk files to K4iOS apps? The app seem to read .mobi files just fine, and the Kindle books that I downloaded from Amazon onto my iPad were either .awz or this weird format with the .kcr manifest. I can be certain now that K4iOS supports .azk, but is there any evidence that Amazon is still using this format? Again, thanks very much for your help with this esoteric corner of the eBook world! Last edited by mattmc; 05-04-2015 at 08:07 PM. |
||
05-05-2015, 02:29 AM | #4 |
Grand Sorcerer
Posts: 5,635
Karma: 23191067
Join Date: Dec 2010
Device: Kindle PW2
|
Since you've excluded all other possibilities, it seems that the OSX version of azkcreator has a bug. (I used the Windows version.) Maybe you could ask another Mac user to install Kindle Previewer and convert your book with it.
(I could also create an .azk file for you if you pm me the file.) Since there weren't any warnings or errors at the end of the build log and .azk files support most .azw3 features, your .azk file will probably render exactly as the corresponding .azw3 file. AFAIK, Amazon never delivered .azk files to iOS Kindle apps. They're only for testing purposes. Sideloading .azk files apparently forces the iOS Kindle app to generate the same file that readers will get, if they download a book for which a KindleGen generated .mobi file exists (which contains the old mobi7 and new mobi8/KF8/AZW3 formats) and their app supports the newer AZK/AZW3 format. |
05-05-2015, 08:33 PM | #5 | |||
Connoisseur
Posts: 89
Karma: 185923
Join Date: May 2015
Device: iPad 1/2/Air, K3/PW2/Fire1, Kobo Touch, Samsung Tab, Nook Color/Touch
|
Quote:
Quote:
Quote:
Otherwise, I'll report back when I've tried the Windows version of azkcreator. |
|||
Advert | |
|
05-06-2015, 05:30 AM | #6 |
mostly an observer
Posts: 1,515
Karma: 987654
Join Date: Dec 2012
Device: Kindle
|
I think you're expected to use iTunes to get the file onto the device, not sideload it yourself.
(I have a Kindle app on my iPhone and have never had any problem with how my books display. I don't use an iPad, except occasionally to check a book on a friend's tablet, and I've never had occasion to load a book onto it other than by downloading from the Amazon store. So I am not au courant on this.) |
05-06-2015, 01:25 PM | #7 | |
Grand Sorcerer
Posts: 11,470
Karma: 13095790
Join Date: Aug 2007
Location: Grass Valley, CA
Device: EB 1150, EZ Reader, Literati, iPad 2 & Air 2, iPhone 7
|
Quote:
Dale |
|
05-07-2015, 06:36 PM | #8 | ||
Connoisseur
Posts: 89
Karma: 185923
Join Date: May 2015
Device: iPad 1/2/Air, K3/PW2/Fire1, Kobo Touch, Samsung Tab, Nook Color/Touch
|
Quote:
Quote:
However, what I see on the iPad is not simply an unzipping of the .azk archive. Here's what the contents of my .azk look like if I open it up: (Note that I trimmed the quantity of files or the post would be 3 pages long.) Code:
└── book ├── frags │** ├── gz_frag0.jsonp │** ├── gz_frag1.jsonp │** ├── gz_frag10.jsonp │** ├── gz_frag100.jsonp │** ├── gz_frag101.jsonp │** ├── gz_frag102.jsonp │** ├── gz_frag103.jsonp │** ├── gz_frag104.jsonp │** ├── gz_frag105.jsonp │** ├── gz_frag106.jsonp │** ├── gz_fragmap.jsonp │** ├── gz_skeleton0.jsonp │** ├── gz_skeleton1.jsonp │** ├── gz_skeleton10.jsonp │** ├── gz_skeleton11.jsonp │** ├── gz_skeleton12.jsonp │** └── gz_skeleton13.jsonp ├── kcrManifest.jsonp ├── metadata.jsonp ├── mimetype.json ├── mobilocations │** ├── gz_mobilocations0.jsonp │** ├── gz_mobilocations1.jsonp │** ├── gz_mobilocations10.jsonp │** └── gz_mobilocations11.jsonp ├── output.json ├── rawManifest.jsonp └── resources ├── resource0 ├── resource1 ├── resource10 ├── resource100 ├── resource101 └── resource102 Code:
└── my_book_title ├── book.kcr ├── gz_fragment555.jsonp ├── gz_fragment556.jsonp ├── gz_fragment557.jsonp ├── gz_fragment558.jsonp ├── gz_fragment559.jsonp ├── gz_fragment560.jsonp ├── gz_fragment561.jsonp ├── gz_fragment562.jsonp ├── gz_fragment563.jsonp ├── gz_fragment564.jsonp ├── gz_fragment565.jsonp ├── gz_fragment566.jsonp ├── gz_fragment567.jsonp └── resources ├── 1004544089 ├── 621764591 ├── 624248723 ├── 689149105 ├── 716323204 ├── 760635517 ├── 768981550 ├── 769084276 ├── 874920377 ├── 884243033 ├── 885818391 ├── 966245230 ├── 975879832 └── 98527426 So, it's more than a simple unzip, there's some sort of processing that's done on these files. On a related note, this is the directory structure of one of the books that I synced down straight from the Amazon cloud: Code:
└── B00AFR4CK2_EBOK ├── book.kcr └── resources ├── 1198181274 ├── 1213596095 ├── 1381148118 ├── 1508313587 ├── 15464168 ├── 1716487958 ├── 1866276677 ├── 1983956603 ├── 2059389305 ├── 242373662 ├── 308183104 ├── 517793479 ├── 582929403 ├── 657402086 ├── 808580617 └── toc.ncx All that said, I did download the latest Kindle Previewer for Windows and gave that a spin. I loaded in the generated .azk via iTunes, it processed correctly (and shows the cover image in the library, again) but when I tried to open it I get the "Please delete and re-download this book" error. So, same thing as Previewer for OSX. If it's an azkcreator bug, it's common across the platforms. Any other ideas? I may start hacking apart my file to see if something in it is foiling azkcreator or the K4iOS .azk conversion process... |
||
05-08-2015, 02:05 AM | #9 |
Grand Sorcerer
Posts: 5,635
Karma: 23191067
Join Date: Dec 2010
Device: Kindle PW2
|
If you haven't already done so, check the ePub source file with epubcheck online.
Also try to sideload the hybrid .mobi file that Kindle Previewer creates to find out if the mobi7 file has problems, too. |
05-08-2015, 09:05 PM | #10 |
Grand Sorcerer
Posts: 11,470
Karma: 13095790
Join Date: Aug 2007
Location: Grass Valley, CA
Device: EB 1150, EZ Reader, Literati, iPad 2 & Air 2, iPhone 7
|
I think that you may have to just pull some of those files into a text editor and look at them. That is what I did to decode the AZK in the first place and then poked around a bit. The file command in Unix can sometimes magic a value. The AZK I took apart was one assembled as part of the previewer. It is likely that Amazon would massage that file before shipping it as a real file. In the very least they would remove the redundancy of file data in source formats. I suspect it was a single file download that was unpacked for use and there may have been some massaging but whether at the source or on the unit is anybody's guess.
Dale |
05-12-2015, 03:46 PM | #11 | |||
Connoisseur
Posts: 89
Karma: 185923
Join Date: May 2015
Device: iPad 1/2/Air, K3/PW2/Fire1, Kobo Touch, Samsung Tab, Nook Color/Touch
|
Okay, back for more!
Quote:
Quote:
Quote:
Code:
sqlite> .schema CREATE TABLE ZBOOKPIECE ( Z_PK INTEGER PRIMARY KEY, Z_ENT INTEGER, Z_OPT INTEGER, ZPIECEID INTEGER, ZTYPE INTEGER, ZENCRYPTEDJSON BLOB ); CREATE TABLE ZLOCATIONMAP ( Z_PK INTEGER PRIMARY KEY, Z_ENT INTEGER, Z_OPT INTEGER, ZLOCATION INTEGER, ZPOSITION INTEGER ); Code:
737|1|1|43|2|I���F�9�f�{��1�g�P�����ʤg��F�F=�3�q�^�ߔ��X~�W�Z���X���x>�W�����\�ZQ�j�ƾ*�aAd�¹�e^���2`7�`�:O$�D�_�x�z��Xݴ�'�D�ߥ],����f�K�~tA�����.���V�ǕuA=�s�OT$1#���<#g�@�n��4��� hƳ��!��+M�`{V�ڡ3V�j�}��+�w�SdTۚ!(Ef�=_��j�� ���N��"g|°�*"��ޙ����Ȩ�8��y�H��hd��@�<Ȍvpr�{s��A��½��r�J�9&�'���x���+}���#����@:�q��Z��:ϥ&���WUt������q� The files in "resources" seem to be straight-up binary, so they may be encrypted in some fashion. ------------------------------------------- Okay, further down the rabbit hole! I pulled the Kindle.app file off of my iPad with iFunbox (only possible with iOS 5). Examining the innards, I found this: Code:
KcrBookData.momd/ ├── KcrBookData.mom ├── KcrBookData.omo └── VersionInfo.plist There's also this: Code:
KindleCloudReader/ ├── KindleCloudReader.html ├── bridge.min.js ├── preamble.min.js └── renderer.min.js The preamble.js file contains some simple functions for determining what iOS platform and OS is in use (iPad, iPhone, iOS 4/5). The renderer.js file is massive, and contains stuff for rendering glyphs, page turns, displaying images, zooming, building DOM, topaz stuff, default CSS styles, lists of colors, font loading, the whole nine yards. It's 8600 lines of code. Whoever made this thing is definitely a pro. The bridge.js file is where things get interesting. It has some sort of bridging code (go figure) between the WebView Javascript and the iOS app. First, there's an object called "KCRiOSBridge". It has a method called "openBook". In "openBook", there's a number of sub-commands such as "loadMetadata", "loadFragmap", "loadManifest", "loadSkeleton". Why is this interesting? In the unprocessed AZK file, we have a "kcrManifest.jsonp". It looks like a javascript file with a function call, "loadManifest", using a JSON blob as an argument. Code:
loadManifest({"json":"somejson"}); Well...mostly. I noticed that one of my resources is one of my original CSS files, untouched, while another resource appears to be some sort of processed version of the same CSS file. Some of them are fonts (you can see the license & trademark information as cleartext in the middle of the binary font data) and some are straight-up binary, which are probably images. Anyway, I'm guessing that these various files are executed as javascript snippets by the KCRiOSBridge, inside a Webview. The bridging object communicates with the iOS app via requests with special protocols, such as "command" and "kcr". So you get a URL like "command://KCRPostCall". The iOS app sees that it's not a normal HTTP request like "http://google.com", then intercepts it and uses it to perform some sort of operation. (This is a standard method of communicating between a WebView--miniature web browser embedded in an iOS app--and the app itself.) Anyway, I might be getting over some peoples' heads at this point, but my overall guess is that the contents of the AZK file are a series of Javascript snippets, which have the book encoded as JSON. The Javascript snippets are executed by the KCRiOSBridge, which loads the JSON data, and as bridge gets that data it is passed up to the iOS app itself. The app would then create a new "book.kcr" sqlite file, encrypt the JSON and save it into the database with Core Data. When this process is fully complete, K4iOS would destroy the AZK file and simply use the newly created KCR. This is, from what I can see, the process of converting AZK into KCR. ----------------------------------------- Now, that doesn't really solve my problem. I may have a grasp of the process being used, but the resultant KCR that K4iOS is making is still erroring. The process is too opaque to me to divine what could be going wrong, so I'm left with three options:
So, that's where I stand. Any thoughts? Avenues I may have missed? |
|||
05-12-2015, 04:16 PM | #12 |
Grand Sorcerer
Posts: 11,470
Karma: 13095790
Join Date: Aug 2007
Location: Grass Valley, CA
Device: EB 1150, EZ Reader, Literati, iPad 2 & Air 2, iPhone 7
|
Wow, great effort at decoding this mysterious format. It looks like a very complete analysis. Unfortunately I don't have any idea why you are getting failures. Possibly simplifying your source and getting something to work and then adding back pieces seems like it would help.
Dale |
05-12-2015, 04:53 PM | #13 | |
Grand Sorcerer
Posts: 5,635
Karma: 23191067
Join Date: Dec 2010
Device: Kindle PW2
|
Quote:
Try the following:
|
|
05-15-2015, 10:38 PM | #14 | ||
Connoisseur
Posts: 89
Karma: 185923
Join Date: May 2015
Device: iPad 1/2/Air, K3/PW2/Fire1, Kobo Touch, Samsung Tab, Nook Color/Touch
|
Quote:
Quote:
Before I dive into the carve-try-repeat process, I wanted to see if I could get some more data from log files. With iFunbox, I was able to find some text logs in Kindle.app/Library/Caches/logs. This is roughly the log of what occurs when I try to open the broken book: Code:
I KindleDocViewerFactory:+[KindleDocViewerFactory openHtmlBook:]:Thread(707)::Open Mobi8 Book with ASIN KPR2015-MAY-16_01_57_53.789863: (info | KindleDocViewerFactory.mm:201) I OCIPHAnimatedBookView:-[OCIPHAnimatedBookView openBookWithReaderItem:periodicalsModel:readerStyle:inReplicaMode:error:]:Thread(707)::Open book successfully with ASIN: KPR2015-MAY-16_01_57_53.789863 (info | OCIPHAnimatedBookView.mm:230) I KcrDocWebViewBridge:-[KcrDocWebViewBridge openBook:rendererSize:fontName:fontSize:colors:margins:lineSpacing:numColumns:startPosition:leftJustification:]:Thread(707)::Open KCR book with ASIN KPR2015-MAY-16_01_57_53.789863: , fontName: Georgia ,fontSize: 20.900000 ,renderer's width: 628.000000 ,and renderer's height: 899.000000 (info | KcrDocWebViewBridge.mm:257) E EANData:-[EANData initWithSidecarData:bookData:onBookOpen:]:Thread(707)::End-Actions: Unable to successfully load sidecar (error | EANData.m:56) E KcrDocWebViewBridge:-[KcrDocWebViewBridge pageRenderError:]:Thread(707)::Renderer error: No Message (error | KcrDocWebViewBridge.mm:763) The character all the way to the left of those lines would be the type of the log message: I'm guessing "I" for info and "E" for error. The EANData error can be ruled out, I think, because EAN is essentially a product code, like the ISBN. I loaded up Doitsu's AZK that he gave me, and it also had that EANData error, but it displays fine. So the "pageRenderError" from "KcrDocWebViewBridge" is the killer. So then, what logs do I have when I load a book in? Code:
I LibraryManager:-[LibraryManager deleteBooksAtPaths:]:Thread(707)::Removed 0 files in library. 0 books archived, 0 books deleted. (info | LibraryManager.mm:649) I LibraryFileManager:-[LibraryFileManager scanDirectoryAtPath:]:Thread(5093)::Library scanning completed for path '/var/mobile/Applications/0931E3A7-F562-4BE9-BB60-ACF67D118443/Documents'. 1 files in path, 0 books added, 1 M8 books sideloaded, 0 books archived, 0 books deleted from core data, 0 files deleted (info | LibraryFileManager.mm:770) I KindleApplicationDelegate:-[KindleApplicationDelegate applicationDidReceiveMemoryWarning:]:Thread(707)::Memory warning received. (info | KindleApplicationDelegate.mm:549) I LibraryManager:-[LibraryManager deleteBooksAtPaths:]:Thread(707)::Removed 0 files in library. 0 books archived, 0 books deleted. (info | LibraryManager.mm:649) I LibraryFileManager:-[LibraryFileManager scanDirectoryAtPath:]:Thread(3c7b)::Library scanning completed for path '/var/mobile/Applications/0931E3A7-F562-4BE9-BB60-ACF67D118443/Library/eBooks'. 9 files in path, 1 books added, 0 M8 books sideloaded, 0 books archived, 0 books deleted from core data, 1 files deleted (info | LibraryFileManager.mm:770) I LibraryFileManager:-[LibraryFileManager scanDirectoryAtPath:]:Thread(3c7b)::Library scanning completed for path '/var/mobile/Applications/0931E3A7-F562-4BE9-BB60-ACF67D118443/Documents'. 0 files in path, 0 books added, 0 M8 books sideloaded, 0 books archived, 0 books deleted from core data, 0 files deleted (info | LibraryFileManager.mm:770) I LibraryManager:-[LibraryManager deleteBooksAtPaths:]:Thread(707)::Removed 0 files in library. 0 books archived, 0 books deleted. (info | LibraryManager.mm:649) Alright, time to start slicing and dicing... |
||
05-17-2015, 10:32 PM | #15 | |
Bookmaker & Cat Slave
Posts: 11,482
Karma: 158448243
Join Date: Apr 2010
Location: Phoenix, AZ
Device: K2, iPad, KFire, PPW, Voyage, NookColor. 2 Droid, Oasis, Boox Note2
|
Quote:
You all know I'm not coder, just a poor bookmaker. But in terms of troubleshooting, using fonts, for the purposes of testing an AZK, is worthless, as K4iOS is a KF7 reader. That might at least shorten your troubleshooting efforts. The analysis is truly great, but as you noted--doesn't help you much with the "why" of why your file is failing. Did you take up Doits, I think it was, on his generous offer to try to build the file for you and see if it would then work on your device(s)? Or did I miss that (entirely possible)? In terms of troubleshooting Amazon build issues--something I do have vast experience in--I highly recommend simple bracket navigation. Start out with something like, rip everything out of your ePUB except, say, Chapter 1. (Or even just the frontmatter, one page/section/file at a time). Build, sideload, and see if that works. If it does, add Chapter 2. It's tedious as crap, but it's the only way I've ever seen really work in narrowing down the problems. I usually manage (except with font issues, which can still remain mysterious, no matter HOW much troubleshooting you do) to find the single item that's screwing up the file that way. Horrifically time-consuming, but at least it's (somewhat) faster than having to load builds from the KDP. (I'm not really posting this for you--you obviously don't need help in troubleshooting. I'm posting this for other, far less experienced folks who come along later and read this thread.) Speaking of which, for you, however: have you tried loading the same source file at the KDP, and seeing if the post-build MOBI (Step 7, I understand it is now) works on your iPad? Just as a comparative test? Hitch |
|
Tags |
kindle azk mobi |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Converting to azk through KindlePreviewer | AThirstyMind | Kindle Formats | 36 | 01-29-2015 02:26 AM |
AZK issues | gdgibson | Kindle Formats | 3 | 02-04-2014 05:24 PM |
Request support for Amazon AZK format | HarryT | Calibre | 1 | 01-14-2014 06:28 AM |
azk files | exaltedwombat | Workshop | 1 | 10-21-2013 11:38 AM |