Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Formats > Kindle Formats

Notices

Reply
 
Thread Tools Search this Thread
Old 08-11-2015, 10:15 AM   #1
jhowell
Grand Sorcerer
jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.
 
jhowell's Avatar
 
Posts: 6,465
Karma: 84000001
Join Date: Nov 2011
Location: Tampa Bay, Florida
Device: Kindles
KFX Format

After installing the latest PW2 firmware with enhanced typesetting I downloaded a book that I had previously purchased so that I could see the new features in action. I looked in my Kindle's documents folder and found the book there with a 'kfx' extension instead of 'azw' or 'azw3'. Does this indicate a new file format for enhanced typesetting Kindle books?
Attached Thumbnails
Click image for larger version

Name:	dump.png
Views:	4218
Size:	24.1 KB
ID:	140941  

Last edited by jhowell; 08-11-2015 at 10:25 AM. Reason: Add image
jhowell is online now   Reply With Quote
Old 08-11-2015, 11:38 AM   #2
Nate the great
Sir Penguin of Edinburgh
Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.Nate the great ought to be getting tired of karma fortunes by now.
 
Nate the great's Avatar
 
Posts: 12,375
Karma: 23555235
Join Date: Apr 2007
Location: DC Metro area
Device: Shake a stick plus 1
Both of the ones I checked were AZW3, not KFX.

On the other hand, neither are showing the improved typography that Amazon promised, so perhaps that is a clue?
Nate the great is offline   Reply With Quote
Advert
Old 08-11-2015, 11:49 AM   #3
odamizu
just an egg
odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.
 
odamizu's Avatar
 
Posts: 1,583
Karma: 4300000
Join Date: Mar 2015
Device: Kindle, iOS
It does indeed look like Amazon has introduced a new ".kfx" format to go with the enhanced typesetting engine:

http://allesebook.de/firmware-softwa...-kindle-65388/

English translation: https://translate.googleusercontent....OhpVElocmuTYvw

Among other things, the new format appears to include soft hyphens.
odamizu is offline   Reply With Quote
Old 08-11-2015, 12:32 PM   #4
jhowell
Grand Sorcerer
jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.
 
jhowell's Avatar
 
Posts: 6,465
Karma: 84000001
Join Date: Nov 2011
Location: Tampa Bay, Florida
Device: Kindles
Some interesting strings from inside a kfx book:

{key:"kfxgen_package_version", value:"PackageVersion:YJReaderSDK-1.1.67.2 Month-Day:06-18"},
{key:"kfxgen_application_version", value:"6.24.1.0"}

From this I'm guessing that Amazon has a kfxgen tool that they use internally to produce these files.

ETA: Perhaps this will become available more broadly if Amazon updates their Kindle Previewer tool to support enhanced typesetting.

Edit 2: Another interesting thing is that the DRM strings are present in the file even for supposedly DRM-free books. It looks like Amazon may be treating this new format as more proprietary than their older ones.

Last edited by jhowell; 08-11-2015 at 02:00 PM.
jhowell is online now   Reply With Quote
Old 08-12-2015, 12:49 PM   #5
jhowell
Grand Sorcerer
jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.
 
jhowell's Avatar
 
Posts: 6,465
Karma: 84000001
Join Date: Nov 2011
Location: Tampa Bay, Florida
Device: Kindles
I did some tests with my PW2 to experiment with the possibility of side-loading a book in kfx format. To do this I purchased and directly downloaded a free book to the kindle. I then switched into airplane mode and using USB I saved a copy of the book's main kfx file and the associated sdr directory. I deleted the book from the kindle and then made several attempts to side-load the book back to it, using a different file name each time.

I was able to successfully side-load. For this to work I needed the main kfx file, a directory of the same name with the suffix of '.sdr', and a subdirectory named 'assets' containing a small file named 'voucher' copied from the original download. This was sufficient to have the book show up as a personal document, but without a cover thumbnail, images, or a table of contents.

I then added the 'metadata.kfx' file that had originally been contained in the assets directory. That made the book functional again with a proper cover thumbnail, internal images and a working table of contents.

Finally I added an 'attachables' subdirectory within assets containing a file named 'CR!FYVV6BXDES0YQ4MCX4CG93WB1Z0B.kfx'. This didn't make any apparent difference in the working of the book in my quick test.

That last file contains a number of high quality jpeg-xr images from the book. I don't understand how the book worked without them. Perhaps there are lower quality images elsewhere as a fallback or perhaps there were cached images somewhere on the kindle.

I only have one kindle so I am unable to test whether this process would have allowed to book to be side-loaded to a different kindle. I suspect that it would not have worked due to DRM.

All of this gives me hope that if the kfx format is ever understood well enough, it may be possible to create and side-load non-Amazon books in kfx format and have them rendered with enhanced typesetting.
jhowell is online now   Reply With Quote
Advert
Old 08-12-2015, 10:58 PM   #6
odamizu
just an egg
odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.
 
odamizu's Avatar
 
Posts: 1,583
Karma: 4300000
Join Date: Mar 2015
Device: Kindle, iOS
Thank you jhowell for all this testing and taking the time to report your findings.

It will be interesting indeed to see if Amazon releases a kfxgen tool or does all the conversion internally.

What does everyone think about the future of mobi and azw3? Do you think Amazon will support three?!? formats going forward, or will they deprecate at least one? I don't mind losing mobi, but at this point I'm feeling rather attached to azw3.
odamizu is offline   Reply With Quote
Old 08-12-2015, 11:26 PM   #7
eschwartz
Ex-Helpdesk Junkie
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 19,422
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
They might kill off MOBI as soon as they feel comfortable abandoning all K2/DX* owners. Probably only a few years away.
AZW3 is currently the bleeding edge as far as the K3/K4/KT know, as well as however many never-connected-to-WiFi Pw1/PW2/PW3/KV's there are. All seven of which Amazon are not going to cut off support for, going by their track record.



* -- I don't know if there are any owners of an Original Kindle, though.
eschwartz is offline   Reply With Quote
Old 08-12-2015, 11:39 PM   #8
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,749
Karma: 22446736
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
While I dont have a Kindle capable of enhanced typesetting myself, here are my guesses about KFX:

1) It is most likely a modified form of azw6 -- azw6 is a format that appeared a few months ago, where a normal azw3 file gets its high-res images downloaded separately from the text and put into the sdr folder. This was likely a bandwidth saving feature. https://bugs.launchpad.net/calibre/+bug/1429007
Alternately, if they have finally gotten tired of Palm Databases, it could be a modified for of AZK -- their special snowflake for iOS.

2) KFX files are likely to be "pre-rendered" to some extent, such as having hyphenation pre calculated and added to the text as either soft-hyphens or html tags. This is similar to what Kobo does with kepub.

3) I would not be surprised if amazon is encrypting all kfx files regardless of whether the original has DRM or not. This would be consistent with their "boiling frog" approach to closing off their ecosystem. Over the years they have gradually made their ecosystem more and more closed and hoped that if they do it slowly enough people wont notice. Cases in point: No email delivery of azw3, the completely unnecessary personal document category which tries to separate sideloaded books from purchased books, azw3 itself which was a completely unnecessary obfuscation of EPUB. It should be easy to check if kfx files are encrypted -- just do a binary search for strings you know appear in the text (preferably short ones so that soft hyphens dont interfere)

Last edited by kovidgoyal; 08-12-2015 at 11:42 PM.
kovidgoyal is online now   Reply With Quote
Old 08-13-2015, 01:56 AM   #9
quiris
Groupie
quiris understands when you whisper 'The dog barks at midnight.'quiris understands when you whisper 'The dog barks at midnight.'quiris understands when you whisper 'The dog barks at midnight.'quiris understands when you whisper 'The dog barks at midnight.'quiris understands when you whisper 'The dog barks at midnight.'quiris understands when you whisper 'The dog barks at midnight.'quiris understands when you whisper 'The dog barks at midnight.'quiris understands when you whisper 'The dog barks at midnight.'quiris understands when you whisper 'The dog barks at midnight.'quiris understands when you whisper 'The dog barks at midnight.'quiris understands when you whisper 'The dog barks at midnight.'
 
quiris's Avatar
 
Posts: 195
Karma: 42216
Join Date: Oct 2013
Location: Poland
Device: Kindles: KOA1, KV
Quote:
Originally Posted by kovidgoyal View Post
It should be easy to check if kfx files are encrypted -- just do a binary search for strings you know appear in the text (preferably short ones so that soft hyphens dont interfere)
They are indeed encrypted. Even free books from Public Domain. The only "readable" part of the binary blob is the beginning of file (DRMION, ProtectedData etc.). Check a screenshot in the first post in this thread.
quiris is offline   Reply With Quote
Old 08-13-2015, 02:08 AM   #10
AcidWeb
KCC Co-Author
AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.AcidWeb ought to be getting tired of karma fortunes by now.
 
AcidWeb's Avatar
 
Posts: 845
Karma: 765434
Join Date: Mar 2013
Location: Poland
Device: Kindle Oasis 2
After observing Amazon movements for last few years another move to close ecosystem even more is not a surprise...
AcidWeb is offline   Reply With Quote
Old 08-13-2015, 10:18 AM   #11
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 7,469
Karma: 5432724
Join Date: Nov 2009
Device: many
Hi,

In case it matters. "drmion", based on a simple Google search, is one variation of a Permutation Cipher that permutes the positions of bytes/characters in each set of K characters to simply protect the data (not really encrypt it). To decipher it you either need the key (or its key length and a piece of known plaintext). It is not a strong cipher.

Of course, the "DRMION" and "ProtectedData" shown in the picture in an earlier post may have nothing whatsoever to do with the drmion permutation cipher (there was an AES/CBC string there as well according to the picture as well) but if someone had access to the full .kfx file and got the same version of the .azw3 or mobi file) you may be able to figure out if they are simply using a permutation cipher to confuse the issue.

My 2 cents ...

KevinH
KevinH is offline   Reply With Quote
Old 08-13-2015, 10:57 AM   #12
eschwartz
Ex-Helpdesk Junkie
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 19,422
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
According to @shalym, there is no DRM:

Quote:
Originally Posted by shalym View Post
Ok...I just tested with a DRM free book that is in the .kfx format, and I was able to transfer the book from my Voyage to my pw3 and have it transfer correctly. I had to move both the .kfx file and the .sdr folder associated with the book.

There were a couple of files that were not able to transfer. The error given was that the filename was too long. (This is using Win 8.1) Those files being missing didn't affect the functioning of the book, at least not as far as I could tell. Hyphenation still works, and the enhanced spacing at larger font sizes works too. I can't judge the kerning and other stuff, as I really can't see that.

Shari
If the book works on two different devices, it cannot be DRMed (I suppose Amazon *could* have moved backwards to per-account keys), so I am going to assume quiris' "unreadable blobs" are just the "new format we don't understand yet" part.

Thanks for all the pessimism.



EDIT2: s/encrypted/DRMed/g because of nitpicking.

Last edited by eschwartz; 08-13-2015 at 03:17 PM. Reason: confirmation below
eschwartz is offline   Reply With Quote
Old 08-13-2015, 11:40 AM   #13
shalym
Wizard
shalym ought to be getting tired of karma fortunes by now.shalym ought to be getting tired of karma fortunes by now.shalym ought to be getting tired of karma fortunes by now.shalym ought to be getting tired of karma fortunes by now.shalym ought to be getting tired of karma fortunes by now.shalym ought to be getting tired of karma fortunes by now.shalym ought to be getting tired of karma fortunes by now.shalym ought to be getting tired of karma fortunes by now.shalym ought to be getting tired of karma fortunes by now.shalym ought to be getting tired of karma fortunes by now.shalym ought to be getting tired of karma fortunes by now.
 
shalym's Avatar
 
Posts: 3,031
Karma: 52740263
Join Date: Feb 2012
Location: New England
Device: PW 1, 2, 3, Voyage, Oasis 2 & 3, Fires, Aura HD, iPad
Quote:
Originally Posted by eschwartz View Post
According to @shalym, there is no DRM:



If the book works on two different devices, it cannot be encrypted (I suppose Amazon *could* have moved backwards to per-account keys), so I am going to assume quiris' "unreadable blobs" are just the "new format we don't understand yet" part.

Thanks for all the pessimism.
I just tried adding the book to an unregistered Kindle by using the same method, and it worked. It would appear that DRM is not added to all books in the new format.

Shari
shalym is offline   Reply With Quote
Old 08-13-2015, 11:56 AM   #14
eschwartz
Ex-Helpdesk Junkie
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 19,422
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
Unlikely possibility confirmed averted!
eschwartz is offline   Reply With Quote
Old 08-13-2015, 11:57 AM   #15
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 7,469
Karma: 5432724
Join Date: Nov 2009
Device: many
So, it might just be obfuscated using the drmion cipher. If someone can pm me a link to all of the pieces needed to side load an kfx ebook and a link to the same ebook not in kfx format, I'd be happy to look at reversing it.

KevinH
KevinH is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Using Kindle format vs ePub format is like using a compiler vs winzip? Julius Caesar Workshop 1 09-01-2013 07:34 PM
iPhone Convert epub format to kindle for iPhone format. Is it possible? thecyberphotog Apple Devices 16 03-14-2013 01:04 AM
No 'epub' format shown in Plugboards Format dropdown list kakkalla Library Management 3 06-16-2012 04:23 AM
Ebook in PRC format will not convert to any other format Katelyn Calibre 0 10-01-2010 07:02 PM
Master Format for multi-format eBook Generation? cerement Workshop 43 04-01-2009 12:00 PM


All times are GMT -4. The time now is 07:50 AM.


MobileRead.com is a privately owned, operated and funded community.