KFX Format

[lkmiller]

Originally Posted by odamizu

Oy. Just, oy.



Because I want KF8, and if there's an easy way to tell if a book was created with Kindle Create, which will leave me with KFX and MOBI, I won't buy it from Amazon and will go to Kobo for an epub, instead.

In that case, just downloading a sample to a device/app that doesn't use kfx will tell you, right?

[shamanNS]

Originally Posted by jhowell

As I mentioned previously, someone has already worked out a KFX DRM removal algorithm and Apprentice Harper's github repository has working code.

So far no one has bothered to integrate it into the packaged DRM tools. After all, a book in KFX format without DRM is no more useful than one with DRM at the moment.

The working code you're referring to is the one written in Pascal? AmznKFX.exe ? It seems someone also "ported" it to Python:

https://gist.github.com/apprenticenaomi/b0103db2ac134cb51a3ead92f1d99f3a

[jhowell]

Originally Posted by shamanNS

The working code you're referring to is the one written in Pascal? AmznKFX.exe ?

Yes. There is some discussion of it here. It was originally posted on June 16 by someone using the name lulzkabulz in the comments on Apprentice Alf’s blog.

[pdurrant]

Originally Posted by shamanNS

The working code you're referring to is the one written in Pascal? AmznKFX.exe ? It seems someone also "ported" it to Python:

https://gist.github.com/apprenticenaomi/b0103db2ac134cb51a3ead92f1d99f3a

I think a lot more work will be needed to integrate it into the tools than just porting to python.

[darryl]

Originally Posted by pdurrant

I think a lot more work will be needed to integrate it into the tools than just porting to python.

Fascinating reading. My understanding is that it can be integrated into the tools without a lot of trouble, but with limited functionality. It can remove the DRM which will create an unencrypted kfx file. I would think the removal of the DRM infection alone is worthwhile.

What is lacking at the moment is software to convert the unencrypted file to epub or other more useful formats or to read it other than on an Amazon device or Amazon software which supports it. This is not an easy task, but not really a job for the Apprentices. It has been suggested that it be developed by a separate project, perhaps Calibre. I don't know that Kovid is going to have the time or the inclination to do this, particularly not while kf8's are still available and will produce a superior result.

Nevertheless, given the nature of the open source movement I don't think we will be waiting long.

[kovidgoyal]

Until a significant portion of books from amazon come only in KFX format, I am not interested in writing conversion code for it. KFX is, and always will be, an inferior input format for conversion to KF8, because KFX is typically specialised for a particular device. This means that converting it will yield a book that is also specialised for the same class fo device.

If/when large numbers of DRM free/liberated KFX only books start to exist, it should be the work of a week or two to write a KFX input plugin for calibre.

[darryl]

Thanks Kovid. A very rational position and exactly what I expected.

[Notjohn]

Originally Posted by HarryT

KF8 certainly supports Arabic.

For Big Five publishers, I assume? Because Farsi is not a supported language on the KDP platform. Eastern Frisian yes, Farsi no.

https://kdp.amazon.com/en_US/help/topic/A9FDO0A3V0119

[jhowell]

Originally Posted by Notjohn

For Big Five publishers, I assume? Because Farsi is not a supported language on the KDP platform.

Out of curiosity I searched for kindle books in Arabic, but couldn’t find any at Amazon.com. Since that is now implemented as part of Kindle Previewer I expect it will be coming soon, but likely KFX only.

[Doitsu]

Originally Posted by jhowell

Out of curiosity I searched for kindle books in Arabic, but couldn’t find any at Amazon.com. Since that is now implemented as part of Kindle Previewer I expect it will be coming soon, but likely KFX only.

As Notjohn has already pointed out, Arabic is not a supported KDP language, however, all eInk Kindles with KF8/AZW3 support (i.e. Kindle 3 or better with the latest firmware) can display RTL languages in KF8/AZW3 books thanks to the Code2000 fallback font, which is also bundled with Kindle Previewer.

However, the Arabic glyphs in that font are very hard to read and some diacritics aren't properly displayed. (You can see this if you convert this epub3 book with diacritics to azw3. It should look like this.)

In order to fully support Arabic, Amazon will have to bundle an Arabic font that is better suitable for eInk displays, e.g. Noto Naskh Arabic. (They already added Indic Noto fonts some time ago in one of the firmware updates.)

BTW, if you want to run some KFX tests, I uploaded epub and azw3 versions of an Arabic PD book to MR.

[jhowell]

Originally Posted by Doitsu

However, the Arabic glyphs in that font are very hard to read and some diacritics aren't properly displayed. (You can see this if you convert this epub3 book with diacritics to azw3. It should look like this.)
...
BTW, if you want to run some KFX tests, I uploaded epub and azw3 versions of an Arabic PD book to MR.

I was able to convert both your sample book (after removing extraneous chapters and embedded fonts) and your PD book to KFX using Kindle Previewer 3 and a yet to be released version of the KFX Output plugin. I tried viewing them in the Previewer and in the Kindle iOS and Android apps. The books display in all three, but as you stated they appear to be using a fallback font for the Arabic characters. On top of that, the Android app crashes if I try to highlight an Arabic word. Apparently Amazon still has work to do.

[jhowell]

Originally Posted by Doitsu

In order to fully support Arabic, Amazon will have to bundle an Arabic font that is better suitable for eInk displays, e.g. Noto Naskh Arabic.

After digging more deeply in the 3.14 Previewer release I found references to two Arabic fonts: Diwan Muna and Sakkal Kitab.

The actual font files do not appear to be included at this point.

[Doitsu]

Originally Posted by jhowell

On top of that, the Android app crashes if I try to highlight an Arabic word. Apparently Amazon still has work to do.

That's a long-standing bug. On my PW2, highlighting Arabic words and dictionary lookups works fine. However, it takes slightly longer than highlighting words in in Latin alphabet books.

Originally Posted by jhowell

After digging more deeply in the 3.1.4 Previewer release I found references to two Arabic fonts: Diwan Muna and Sakkal Kitab.

Interesting. I thought Amazon would simply include Arabic Noto fonts (Noto Naskh/Kufi Arabic), because they added Indic Noto fonts (Noto Sans Bengali/Devanagari/Gujarati/Kannada/Malayalam/Tamil/Telugu) to the Kindle firmware in 2016. AFAIK, Indic Noto fonts aren't bundled with KP 3.1.4 either. But then again, Caecilia and Bookerly have also been replaced with similar fonts in KP 3.1.4, presumably for licensing reasons.

BTW, both Diwan and Sakkal are well-known font foundries; they licensed some of their fonts to Microsoft and Apple.

[jhowell]

Originally Posted by Doitsu

I thought Amazon would simply include Arabic Noto fonts (Noto Naskh/Kufi Arabic), ...

Kindle Previewer version 3.15 is now out. I haven't had time to look at it much yet, but I did notice that it includes four new fonts:

Sakkal Kitab
Diwan Muna
Noto Naskh Arabic
Noto Naskh Arabic UI

All have regular and bold variants.

[jhowell]

The Kindle Previewer Q&A now states:

Q6. My book is not compatible with Enhanced Typesetting. What should I do to make it compatible with Enhanced Typesetting?

A6. We are actively working to provide guidance so that you can ensure that your book supports Enhanced Typesetting. Meanwhile, we continue to expand the capabilities of Enhanced Typesetting and we will automatically enable it for your book on the store when possible. You need not resubmit your book. When Enhanced Typesetting is enabled for your book, you will see "Enhanced Typesetting: Enabled" on that book’s detail page.

I noticed that Kindle Previewer version 3.15 has a hidden feature called YJ Coach. That provides guidance in fixing problems that prevent books from converting to KFX and supporting Enhanced Typesetting. I assume that this feature will be enabled for use in a future version of the Previewer and at that time the advice will change from "wait for us to improve ET conversion" to "fix the problems in your damn book".



As a side note, the Q&A also states:

Q13. How can I side load and preview my book with Enhanced Typesetting on my Kindle device or application?

A13. You can’t side load your books with Enhanced Typesetting. You can, however, side load the Mobi file without Enhanced Typesetting. ...

That is true using Amazon's software, but the KFX Output calibre plugin allows you to do just that! (The next version will also capture the hidden guidance mentioned above.)



And as a side-side note, I am close to releasing a KFX Input calibre plugin that will convert from KFX to other e-book formats.