KFX Format

[DiapDealer]

Even if one could turn off hyphenization, I would never support putting that control into the hands of the ebook designers. Not for normal body text anyway. The the power to control normal body text hyphenization/justification should lie solely in the hands of the end user. Between designers having the control and no one having control, I choose no one.

[mattmc]

Originally Posted by JSWolf

That will not work at all. Not one bit. Nope, nada. Not a hope in heck.

Why won't it work? because you cannot modify a KFX file to change the CSS.

We're not talking about modifying a KFX file; we're talking about putting CSS in your ePub, which will then run through Kindlegen and eventually Kfxgen.

Originally Posted by jhowell

No, I do mean 4.9. The current version is 4.10, not 4.1. (Version numbers often use a period as a separator, not a decimal point.)

Wow, I missed that. You're right, it's 4.10 now.

Originally Posted by jhowell

ETA: I see upon further reading that you are looking at hyphenation control from the point of view of an author/publisher rather than a reader.

Yes.

Originally Posted by jhowell

I just did a test using a personal document I have in my account, created by emailing a combined MOBI/KF8 file from kindlegen.

Sending it to my PW2 results in an AZW3 file. But doing the same to my iPad with the Kindle app results in a MOBI file.

So it seems that Amazon doesn't deliver an AZK file for personal documents as it does for purchased books.

That is interesting.

At the risk of being nitpicky, AZK is not, to my knowledge, delivered by Amazon. It's an intermediate file format that Kindle Previewer produces, using a piece of software called azkcreator. When you load it into K4iOS, the app converts it into a .KCR file, which I believe stands for "kindle cloud reader". (I suppose that format is what's used for Amazon's browser-based reader of the same name.)

Anyway, that was what I found after a series of tests, both sideloading AZK and downloading books from Amazon into K4iOS, and then inspecting the downloaded files.

The real point here is that Amazon's Send-To-Kindle delivers a MOBI to K4iOS instead of a KCR, so that's not a valid proofing method. Doing an AZK side-load may be more accurate, but Hitch has claimed that even that method will not reproduce the final product you'll get when downloading straight from Amazon.

(Speaking of Hitch, has she weighed in on this discussion? I'm curious to know if she's talked to Amazon about the KFX hijinks.)

Odamizu, have you tested any hyphenation settings with AZK side-loading? MOBI certainly doesn't support that kind of thing, but KCR may.

Originally Posted by DiapDealer

Even if one could turn off hyphenization, I would never support putting that control into the hands of the ebook designers. Not for normal body text anyway. The power to control normal body text hyphenization/justification should lie solely in the hands of the end user. Between designers having the control and no one having control, I choose no one.

I get your concern, but I think the middle path is better. The best way would be to allow the designer to have control, but let the reader override if they really want to.

For example, in iBooks, if you have the "Original" font selected, the creator's CSS settings for justification, hyphenation and all that will apply. If you change to another font, iBooks' settings takes over (which the user can change).

Amazon's approach is just a heavy-handed dictatorship of the typography settings.

[DuckieTigger]

Originally Posted by mattmc

Amazon's approach is just a heavy-handed dictatorship of the typography settings.

No it is not, IMPO. I agree with DD, if I (as reader) don't have control over hyphenation, then I sure wouldn't like the designer of the ebook forcing stuff down my throat. I happen to like hyphenation - if the book is not in KFX then I can add the hyphens myself to the KF8 file. For those that don't want hyphenation, they can D/L the KF8 version and sideload that and not use the KFX version.

It is bad enough that some books slip through that are still delivered with forced left justification for body text.

I don't like the special snowflake books that insist it is fun to mess with the style of the body text. Please leave it at the default. Chapter headings, section breaks, and such? Sure get creative and mix it up. For full body text all you do is aggravate those that just want to read.

[jhowell]

Originally Posted by mattmc

At the risk of being nitpicky, AZK is not, to my knowledge, delivered by Amazon. It's an intermediate file format that Kindle Previewer produces, using a piece of software called azkcreator. When you load it into K4iOS, the app converts it into a .KCR file, which I believe stands for "kindle cloud reader". (I suppose that format is what's used for Amazon's browser-based reader of the same name.)

To be precise, in the case I was describing Amazon delivers a set of JSON files that have the same format as the contents of an AZK file. I hadn't made the connection before, but it does look like the contents of an AZK file are the same type of JSON files that the cloud reader app uses. I guess it was easier for Amazon to put their cloud reader web app into the Kindle for iOS app than to implement full KF8 support there. That could be the explanation for the existence of AZK.

(That still leaves of question of why KFX was created. I doubt that it was done just to introduce some typographic enhancements.)

[AaronShep]

Originally Posted by mattmc

So, Amazon is using Webkit? Missed the memo on that one.

Yes, Webkit is the basis of KF8, and now KFX.

Originally Posted by mattmc

You mention spacing issues between paragraphs (obviously talked about more in this thread) and the apparent conversion of images to opaque JPEGs, but have you noticed anything else off about KFX?

Mostly the things I mention in my blog posts. To me, one of the biggest is ignoring the nonbreaking space. I was using it for tricks like avoiding orphan words at the ends of paragraphs -- though I've now worked out an alternative.

Originally Posted by mattmc

It's this concern that may lead me into getting my hands dirty with the KFX format. Something I'd really rather not do, but I have to be able to proof my books.

Please do. It's going to take a heap of complaints to get Amazon to change course and allow us control over our own books. And it will be hard to get those complaints when most authors and publishers will never see the changes that are made after the fact.

Aaron

[AaronShep]

Originally Posted by odamizu

I've never been able to turn off hyphenation on Kindle for iOS app.

That app has only recently started using KFX. Previously, it was a different system of hyphenation -- and a horribly inaccurate one. It's all different now, for books with Enhanced Typesetting enabled.

Aaron

[AaronShep]

[QUOTE=JSWolf;3178360]

Originally Posted by AaronShep

That will not work at all. Not one bit. Nope, nada. Not a hope in heck.

Why won't it work? because you cannot modify a KFX file to change the CSS.

The CSS goes into the file you submit. Amazon lets it pass through unchanged in this instance.

I ran my own tests using different CSS instructions in combination, because I didn't know which, if any, would work. I managed to turn hyphenation both on and off -- in fact, I was able to turn it on in KF8, on the Kindle Previewer!

At the same time, though, I wrote KDP and asked for their advice on this. They're the ones who singled out the Webkit CSS as the CSS that was effective.

By the way, I do not turn off hyphenation in straight body text. But I do turn it off for headings, verse, and anywhere else it isn't appropriate. And without my even asking about it, KDP mentioned such instances as appropriate uses for these controls. Maybe eventually they'll document it for others, but right now, a lot of things aren't getting shared.

Aaron

[mattmc]

Originally Posted by DuckieTigger

I don't like the special snowflake books that insist it is fun to mess with the style of the body text. Please leave it at the default. Chapter headings, section breaks, and such? Sure get creative and mix it up. For full body text all you do is aggravate those that just want to read.

Obviously this is a rather heated topic.

To me, the best solution is to let the user choose whether to use the publisher defaults or not, the way iBooks does. Adding or removing hyphenation by downloading different file formats and/or modifying them isn't really very user-friendly. That's why I said Amazon was being a dictator on the typography.

Originally Posted by jhowell

To be precise, in the case I was describing Amazon delivers a set of JSON files that have the same format as the contents of an AZK file. I hadn't made the connection before, but it does look like the contents of an AZK file are the same type of JSON files that the cloud reader app uses. I guess it was easier for Amazon to put their cloud reader web app into the Kindle for iOS app than to implement full KF8 support there. That could be the explanation for the existence of AZK.

(That still leaves of question of why KFX was created. I doubt that it was done just to introduce some typographic enhancements.)

Thanks, I think we're on the same page here on AZK, even if we're coming from slightly different angles

I think the easiest answer to "why KFX" is probably the opacity. If the whole file is encrypted in AES/CBC, regardless of whether the file is DRM'd or not...I mean, it's pretty obvious. They don't want anyone looking at the internals. But the "enhanced typography" certainly provides a good cover.

I mean, I could be wrong, for sure. If Amazon starts passing out the kfxgen utility so creators can properly proof their books, then I was wrong.

---

Now, general question: I know that MobileRead rules prohibit discussion of removal of DRM--what about the removal of KFX obfuscation (or encryption, if you want to call it that)? What are the moral and legal implications there?

I didn't think there was an issue, but jhowell's kfxmeta.py script has this clause:

This program does NOT extract actual book content. It does not deal with DRM or encryption. Use of this tool to aid in content extraction or DRM removal is not sanctioned!

So I thought I'd ask.

[AaronShep]

Originally Posted by DuckieTigger

It is bad enough that some books slip through that are still delivered with forced left justification for body text.

I don't like the special snowflake books that insist it is fun to mess with the style of the body text. Please leave it at the default. Chapter headings, section breaks, and such? Sure get creative and mix it up. For full body text all you do is aggravate those that just want to read.

The Kindle uses a primitive form of justification that is common to word processors and Web browsers but is unsuited to books, and even more unsuited to text in narrow columns -- yes, even with hyphenation added. I have no idea why you think that turning off such amateurish typography should be aggravating to readers.

Aaron

[DiapDealer]

Originally Posted by AaronShep

I have no idea why you think that turning off such amateurish typography should be aggravating to readers.

Because it IS aggravating to some readers. Myself included. "Amateurish typography" is simply not an objective thing that's agreed upon by everyone. Opinions abound. That's why things like justification, hyphenization, base font size, and linespacing (for regular body text) should be left up to the people that ultimately have to look at it.

[jhowell]

Originally Posted by mattmc

Now, general question: I know that MobileRead rules prohibit discussion of removal of DRM--what about the removal of KFX obfuscation (or encryption, if you want to call it that)? What are the moral and legal implications there?

I am more concerned about Amazon (and their lawyers) than I am about MobileRead.

I have intentionally stayed away from examining the details of encrypted KFX files. As can be seen in the first post in this thread, even "Simultaneous Device Usage: Unlimited" books contain strings like "amzn1.drm-key.v1" and I don't see Amazon calling these books "DRM-Free" anywhere. I don't want to be put in a position of having to defend against an accusation of DMCA violation so I am going to avoid reverse engineering this aspect of KFX. (But I will be happy if someone else does so.)

[AaronShep]

Originally Posted by DiapDealer

Because it IS aggravating to some readers. Myself included. "Amateurish typography" is simply not an objective thing that's agreed upon by everyone. Opinions abound. That's why things like justification, hyphenization, base font size, and linespacing (for regular body text) should be left up to the people that ultimately have to look at it.

It's a nice sentiment, but what I've seen over and over is that people are conditioned to favor poor typography because that's what they're used to nowadays. So, principles of good typography that were developed over centuries have been discarded in favor of the preferences of engineers raised on Microsoft Word. I don't think anyone with a grounding in book typography would see the Kindle's typography as any better than a necessary evil.

Still, I wouldn't object to reader control if that's actually what we were offered. But if it's a choice between Amazon's dictating the typography or me, I won't willingly leave it in the hands of people who have proven themselves clueless.

Aaron

[odamizu]

Originally Posted by mattmc

The real point here is that Amazon's Send-To-Kindle delivers a MOBI to K4iOS instead of a KCR ...

Yes!

Originally Posted by mattmc

Odamizu, have you tested any hyphenation settings with AZK side-loading? MOBI certainly doesn't support that kind of thing, but KCR may.

Just tried it. Thanks for the idea! If I use Kindle Previewer to create an AZK, then sideload it directly to my iDevice, the hyphen CSS works as expected, so it definitely survives the conversion from epub > AZK (and so does some other formatting that gets lost when going from epub > mobi).

Sadly, this isn't much help to me since Send-to-Kindle doesn't accept AZK, and I need S2K to get WhisperSync. But good to know AZK/KCR supports hyphen CSS.

Originally Posted by AaronShep

That app has only recently started using KFX. Previously, it was a different system of hyphenation -- and a horribly inaccurate one. It's all different now, for books with Enhanced Typesetting enabled.

Yes, old hyphenation system is terrible. But KFX isn't of interest to me as long as I can't crack it open, tweak the formatting to suit my visual impairment needs, and sideload it.

Originally Posted by DiapDealer

... things like justification, hyphenization, base font size, and linespacing (for regular body text) should be left up to the people that ultimately have to look at it.

Yes, yes, YES!

[JSWolf]

Please delete any text relating to turning off hyphenation in Amazon eBooks. This just leads to abuse. I've seen many ePub with CSS commands to turn off hyphenation for ADE and webkit and that's just wrong on so many levels.

[DuckieTigger]

Originally Posted by AaronShep

I won't willingly leave it in the hands of people who have proven themselves clueless.

That is a rather harsh assumption. Maybe you be so kind and suggest an algorithm for your perfect dream book typography that also meets these basic requirements: that it shall work with any font (obviously with different results if font lacks a kerning table) and that it is sufficiently fast on a typical ereader (less than half a second per page) and on the fly. If not it won't work - nobody wants to wait forever after changing the font size or font.

Maybe you could be nice enough to give an example of program that outputs a nice typeset book no matter how long it takes.