Amazon's vertical integration/monopoly

[pdurrant]

Originally Posted by notimp

kfx is

- Scrambled by design, which makes it "unfit for archival purposes" not my words, the words of the guy that tried to to reverse engineer the container, for several months.

Please provide a reference for that quote, or remove the implication that someone else said it. "unfit for archival purposes" does not appear on MobileRead except in your post, as far as I can tell.

[notimp]

No.

I will not remove what I have written from the public record - what world are we living in, that something like this is even suggested. Even more from a person with some structural power in this community behind them.

What I will do - is to excuse myself for not having kept the comment exactly in memory and having produced a wrong citation.

There was no malice behind it and no nefarious intent.

The apology also extends especially to jhowell for bringing him in a situation where he felt he had to defend himself, and had to take the time to resurface the original quote.

Now, that this two minute formality is dealt with - lets talk about semantics.

The format is unfit for archival purposes.

Not just "worse than all formats that came before" - unfit.

Why?

- Unscrambling the format still is not an option, valid attempts, potentially unreleased code and structural decisions aside - this community has decided distinctly - not to open up .kfx, or was unable to do so - at least for the time being, touting the structural "let us have all our users move through the Amazon "legacy format soupkitchen" process" instead - as a better solution.

The format is trash. Binable. From the perspective of the customer it has no other properties other than allowing him to get his license agreements worth and be able to look at that thing. Then he/she has the option to delete it. Thats it. Thats the featureset.

And the look at it part is conditional on having a Kindle around, with a registered user account, thats at least somewhat up to date - because as of now, its the only device that allows us to even look at those things.

- Second point. The structural change in rollout, makes it absolutely certain - that the moment anyone releases information on how to unscramble the format - the format will be changed again. Because structurally the only one that has any stakes in creating it is set up to be Amazon alone. And in the new ecosystem they can rollout a "patched" format in hours not days.

Authors have to produce something different, then comes Amazon and produces the book. Readers buy something different, then comes Amazon and auto delivers the file format.

No one but Amazon comes in direct contact with the format itself again, rollout is automated, blogs only mention in a byline - if a new format is released - no one cares. Structurally. Because no one is allowed to have any stakes in the new format anymore.

If any of you is willing to make the case, that such a format, in its current state is "fit for archival purposes" - make the case.

Don't threaten me with powerplays into retracting that statement.

The citation was wrong (in as much that jhowell only stated, that it is a worse format than any other format that came before)), but the fact is correct. [Moderator addition: Note that jhowel never said that KFX is a worse format than any other format that came before. Mod note added following notimp's refusal to edit this post.]

If you can look your constituency in the eye, and are able to tell them, that they don't have to worry about the fact, that their Kindles are filling up with scrambled files, that might never be recoverable (in a sense, that you shouldnt need to have to be registered, and buy a certain reader to even look at them), do so.

Don't play power games around semantics that have no real world equivalent at all.

You had people in here on the last page who had the impression, that .kfx would "just be .html" - even on the functional level ("Those are all .html based file formats, where is the difference -") - how long do you wan't to keep up this notion?

Its wrong.

We all know it - but we are very careful to dress it in words. I haven't found out exactly whose public image we are guarding here - but I'm not sure, that it is in any of our interests.

[jhowell]

Originally Posted by notimp

The citation was wrong (in as much that jhowell only stated, that it is a worse format than any other format that came before)), but the fact is correct.

As I wrote previously, I believe that KFX is inferior to KF8 for some purposes. I did not compare it to other Amazon formats.

Please do not attribute your opinions to me.

[pdurrant]

Originally Posted by notimp

No.

I will not remove what I have written from the public record - what world are we living in, that something like this is even suggested. Even more from a person with some structural power in this community behind them.

Moderator Notice

Edit your misleading posts now. If you do not wish to delete any of your words, add retractions and explanations immediately before or after the offending lines.

[notimp]

Originally Posted by jhowell

As I wrote previously, I believe that KFX is inferior to KF8 for some purposes. I did not compare it to other Amazon formats.

Please do not attribute your opinions to me.

This is such a backpaddling weighing of words with an intent not to make it possible to attribute anything to you.

In the end it is your deeply held believe that daisies smell better than daffodils, and please let this be the only thing thats ever attributed to your statement.

What does that mean for the archiveability of kfx - and why do you insist, that you didn't say .kfx was worse than all other Amazon formats (it is..), but just the one you'd like to compare it to today?

How about you take a step back, look at the situation and start commenting on it structurally, and not just in the ultra micro focused way of "the only thing you thought would be worth commenting on was how two formats compared" - and apart from that there is no story.

For what its worth, in context - your comment back than was more than just a technicality you made an off handed remark about. In context, you talked about .kfx loosing significant "properties" compared to what we had before. This included older file formats as well - because, those are the facts. If you now don't want to extend your comments to have any broader meaning at all -

enlighten me with a comment as to why.

I've said it before, we have some very bright minds in here, who have done remarkable things in the past, but the constant fear to take any political or even societal "stance" in an important matter urks me to no end.

When push comes to shove - all everyone has ever said were nonsequitors, void of any meaning in regard to the bigger picture.

Sure, we can play this game.

So what you are saying is, that .kfx is a format thats TOTALLY suitable for archival purposes, and thats far better than .mobi (or are you talking about topaz?) in oh so many ways - you don't care to specify.

Nice public image about .kfx you are crafting here...

(Cursed if you say something, cursed if you don't - the question is, if there truly is the "neutral" standpoint you want to resort to - in face of the structural power mismatch (between Amazon auto delivering... (fill in the rest)).)

I let myself get shoved around only so much. If you feel the need not to be drawn into this debate - say so openly. I can at least respect that as a stance.

[notimp]

@pdurrant: I edited the paragraph to your liking. Celebrate. Flawless victory.

[pdurrant]

Originally Posted by notimp

@pdurrant: I edited the paragraph to your liking.

Thank you.

Now please correct your other misrepresentation in this post where you manage to mention jhowell by name while you put words in his mouth.

[pdurrant]

Originally Posted by notimp

What does that mean for the archiveability of kfx - and why do you insist, that you didn't say .kfx was worse than all other Amazon formats (it is..)

I imagine that he insists he didn't say that .kfx was worse than all other Amazon formats because he didn't say that.

You are welcome to express your opinion that kfx is worse than all other Amazon formats.

You may not falsely state that another also holds that opinion.

This is really simple. "Please do not misrepresent someone else's views when involved in a discussion."

[pdurrant]

Back to the subject you seem to want to discuss: Amazon file formats.

The Topaz file format is much, much worse than KFX in all ways. And yet that was decoded. Because it was necessary.

AZW4 is a wrapped PDF. That's also worse than KFX for conversion to other formats.

As for freedom to change format - Amazon have always had that option. It's how they've gone from just Mobi to Mobi + Topaz to Mobi + Topaz + KF8 to Mobi + Topaz + KF8 + MobiPDF to Mobi + Topaz + KF8 + MobiPDF + KFX.

Currently there isn't an easy way to decrypt and convert KFX. You seem convinced that because there isn't one now there'll never be one. My experience suggests that you're wrong.

[jhowell]

Thank you pdurrant. I agree with your recent posts above.

Continuing the discussion of KFX...

KF8 (aka AZW3) is a particularly good format for conversion and archival because little is lost when an EPUB is converted to KF8 by kindlegen. File names and directory organization are not retained, but HTML and CSS content is. Using the kindleunpack tool, you can unpack a KF8 to an EPUB that is pretty close to what the publisher provided to Amazon originally. Because of this, I consider KF8 to be the best Amazon format for conversion and archival.

KFX is basically a binary representation of the formatting information conveyed in the source format, designed for quick rendering on devices with limited CPU power. This allows added functionality that might not otherwise be possible on Kindles without impacting performance.

Extraneous information is removed in the production of KFX. CSS styles are flattened and class names are lost. (The result is similar to that obtained by doing a format conversion in calibre.) In addition a number of heuristics are applied intended to improve the look of rendered pages. For example, large vertical spacing is tightened. These changes do not represent the loss anything that is important to actually being able to read the book. Conversion to the old MOBI format results in much more formatting loss than conversion to KFX.

At this point there are no publicly available tools to remove KFX DRM and convert it to other formats. I have been working on a KFX to EPUB converter for my own use. (Example here.) Producing a tool for the removal of KFX DRM could lead to legal issues due to the DMCA in the US so I am unwilling to undertake that. (The DMCA is something that I believe to be worthy of protest, not KFX.)

[notimp]

Thanks to pdurrent we now find ourself in fully postfactual discussion, where people are encouraged to discuss formats that never mattered, never had any traction, never suceeded, and never were relevant at all - just because it gives our two Christmas elves (love your green writing...) an "out" - for not having to call .kfx by its name.

Or featureset.

If we don't like to do that - we can kindle our emotions around the other sidetopic of people in here being forced to retract statements after the fact - because they are debatable, or wrong - just in a dead cynical game of "I have power over you" - also preventing the argument from getting anywhere - in fact, they are hindering it from taking place at all.

In a world where we have to edit and reedit our statements, because we value someones feelings of having been misinterpreted so much, much more - than the actual argument made, with or without their consent - but still using their findings.

To escalate this one further - I see this as an example of structural abuse.

The agenda of the topic is set aside to be replaced with a strange call for "personal political correctness" no one does benefit from. (I don't consider "being able to retract yourself and safe face" to be a benefit in the structural sense, it doesn't solve, underline, explain, focus the issue - its just further loss of positions, options, traction - it creates more unneccesary confusion...)

The issue that .kfx has some really strange and unwanted properties should now be replaced with an artificial focus on topaz which was a format that was intended to be a pdf replacement and never worked. Never succeeded. Never had any significant distribution.

.kfx (.kf10) is set up to be the replacement format of azw3 (.kf8).

Trying to sidetrack the discussion, with the added bonus of structural abuse of power
(try not writing everything in green and lets see how much sense the things you demand actually make - in a world where not everyone has to see every word you say as being a "Machtwort" (a "word of law" in the institutional sense)), is usually something you come to expect from paid trolls and not of forum moderators.

I know that this is something different than the usual 10 "I didn't read the manual, can you help me" posts you have to deal with day in and day out in here -- but a

- lets stick by the manual, and oh - page 10 says, we should also mention topaz as something with different problems - position, doesn't get us very far here.

In fact -

while still insisting, that none of this has to be looked at from a macro level ("there are other industry players, so we all can look the other way...:") you are now doubling down on looking at the ultra micro level -

- "I only ever said it was structurally more unfit, than one specific other format - and I'm unwilling for my statement to be used in a broader context"

and the outright joke of an

- .kfx doesnt look so bad (oh no, it does - but we are hard at mixing metaphers to find any reason for not looking at the stuff that .kfx has caused compared to the previous "format generation") - compaired to a format, thats the pdf equivalent from Amazon.

You are now demanding, that we have a discussion in here why pdf is worse than epub?

While people are loosing all file properties they had for the better part of the digital revolution, with .kfx?

And you do this with the "voice of a moderator", and in a bold font?

Where is the complaint box in this forum - I'm on the receiving end of a mighty creative attempt to kill this conversation.

Oh, and thanks for doubling down on "you also need to *selfcensor* another posting, because I say so" - shows just how used you are to being in a position of power. Ok, children - playtime is over -

Lets talk about the structural changes .kfx brought to the Amazon ecosystem in 2016 - once more. The year is almost over - and large parts of this community still insist, that its just "a different flavor of nice".

It has taken all properties of a usual file format for distribution from us (.mp3, mp4, ... even .doc (was at least reversed) had them ...) but this community refuses to acknowledge its role of having allowed this to take place largely unnoticed.

Don't bother talking about defunct file formats that never played a role - the place in here is too precious.

Oh, and in the spirit of the holidays - answer a few more "can someone read the manual to me" questions while you are at it. For the benefit of the community.

[notimp]

Or as tldr;

I am pulling the "don't push me around card" - and the moderater decides to push me around a little more.

Flawless victory, last time around -

This time, can I encourage everyone to give a big hand of applause - for ongoing attempts in bullying, sidetracking, taking over the intent of the thread, spinning the messaging, ...

(From my POV - the retraction that the comment about .kfx from the reverse engineer has to be seen only as an isolated comparison between two file formats - was just that.)


Oh - and give me a few minutes so I can read jhowells posting as well, because apparently it also makes some structural arguments about the file format, that should be worth considering.

edit: As a final aside - "publicaly agreeing with the moderator - just for the sake of doing so" also hasn't gone by unnoticed. (*cough* bullying *cough*).

Because - what does this mean? "I publicly agree with your point, that he should also censor his other post where he makes the same point?" ("There, teach - he missed one more! And you caught him!")
"I publicly agree with you point, that we have to see this in context with an entirely unrelated file format, that no one used - just so the "its not THAT bad" position gets more believeable"? (No, .kfx doesn't suddenly get less problematic, because somewhere also a .pdf equivalent exists.)

[Cinisajoy]

And again. Other than ranting about kfx, what do you want consumers, developers or Clyde to actually do about it?


Do you have 100% documented proof that kfx will replace azw3 and make all ebooks on ALL devices unreadable?

Just to clarify and I may be wrong, but I do not think kfx will ever be the only ebook format.

It isn't now and Amazon is not the only place to buy ebooks. KFX is just one of their formats that happens to be at the moment proprietary.

Now you have said some things about authors having to upload a very specific file to Amazon or Amazon not giving an author a choice of what format to send. Amazon sends the best format for the particular device the customer is using. The author shouldn't have anything to do with that because if the author sends the wrong file well he has potentially lost a customer. Yes, there are people that have no clue what device they actually own. And there are authors that have no clue how ebooks work.

All the guys are asking for is documented proof of some things you have said.

I just want to know what you expect people to do about the file format you think is going to take over the world.

Oh and ranting about the people you are trying to engage in the discussion is not helping your cause.

Now just tell me what the **** you want us to do about this format.
Preferably in under 5 paragraphs.

[DiapDealer]

It has taken all properties of a usual file format for distribution from us (.mp3, mp4, ... even .doc (was at least reversed), ...) but this community refuses to acknowledge its role of having allowed this take place largely unnoticed.

It hasn't taken a single thing from me. I buy a considerable amount of books from Amazon, and despite the apparent "takeover" of the kfx format, I've yet to buy, download, or read a single Amazon book in the kfx format. Not one. So why do I care about its inability to be converted to other formats again? Kfx continues to be a non-factor for those who place a high value on archiving and/or converting their purchases. And it will continue to be so until there are no easy end-runs around kfx.

Contrast that with the many Topaz books that got foisted upon me over the years (with no way to avoid getting them without going to another retailer).

[notimp]

And here is the rebuttle to @jhowell "creation myth" for .kf8.

I have read it before. (Not sure if by him or someone else.)

I always found it incredibly hard to believe.

The argument goes as follows.

PLEASE UNDERSTAND, that we had to take away your ability to understand the format (= open standard, readable, tweakable, citable (citable outside the Kindle framework) - even human understandable for anyone without a reverse engineering toolkit (how many IDA Pro Usrs do you know in your community...?), and replace it with a "binary representation" -- we were forced to take this step -

because we wanted to give you 10ms faster pageturns. And this really was the ideal way of achieving that.

BULLSH*T

In a world where processors and memory (at least tepidly) get faster every year, and this fuels entire industies - the argument, that .kfx had to be created to service that need is and entirely false one.

Its in fact like saying "everyone had to be moved to the Spotify model - because silence in between two tracks could only be made smaller, going with the streaming model" - its missing the entire point, motive, drive, business model (#walledgarden), of an industry - looking at a small detail - and mistaking it for an "evolution" simply because it is new.

I don't care if the .kfx format has some tangible advantages for Amazon ("if it reduces CPU load, we could make our batteries even smaller!") - as long as we use that as justification for having lost the ability to understand the format.

(The loss is structural - because amazon insisted, that this would be the year where they should be the only ones that would be allowed to create their primary book format. Other people that had stakes in creating books in the past were just presented with a "service" to take over that chore fore them - they were basically bought out. Books have finally become "private goods" again.)

I don't - even for a second - believe that Amazons reason for doing so was to ultimately fractualy reduce CPU load on the Kindles.

Oh, the HUGHE CPU load of rendering a plain new text page every 20 seconds - contrasted with the performance specs you have to reach to make the UI navigateable - at all. Or buy books on the device within the store interface.

Books didn't have to become (scrambled) binaries, because of CPU load.

Thats a fairy tale.

Or to use the more technical term - a false dichotomy. A fools choice.