[Conversion Plugin] KFX Input

[jhowell]

Originally Posted by tomsem

These evidently were Fixed Layout EPUBs that the publisher submitted to Kindle store, and they get converted to fixed layout KFX when downloaded to Kindles or the Kindle apps. These are sorta 'coffee table books- whose print editions would be larger than letter size pages, with 2 page spreads.

I will take a look at the book you mentioned when I have time, but I doubt that I will be able to do much to improve how it converts. My goal is to make the EPUB produced by this plugin reflect whatever is in the KFX file as much as possible. If that KFX file has mangled formatting then it is beyond my scope of work to fix it, except in rare cases where there is an obvious and simple way to do it.

Originally Posted by tomsem

However converting for use on Kindle platform is proving onerous. Send To Kindle does not have good support for fixed layout ePub, in this case it totally mangled it.

Calibre got to 47% and seems to be stuck there. I finally killed the job after 3 hours.

Kindle Previewer puked on it, without even producing error log.

Sounds like a case of garbage in/garbage out. Amazon only supports a very specific subset of fixed layout as (poorly) documented in their Kindle Publishing Guidelines.

[tomsem]

Originally Posted by jhowell

I will take a look at the book you mentioned when I have time, but I doubt that I will be able to do much to improve how it converts. My goal is to make the EPUB produced by this plugin reflect whatever is in the KFX file as much as possible. If that KFX file has mangled formatting then it is beyond my scope of work to fix it, except in rare cases where there is an obvious and simple way to do it.



Sounds like a case of garbage in/garbage out. Amazon only supports a very specific subset of fixed layout as (poorly) documented in their Kindle Publishing Guidelines.

At least From KFX generates error free (according to calibre) ePub, and it renders in Apple Books and Thorium (PocketBook still won't but even image only fixed layout is not well supported).

That said, calibre doesn't convert it to a functioning AZW3 (and never said that it would, but at least it does not get stuck generating something) and Kindle Previewer still thinks there's something wrong with it (without giving any details).

As you suggest, it's possible the conversion to KFX does this mangling -- after all KF8 fixed layout never supported text search or hyperlinks or text selection, so why carry forward all the position data and make sure hyperlinks work?

Even in ePub universe fixed layout is like an unwanted stepchild.

Years after it was specified, they are only now getting around to addressing ePub Fixed Layout Accessibility:

https://www.w3.org/news/2024/group-n...accessibility/

https://epubsecrets.com/the-accessiv...ble-comics.php

There seems no hope that ePub platforms much less publishers will do anything to implement these practices. Much less Amazon doing so.

[Jaws]

Originally Posted by HarryT

It’s a British book. I’m in the UK. I bought the book from Amazon UK. I don’t understand why anything involving US measurements would be involved? I collect the Penguin Classics series and over the years I’ve bought well over a hundred books in the range. I’ve never had any issue with one prior to this.

However, Amazon's internal processes and programs are all "US-centric," so even though you bought it in the UK the bits were looked at from a US perspective at some point. I've had repeated problems with books with large tables — UK editions purchased through 'zon.uk — that traced back to "I foolishly tried to work from a Kindle edition instead of getting an epub in the first place."

[jhowell]

Originally Posted by tomsem

Having a couple of issues with this 'flavor' of KFX, e.g.:
https://www.amazon.com/gp/product/B0DJSQKJG5/

Since I am currently working on improving the handling of page images when converting Print Replica books I took a look at the book you mentioned.

Originally Posted by tomsem

These evidently were Fixed Layout EPUBs that the publisher submitted to Kindle store, and they get converted to fixed layout KFX when downloaded to Kindles or the Kindle apps. These are sorta 'coffee table books- whose print editions would be larger than letter size pages, with 2 page spreads.

Unlike comic book format, they contain 'positioned' text in the XHTML files. On ePub platforms, you're usually able to search, and fonts are embedded so that the text remains crisp when zooming in.

On Kindle platform (at least books from this publisher) the embedded fonts are gone, hyperlinks and text search are not supported (even though Kindles will happily spend time indexing them). But they are defined in the downloaded file.

In the past, I would download AZW3 and convert to fixed layout ePub with KindleUnpack. (I can still do this using my Kindle Touch, even though it's impossible to actually read the book on it!). These do contain the positioned text and hyperlinks of the original ePub.

Examining the book in KF8 format indicates that it was published to Kindle as a comic and sourced with a fixed-layout EPUB. Each page of the book is composed of a 1607x1920 JPEG background image placed on a fixed-layout 1611x1924 page with an overlay of invisible text.

In that book the background images have everything the reader sees so the invisible text was possibly intended for annotation, dictionary lookup, and to provide links to other pages in the table of contents.

Originally Posted by tomsem

But with KFX, From KFX conversion to ePub (or PDF) currently mishandles the text and hyperlinks.

In the ePub, the text is put inside a <div> tag's alt= property, the positioning is lost, and can no longer be target of text search. Hyperlinks are not instantiated.

The conversion by Amazon of that book to KFX format resulted in significant differences.

The background image for each page becomes a recompressed 1608x1920 JPEG image encapsulated within a single page PDF file.

The invisible text from each page is only provided as alt-text to the background image with no links or formatting retained.

Originally Posted by tomsem

In PDF, I can see text objects with Acrobat object inspection but they are not placed over the word images they correspond to, and there are PDF links but they do not work (see attached screenshot).

The PDF pages corresponding to the book's table of contents contain unused, non-functional link annotations with destinations URIs such as: file:///opt/amazon/tmp/c8a7ae45-421b-456b-86d1-761a8cfa14cb-3e976d4c-db0f-47f5-a9d4-dc167f1fd903/f2d4b40f-762f-41f0-995d-a3f20de639da/extractedEpub/OEBPS/page_000010.xhtml

Trying to turn them into something useful would be hit-or-miss so I am not going to attempt it.

Originally Posted by tomsem

So the information is in there, but is lost in translation. It would be nice if it was not, but I would rank it a low priority on your feature backlog, given this is kind of an edge case.

The next release of the KFX Input plugin will obtain slightly higher quality background images from this book and those like it when converting to EPUB or CBZ, but it will still be less than the image quality of the same book in KF8 format with an AZW6 HD image container.

That is the best I can do. The conversion to KFX by Amazon does not leave enough information to properly reconstruct the overlay text and links from the original EPUB that was provided by the publisher.

[Jaws]

Why does what the Grand Sorceror just described sound like "Amazon is trying to do DJVU but proprietarily," especially for the first book described? We've seen this before (remember the origin of .mobi?)...

[tomsem]

Originally Posted by jhowell

Examining the book in KF8 format indicates that it was published to Kindle as a comic and sourced with a fixed-layout EPUB. Each page of the book is composed of a 1607x1920 JPEG background image placed on a fixed-layout 1611x1924 page with an overlay of invisible text.

In that book the background images have everything the reader sees so the invisible text was possibly intended for annotation, dictionary lookup, and to provide links to other pages in the table of contents.



The conversion by Amazon of that book to KFX format resulted in significant differences.

The background image for each page becomes a recompressed 1608x1920 JPEG image encapsulated within a single page PDF file.

The invisible text from each page is only provided as alt-text to the background image with no links or formatting retained.



The PDF pages corresponding to the book's table of contents contain unused, non-functional link annotations with destinations URIs such as: file:///opt/amazon/tmp/c8a7ae45-421b-456b-86d1-761a8cfa14cb-3e976d4c-db0f-47f5-a9d4-dc167f1fd903/f2d4b40f-762f-41f0-995d-a3f20de639da/extractedEpub/OEBPS/page_000010.xhtml

Trying to turn them into something useful would be hit-or-miss so I am not going to attempt it.



The next release of the KFX Input plugin will obtain slightly higher quality background images from this book and those like it when converting to EPUB or CBZ, but it will still be less than the image quality of the same book in KF8 format with an AZW6 HD image container.

That is the best I can do. The conversion to KFX by Amazon does not leave enough information to properly reconstruct the overlay text and links from the original EPUB that was provided by the publisher.

Thanks!

The conversion from KFX to PDF is still useful enough: I can OCR the PDF to add text objects and fix the links manually. For this type of book, it seems to me, PDF is more functional than KFX or fixed layout ePub, to extent one wants to annotate or search.

There are a couple of GitHub projects that I'm planning to investigate more (both as tools to have around, and maybe creating a calibre plugin...):

- https://github.com/mashu3/epub2pdf/

- https://github.com/aourednik/pdf2epub3fixed

There is a current Humble Bundle offer for some of the DK books, they would in ePub format (from Kobo.com):

https://www.humblebundle.com/books/h...sdk_bookbundle

[HarryT]

Originally Posted by jhowell

Since that book is available in the US I was able to examine it to see what is going wrong with the conversion from KFX.

The problem is in how the numbers placed to the right of the text throughout the book are handled. The publisher used unusual HTML formatting to achieve this. It is rendered properly in KF8/AZW3 format. However Amazon's conversion of the book to KFX alters the formatting, causing the numbers to become embedded within the text instead of being kept separate.

The next release of this plugin will no longer produce an error when that unexpected formatting is encountered. Unfortunately conversion will still result in incorrect placement of the numbers, matching how the book renders in KFX.

Thanks for the update!

[JSWolf]

Originally Posted by HarryT

Thanks for the update!

Welcome back. It's been a long time since we've seen you. Hope all is well.

[jhowell]

Version 2.26.0 - 12 Aug 2025

The "From KFX" toolbar action will now perform conversion using a background job to prevent the user interface from being blocked during conversion. A plugin configuration option controls whether or not a popup message will occur when conversion is complete.

The "From KFX" toolbar action will now allow multiple books to be converted in one action. KFX format from all selected books will be converted to the same chosen format if possible.

Extract page images from PDF when possible when converting Print Replica and Comic books that contain PDF content to either EPUB or CBZ format in order to improve the quality of the result. Otherwise PDF pages are rendered as JPEG images at 300 DPI. (150 DPI was used previously.)

Handle unexpected formatting found in "The Nicomachean Ethics (Penguin Classics)", ASIN B07DF9CLGB. Fixes "box-align right with float right in ...".

[kennyc]

Thank You!!

[bustacap]

Originally Posted by jhowell

Extract page images from PDF when possible when converting Print Replica and Comic books that contain PDF content to either EPUB or CBZ format in order to improve the quality of the result.

thank you for the update but i dont think this part is working quite as intended.
instead of just extracting the images (which should take like a second or two, apart from the epub generation) its re-encoding all of them, which takes quite a bit of time and the file size is smaller than the original.
i tested 3 books that ive confirmed can be just extracted (which then would be around the same size as the source +/- 1mb). instead i got the following with the newest update of the plugin:
book 1: kfx/pdf (88.6mb) -> epub (54.9mb)
book 2: kfx/pdf (287mb) -> epub (81.3mb)
book 3: kfx/pdf (135mb) -> epub (82.6mb)

[JSWolf]

Originally Posted by bustacap

thank you for the update but i dont think this part is working quite as intended.
instead of just extracting the images (which should take like a second or two, apart from the epub generation) its re-encoding all of them, which takes quite a bit of time and the file size is smaller than the original.
i tested 3 books that ive confirmed can be just extracted (which then would be around the same size as the source +/- 1mb). instead i got the following with the newest update of the plugin:
book 1: kfx/pdf (88.6mb) -> epub (54.9mb)
book 2: kfx/pdf (287mb) -> epub (81.3mb)
book 3: kfx/pdf (135mb) -> epub (82.6mb)

And don't forget the mess the ePub will be unless you A/B compare the ePub to the PDF.

[jhowell]

Originally Posted by bustacap

thank you for the update but i dont think this part is working quite as intended.
instead of just extracting the images (which should take like a second or two, apart from the epub generation) its re-encoding all of them, which takes quite a bit of time...

The process is more complex and time consuming than it might first appear. The plugin can quickly determine whether or not a page contains a single image. It takes a lot more work to then decide whether or not the image reflects what is shown on the page. PDF has a large number of ways in which the rendered page might look a lot different from an image it contains.

The plugin renders each PDF page and then checks that the extracted image closely conforms it before that image will be accepted as a substitute. Right now that comparison process is fairly slow. I intend to optimize it in the future.

Originally Posted by bustacap

...and the file size is smaller than the original.

It turns out that the image extraction function in the PDF library I use (pypdf) recompresses the extracted image resulting in lowered image quality. I am working on an improved method that will avoid this.

[jhowell]

Version 2.26.1 - 14 Aug 2025

Fix reduction in image quality that would occur in the previous plugin release when JPEG images are extracted from PDF pages.

[bustacap]

Originally Posted by jhowell

Version 2.26.1 - 14 Aug 2025

Fix reduction in image quality that would occur in the previous plugin release when JPEG images are extracted from PDF pages.

works great now, thank you again! only difference in filesize is the cover for some reason (not saying theres something wrong, just an observation).
its the same size as the cover in the calibre library. do kfx files have a separate cover than the included pdf?