[Conversion Plugin] KFX Input

[pdurrant]

Originally Posted by jhowell

I am traveling right now but I will look into it when I get back. It would be very helpful to know the ASIN of the book.

I suspect it's this one:

https://www.amazon.co.uk/dp/B07DF9CLGB/

Which is the Penguin Classics Kindle edition.

[HarryT]

Originally Posted by jhowell

I am traveling right now but I will look into it when I get back. It would be very helpful to know the ASIN of the book.

Much appreciated - thank you. The ASIN is B07DF9CLGB.

[HarryT]

Originally Posted by pdurrant

I suspect it's this one:

https://www.amazon.co.uk/dp/B07DF9CLGB/

Which is the Penguin Classics Kindle edition.

Yes, it is. Thanks, Paul.

[Quoth]

The "house full of random swimming birds" more often needs their ebooks fixed than any others I buy, inc SP titles.

[bustacap]

Originally Posted by JSWolf

On your PW5, do you see the images with the same problem with the KFX version?

fully zoomed in, on dark pages in 2-page view (so i can better see the white line), i cant tell its there, both with the book directly downloaded on the kindle and the kfx downloaded on pc and copied there with calibre.

it only really becomes noticeable when converted from the pdf based kfx to epub (or cbz). my guess is that that sub pixel line gets "amplified" by whatever pdf conversion method this plugin uses (maybe a different cropping box than "artbox" or "cropbox" or it just behaves differently than mutool).

another look zoomed in 6400% into the pdf page, see the tiny white line at the right and bottom (which are likely part of the original, its just that they get more pronounced when converted to epub/cbz):
https://s1.directupload.eu/images/250725/fboj3g6k.png

[Jaws]

Originally Posted by HarryT

Much appreciated - thank you. The ASIN is B07DF9CLGB.

Jumping to that ASIN showed a possible source of the problem: It's a UK-origin book. Which means that in the printed edition, the US page sizes (and often ledding) are different — and due to the flightless bird's process, that's going to present problems with doing a conversion-from-a-conversion. My suspicion, which I can't prove but is consistent with another publisher's problems, is that their internal stylesheets accessed from their typesetting programs have hardcoded metric measurements that get converted to hardcoded "English" measurements for US editions (and ebooks "fulfilled to the US"), even though not visible, and then things go haywire at random deep in the decimal places...

[HarryT]

Originally Posted by Jaws

Jumping to that ASIN showed a possible source of the problem: It's a UK-origin book. Which means that in the printed edition, the US page sizes (and often ledding) are different — and due to the flightless bird's process, that's going to present problems with doing a conversion-from-a-conversion. My suspicion, which I can't prove but is consistent with another publisher's problems, is that their internal stylesheets accessed from their typesetting programs have hardcoded metric measurements that get converted to hardcoded "English" measurements for US editions (and ebooks "fulfilled to the US"), even though not visible, and then things go haywire at random deep in the decimal places...

It’s a British book. I’m in the UK. I bought the book from Amazon UK. I don’t understand why anything involving US measurements would be involved? I collect the Penguin Classics series and over the years I’ve bought well over a hundred books in the range. I’ve never had any issue with one prior to this.

[jhowell]

Originally Posted by bustacap

is there a way to fix these 1-2 pixel borders on the right and bottom edge when converting from kfx to epub using the from kfx menu?

Originally Posted by bustacap

all images in several comics and manga by different (german) publishers. the azw3 versions of the same titles (when they were still available) didnt have this and the versions on the webreader dont have this either (not that i can tell anyway).
the from kfx log says theyre pdf. using mutool (tested with both artbox and cropbox) on the pdf doesnt leave that pixel edge. looking at the pdf, there seems to be a tiny sub pixel (<1/10th maybe) line

The KFX Input plugin uses calibre's built-in "page_images" PDF to JPEG conversion routine which in turn uses pdftoppm included with calibre.

I have looked at several books available to me in KFX format that use PDF as the internal image format. So far I have not seen the problem you describe in the JPEG files produced when converting those to EPUB using From KFX.

In order to research this further I would need access to a book that exhibits the problem. Posting copyrighted material is not allowed here on MobileRead but if you PM me a link to somewhere I can download one the those KFX files I will look into it.

[jhowell]

Originally Posted by HarryT

I’ve bought the book “The Nichomachean Ethics” by Aristotle (Penguin Classics). The book displays correctly in the Kindle app on my iPad. I imported the book successfully into Calibre using the KFX Import plugin. Attempting to convert to ePub fails with numerous lines similar to:

calibre, version 8.7.0
ERROR: KFX conversion failed: <p><b>Cannot convert The Nicomachean Ethics (Penguin Classics)</b><br/><br/>Exception('box-align right with float right in /cKS.xhtml\nbox-align right with float right in /cKS.xhtml...

Since that book is available in the US I was able to examine it to see what is going wrong with the conversion from KFX.

The problem is in how the numbers placed to the right of the text throughout the book are handled. The publisher used unusual HTML formatting to achieve this. It is rendered properly in KF8/AZW3 format. However Amazon's conversion of the book to KFX alters the formatting, causing the numbers to become embedded within the text instead of being kept separate.

The next release of this plugin will no longer produce an error when that unexpected formatting is encountered. Unfortunately conversion will still result in incorrect placement of the numbers, matching how the book renders in KFX.

[jhowell]

Originally Posted by bustacap

is there a way to fix these 1-2 pixel borders on the right and bottom edge when converting from kfx to epub using the from kfx menu?

I compared the images produced from the book you supplied with one that works properly for me. I could not find anything significant in the corresponding PDF files except that your book uses significantly higher resolution images. At this point all I can say is that pdftoppm, which is used to perform PDF page to image conversion, does a better job on my book than on yours.

I did notice that the pages in both your book in mine are actually JPEG images encapsulated in PDF. I will look into changing the plugin to detect this situation and if so simply extract the JPEG. That would result in the best image quality possible.

I am not sure when I will have time to do that. I will be busy with other things for at least the next week.

[bustacap]

Originally Posted by jhowell

I compared the images produced from the book you supplied with one that works properly for me. I could not find anything significant in the corresponding PDF files except that your book uses significantly higher resolution images. At this point all I can say is that pdftoppm, which is used to perform PDF page to image conversion, does a better job on my book than on yours.

I did notice that the pages in both your book in mine are actually JPEG images encapsulated in PDF. I will look into changing the plugin to detect this situation and if so simply extract the JPEG. That would result in the best image quality possible.

I am not sure when I will have time to do that. I will be busy with other things for at least the next week.

thank you for checking it out and finding a way to fix the issues i was having

[kennyc]

Thank you so much for this @Jhowell !!
I just today gave this a try in order to do the end-around to Amazon's killing book downloads. Worked like a charm once I realized I had to import into Calibre directly from the Kindle in question (rather than copying the file over and then importing).

Plan to try it on a new purchase shortly.

Thank You!!

[jhowell]

Originally Posted by kennyc

Plan to try it on a new purchase shortly.

Thank You!!

Best to do it as you purchase books. You never can tell when Amazon will decide to create new roadblocks.

You’re welcome!

[kennyc]

Originally Posted by jhowell

Best to do it as you purchase books. You never can tell when Amazon will decide to create new roadblocks.

You’re welcome!

Yes, understand....that has been my methodology from the beginning.

[tomsem]

Having a couple of issues with this 'flavor' of KFX, e.g.:

https://www.amazon.com/gp/product/B0DJSQKJG5/

These evidently were Fixed Layout EPUBs that the publisher submitted to Kindle store, and they get converted to fixed layout KFX when downloaded to Kindles or the Kindle apps. These are sorta 'coffee table books- whose print editions would be larger than letter size pages, with 2 page spreads.

Unlike comic book format, they contain 'positioned' text in the XHTML files. On ePub platforms, you're usually able to search, and fonts are embedded so that the text remains crisp when zooming in.

On Kindle platform (at least books from this publisher) the embedded fonts are gone, hyperlinks and text search are not supported (even though Kindles will happily spend time indexing them). But they are defined in the downloaded file.

In the past, I would download AZW3 and convert to fixed layout ePub with KindleUnpack. (I can still do this using my Kindle Touch, even though it's impossible to actually read the book on it!). These do contain the positioned text and hyperlinks of the original ePub.

But with KFX, From KFX conversion to ePub (or PDF) currently mishandles the text and hyperlinks.

In the ePub, the text is put inside a <div> tag's alt= property, the positioning is lost, and can no longer be target of text search. Hyperlinks are not instantiated.

In PDF, I can see text objects with Acrobat object inspection but they are not placed over the word images they correspond to, and there are PDF links but they do not work (see attached screenshot).

So the information is in there, but is lost in translation. It would be nice if it was not, but I would rank it a low priority on your feature backlog, given this is kind of an edge case.

----

In the future I will think more about purchasing these from Kobo. I did so in this case ($1.99 today).

The ePub version has embedded fonts, so it will look better when viewed in some ePub app, and also it can be searchable (but seems this is not supported on Kobo platform).

However converting for use on Kindle platform is proving onerous. Send To Kindle does not have good support for fixed layout ePub, in this case it totally mangled it.

Calibre got to 47% and seems to be stuck there. I finally killed the job after 3 hours.

Kindle Previewer puked on it, without even producing error log.

calibre Edit reports a number of problems, but Apple Books and Thorium render it without any problems. My PocketBook InkPad C 3 has some serious problems with it, though. I added this from my Kobo Clara where it gets converted from KePub to ePub - I also tried ACSM method to download ePub, but results were the same (calibre quickly gets to 47% with conversion to AZW3 and then gets stuck; PocketBook doesn't render it correctly).