[Conversion Output] KFX Output

[innocenat]

Originally Posted by jhowell

Thanks for supplying the log. There is no problem with posting that here since you are not asking for assistance with DRM removal.

It appears that the Kindle Previewer is emitting an empty mime type field for the cover image which is something that I have not encountered before. It may be a difference with how it works under MacOS vs. Windows. I will change the KFX Output plugin so that it will not fail in this case.

The change will be in the next plugin release which I plan to have out sometime in the next week. Hopefully that will resolve the problem.

Thank you.

So I was investigating some more, and found that if I took the immediate epub generated during the coonversion by this plugin, and convert them manually using Kindle Previewer 3 and the plugin CLI, the error still showed up. But surprisingly, the KFX was also generated despite the error. I checked the KFX against the original epub and it seemed that nothing is really missing.

[jhowell]

Originally Posted by innocenat

So I was investigating some more, and found that if I took the immediate epub generated during the coonversion by this plugin, and convert them manually using Kindle Previewer 3 and the plugin CLI, the error still showed up.

If I could get a copy of the EPUB that produces the error that would be of great help to me in nailing down the root cause of the problem.

If the book is not under copyright you can post it here if you want. Or if you do not want to post it you could email it to me.

[jhowell]

Originally Posted by innocenat

So I was investigating some more, and found that if I took the immediate epub generated during the coonversion by this plugin, and convert them manually using Kindle Previewer 3 and the plugin CLI, the error still showed up.

Thanks to your help I was able to replicate the problem and determine the cause.

When performing conversion calibre normally replaces the cover image in the book with the one from the calibre library. Sometimes, depending on the HTML coding of the book, this goes wrong and results in the converted book have two cover pages. If the extra leftover cover uses an SVG wrapper around the image then a bug in the Kindle Previewer causes it to emit the image with an incorrect empty mime type.

The next release of the KFX Output plugin will check for this condition and fix the mime type in the KFX file produced.

[jhowell]

Version 1.59.0 - 15 Feb 2022

Detect Kindle Previewer versions up to 3.61.0.

Fix the empty mime type for an image sometimes produced by the Kindle Previewer when a book contains a duplicate cover. (Fixes "ERROR: External resource e6 has unknown mime type".)

[jhowell]

Amazon has released Kindle Previewer version 3.62.0. It is compatible with the current release of this plugin.

[shamanNS]

@jhowell: I'm not quite sure who (you or @NiLuJe) and which of the 2 forum topics would be more appropriate person & place to ask for help with trying to troubleshoot a bug regarding KFX files generated with this KFX Output plugin + @NiLuJe's ScreenSaver hack used in "Cover mode" (= "book cover as a screensaver" thing).

The problem: only for KFX books generated via Calibre using KFX Output addon ( = for couple of KFX books downloaded from Kindle Store the correct image is extracted and used as a screensaver by SS hack) the SS hack doesn't pick / extract & use correct image as a screensaver image (the cover image, that was correctly marked as such in input epub file) . Instead it picks a random? (it not using the first or last or Nth file as ordered inside epub's manifest) image contained inside the book file and usually it is quite small, like something used for chapter headings or separators / scene breaks.

The sideloaded KFX files were created on Windows 10 x64 (build 21H1) using:

• latest version (v.1.59.0) of KFX output addon
• Calibre v5.37 64-bit
• Kindle Previewer 3.61.0

I'm using latest version (v0.25N @r18833 on 2021-Nov-15 @00:37) of ScreenSaver hack from snapshots on a Kindle Paperwhite 3 with firmware version v5.9.7.

Just in case here are link for 2 book conversion logs but I doubt they will be of any use since "Show full Kindle Previewer conversion logs" option wasn't selected at the time of those conversions.

https://pastebin.com/jK1dLDQJ

https://pastebin.com/6bhvfgaf

[jhowell]

Originally Posted by shamanNS

The problem: only for KFX books generated via Calibre using KFX Output addon ... the SS hack doesn't pick / extract & use correct image as a screensaver image ... . Instead it picks a random?

That is a known problem. See this post.

[shamanNS]

I would say that even though many years have passed since 2016. you are still the most knowledgeable person regarding KFX internals and in best position to figure out a simple fix
Could you please try to fix that cover extraction problem?

I tried reading screensaver hack source code and comparing its version of kfxmeta.py with version (newer ?) used by ExtractCoverThumbs script and the only semi-useful thing I've managed to check (by pointing the GUI app to folder containing required subfolders, same as on actual Kindle drive: documents & system > thumbnails) is that ExtractCoverThumbs uses correct image to create thumbnail file for the same kfx file created by this Calibre plugin that results in incorrect image extraction by SS hack version and that just adding " 424: b"cover_image" " to SYMBOL_TABLE and using that key instead of 414 (bcRawMedia) doesn't fix the problem but causes "TypeError: Incorrect padding" when script calls " base64.b64decode(data[cover_key])".

[jhowell]

Originally Posted by shamanNS

ExtractCoverThumbs uses correct image to create thumbnail file for the same kfx file created by this Calibre plugin that results in incorrect image extraction by SS hack version

ExtractCoverThumbs uses a newer and more functional version of the kfxmeta.py script. It is able to locate the cover image within a KFX file containing multiple images. It is not 100% correct but far better than the older version that the SS hack uses.

Originally Posted by shamanNS

just adding " 424: b"cover_image" " to SYMBOL_TABLE and using that key instead of 414 (bcRawMedia) doesn't fix the problem but causes "TypeError: Incorrect padding" when script calls " base64.b64decode(data[cover_key])".

The version of kfxmeta used in the screensaver hack was changed quite a lot from what I originally produced. I assume that was done for compatibility with the Kindle environment. Updating it to use the logic from ExtractCoverThumbs will take a lot more than a simple cut and paste. There were a LOT of changes added in the version used by ExtractCoverThumbs in order to make extraction of the correct image happen.

Originally Posted by shamanNS

I would say that even though many years have passed since 2016. you are still the most knowledgeable person regarding KFX internals and in best position to figure out a simple fix
Could you please try to fix that cover extraction problem?

I am not going to undertake that effort since I am not familiar with the Python environment used, do not have any jailbroken devices on which to test the results, and my Kindle already displays cover images for KFX files on the sleep screen.

[shamanNS]

Fair enough.
But as far as "python environment" is concerned both versions of kfxmeta.py run just fine on my Windows PC (when you just call that particular .py file and "feed" it a kfx file and -c argument) and the end result (extracted image) is visible and accessible without the need for a jailbroken Kindle.

[jhowell]

Originally Posted by shamanNS

Fair enough.
But as far as "python environment" is concerned both versions of kfxmeta.py run just fine on my Windows PC (when you just call that particular .py file and "feed" it a kfx file and -c argument) and the end result (extracted image) is visible and accessible without the need for a jailbroken Kindle.

There can be issues related to Python 2 vs Python 3 and the libraries available under different platforms. I am unfamiliar with the environment provided by the SS hack on a jailbroken Kindle. I do not know the reasons for the changes that were made to my original program and what further changes would be needed to make the newer code work properly.

[shamanNS]

Yay, it works

I've managed to edit "extract_cover_thumbs.py" (from "ExtractCoverThumbs" app) code to process only files with .kfx extensions and dump images with correct filename (cover_ASIN.png) to needed location (/mnt/us/linkss/cover_cache) folder and to save them as PNG and resize even smaller cover images to 1072 x 1448. Luckily ScreenSaver hack doesn't redo cover extraction if it finds already cached file for that book.

And I've created KUAL extension to run that script

[jhowell]

Originally Posted by shamanNS

Yay, it works

Great!

[stefano.sb]

Hi, I used this plug-in to convert some MOBI ebooks to KFX and to send them via USB to my Kindle Paperwhite (fw 5.14.2) . It works very well!! I use also the method 1 described in the first post to send the correct cover thumbnails.

A little suggestion... Is it possible to avoid the step in which I have to eject the device, wait a moment until the cover thumbnail for the book has been replaced by a generic cover and then I have to reconnect the kindle to Calibre in order to fix the cover thumbnail??

A solution could be one in which Calibre:

• checks if the Kindle is in airplane mode. If yes it alerts the user to connect it to wifi. If it's not possibile to do this check it would be useful a simple message...
• sends the ebook via USB
• ejects the device "via software"
• waits that Amazon replaces the cover by a generic cover
• reconnects the device (I don't know if it's possible...).
• fixs the cover thumbnail
• produces a little "bip" or a message to inform the user thar the process is complete

Stefano

[jhowell]

Originally Posted by stefano.sb

A little suggestion... Is it possible to avoid the step in which I have to eject the device, wait a moment until the cover thumbnail for the book has been replaced by a generic cover and then I have to reconnect the kindle to Calibre in order to fix the cover thumbnail??

The issue with cover thumbnails on Kindles also applies to AZW3 format and the "fix" for it is managed completely by calibre, not by this plugin. I suggest that you either start a new thread for your proposal or issue an enhancement request on the calibre bug tracker.