[Conversion Plugin] KFX Input

[jhowell]

KFX Conversion Input Plugin

The KFX Input plugin allows books in Amazon KFX format to be imported and converted to other e-book formats, such as EPUB, for archival and for reading on alternate devices. The plugin can convert from several KFX variants: Monolithic KFX, KFX-ZIP, or KPF.

Amazon uses KFX format for the delivery of most e-books to customers. It supports advanced features, including Enhanced Typesetting and Page Flip, on Amazon apps and devices. More information about the KFX format can be found in the KFX Format thread and the MobileRead Wiki (KFX).

Amazon's KF8 (azw3) format is superior to KFX as a source for conversion since it more closely matches the original format provided to Amazon by the publisher. This plugin has limited usefulness as long as KF8 is still easily obtainable. Some cases where KF8 is not available are Arabic, Indian and Traditional Chinese language books/dictionaries, books produced using Kindle Create, Kindle in Motion books with animation, Interactive Textbooks with A/V media, and some newer eTextbooks. In most other cases KFX can be avoided. See the thread Dealing with Kindle for PC/Mac 1.19 and KFX in calibre for details.

Books in KFX format delivered by Amazon are a bundle of files composed of an encrypted main container, a metadata container, auxiliary containers (zero or more) and a DRM voucher. Those each contain only part of the book's content and cannot be converted individually. KFX container files typically have a ".kfx" extension, though other extensions (".azw", ".azw8", and ".azw9") are used by some software.

This plugin will combine the individual files that make up a KFX book into a single file (monolithic KFX format) when a KFX book is successfully imported into calibre. See "Importing KFX books into calibre" below for instructions. Monolithic KFX can also be produced using the KFX Output plugin. Monolithic KFX can be converted to other formats using this plugin.

DRM removal for KFX format is only possible in specific situations, limiting the usefulness of this plugin. See "DRM Concerns" below for more information.

Amazon's Kindle Textbook Creator and Kindle Create software export books in KPF (Kindle Package Format), a variant of KFX, for publishing on Amazon. This plugin allows conversion from KPF to other e-book formats. (Note: Starting with version 1.70.0.0, Kindle Create has the ability to export reflowable books in EPUB format, eliminating the need to convert from KPF format in order to obtain a usable copy of the finished book.)

Starting with calibre version 4.12, calibre will extract the embedded Word document from KPF files that contain one and import the file as DOCX instead of KPF. This DOCX file will not contain any edits made using Kindle Create. If import of the actual KPF file is desired instead you will need to disable the KPF Extract file type plugin by going to Preferences, Plugins, and clicking the arrow to expand the File Type category. Highlight the plugin named KPF Extract and press the Enable/disable button so that it is grayed out.


Installation:

Install the KFX Input plugin and restart calibre as described in the Introduction to plugins thread.


Configuration:

Spoiler:

Configuration of the plugin is done in the KFX Input tab in the conversion options dialog. Defaults for future conversions can be set under Preferences, Input Options, KFX Input.

In addition "tweaks" can be configured to set less frequently used options that apply to all conversions. Set them by going to Preferences->Advanced->Tweaks->Plugin tweaks. Tweaks are entered as Python variable assignments. See the calibre manual for more information.

DRM Concerns:

Spoiler:

Books delivered by Amazon in KFX format always contain DRM (Digital Rights Management) which encrypts the contents and prevents conversion by this plugin. Even books that the publisher has specified should not use DRM will be encrypted, although in that case the book will not be locked to a single device.

Conversion of books from KFX format is only possible if the DRM is removed by using the DeDRM Tools calibre plugin. See the thread New version of DRM removal tools ("DeDRM") for the latest information on obtaining that plugin.

KFX DRM is actively maintained and updated by Amazon to keep it secure. Each time that a solution for KFX DRM has been found Amazon has quickly changed to a new KFX DRM variant in order to block it. The differs from the DRM used with the older Kindle formats which has remained unchanged for many years.

Even with the DeDRM plugin installed, DRM removal for KFX format is only possible in specific limited situations. It is only possible for books downloaded using versions 1.19 through 1.26 of the Kindle for PC/Mac apps and versions 5.6.5 through 5.12.1 of e-ink Kindle device firmware. It will not work for books downloaded using other versions of that software or downloaded using the Kindle for iOS or Android apps.

A book importing into calibre as KFX-ZIP format is usually an indication that DRM removal has failed. If you encounter problems with DRM removal ask for help at the site where you obtained the DRM removal plugin. Assistance with DRM removal will not be provided on the MobileRead forums.

Importing KFX books into calibre for conversion:

Spoiler:

Note: The following import procedures will only work if the software version of the Kindle device or app from which the book is being imported is supported by the DRM removal plugin.

KFX books can be imported from an e-ink Kindle using the calibre "Device" view. The plugin tweak "kfx_input_set_format_virtual" must be set to False to enable this type of import. This can be done by going to Preferences->Advanced->Tweaks->Plugin tweaks and entering:

kfx_input_set_format_virtual = False

As an alternative, KFX books can be manually imported into calibre using the following procedure. First, the main KFX file for the book to be imported must be located:

• For Kindle for PC/Mac version 1.19 or greater the "My Kindle Content" folder will contain a sub-folder for each book, named with the ten character ASIN of the book followed by "_EBOK". The ASIN for each book is shown in the "Product details" of the book's Amazon web page. The main file for the book will have an ".azw" extension, regardless of whether or not it is in KFX format. When locating books it may be helpful to sort "My Kindle Content" by date so that the newest book sub-folders appear first.
  
  The location of the "My Kindle Content" folder can be found from within the Kindle for PC program by selecting the menu choices Tools, Options..., Content.
  
• On e-ink Kindle devices connected via USB, the main file for each book is contained in the "documents" folder or sometimes in the "Downloads" subfolder of that folder. These files are named with both the title of the book and the ASIN. Books in KFX format will have a ".kfx" extension.
  
• No workable KFX import method is currently available for Fire tablets or for the Kindle for Android and iOS apps.

Once located, the main file can be imported to calibre using "Add Books" function ("Add books from a single directory") or using drag-and-drop. (The main book file must be in its original folder structure. For example, books from Kindle for PC should be in a subfolder of the "My Kindle Content" folder.)

Be patient. The import process is complex and may take a minute or more to complete for a large book.

If successful the result will appear as KFX format in the calibre library. This can be sideloaded to Kindle apps & devices, viewed using Kindle Previewer 3, or converted to other e-book formats using this plugin.

If the resulting format in calibre is KFX-ZIP or AZW instead of KFX then the import was unsuccessful. If this occurs make sure that you have both the KFX Input plugin and the DeDRM plugin installed, that you restarted calibre after installation, and that the main book file was imported from its original location.

The reason for import failure can be obtained from a log file. Click the calibre Preferences drop-down menu and choose "Restart in debug mode". Re-import the book and then close calibre. A log file will appear.

For help with DRM removal problems raise an issue at the site where you obtained the DRM removal plugin. Assistance with DRM removal will not be provided on the MobileRead forums.


Automatic adding of books:

The "Automatic adding" function of calibre, which adds any book files placed in a specific directory, cannot be used with books in KFX format because each book in composed of multiple files. You must manually add the main file of the book from its original folder location so that the KFX Input plugin can gather all of the files that make up the book.

Alternatively, you can create a ZIP file containing all of the files that make up the book, change its extension from ".zip" to ".kfx-zip", and place that into the folder configured for automatic adding to calibre.


Import - Behind the scenes:

Calibre handles all e-book formats as a single file, however books in KFX format are usually composed of multiple KFX container files. When the main file of a book in KFX format is imported in calibre a File Type Plugin contained within the KFX Input plugin will automatically locate the rest of the files that make up that book and gather them into a KFX-ZIP archive (a ZIP file with the extension ".kfx-zip".)

As an alternative a KFX-ZIP archive can be created manually. To do this locate the files/folders containing the book's contents, combine them into a ZIP file, change the extension of the ZIP file from .zip to .kfx-zip, and import that into calibre instead of the main book file.

If the DRM removal plugin with KFX support is properly installed and configured it will process the KFX-ZIP and remove its DRM.

The KFX-ZIP archive will then be processed again by the KFX Input plugin to package the content into a monolithic KFX file. (This will not be possible if DRM removal was unsuccessful, resulting in KFX-ZIP being seen as the imported books format.)

Viewing books in KFX format:

Spoiler:

To view a KFX book within calibre right click the book entry in the calibre library and from the menu choose "View" and then "View with calibre E-book viewer". This will be slow since the book will be converted from KFX to EPUB each time it is viewed this way. One alternative is convert to EPUB first and then view that.

Another way to view KFX is using Kindle Previewer 3. See "Open KFX with Kindle Previewer 3".

Converting from KFX or KPF to other formats:

Spoiler:

The KFX Input plugin is activated by selecting either (Monolithic) KFX, KFX-ZIP, or KPF as the input format when converting books in calibre.

By default calibre will reduce the resolution of images during conversion. In the "Page setup" tab the "Output profile" may be set to "Tablet" in order to avoid this.

A command line interface is also available for this plugin. (See below.)

Converting Print Replica Books:

Spoiler:

Print replica books contain the original PDF content that was used to produce the book. PDF pages are rendered as JPEG images if the book is converted to EPUB. A PDF file can be extracted instead using the following procedure:

• Import the book into calibre and verify that the imported format is KFX. (KFX-ZIP format indicates a failure.)
• Use the "Save to disk", "Save only KFX format to disk in a single directory" to export the book from calibre.
• Open a command/terminal window in your OS and change to the folder where the KFX file was saved.
• Use "--pdf" option of the KFX Input plugin CLI to extract PDF content. Eg:
  calibre-debug -r "KFX Input" -- --pdf "my book file.kfx"

Links between pages, such as in the Table of Contents, may not not be present in the extracted PDF file.

Generating content/position information:

Spoiler:

In Kindle e-book formats "position" numbers are used to refer to content within a book. They are used in Kindle software to indicate the start and end of annotated text, link to text for the X-ray and Word Wise features, synchronize the last page read across devices, and for other similar purposes.

In MOBI and KF8 formats position numbers are byte offsets within the raw HTML content of the book. However KFX format is not based on HTML. For KFX the position numbers are instead offsets of Unicode characters within text strings.

The "--json-content" option of the plugin command line interface can be used to produce a JSON file containing the information needed to associate KFX position numbers with the text and images that they refer to. This is intended to aid in the creation of WordWise and X-Ray files for books in KFX format and could be useful to determine the text of annotations.

This JSON file uses a similar format to that produced by Amazon's MobiContentDumper program. The "data" field holds a list of content chunks ordered by position.

For chunks representing text:

• "type" is 1.
• "content" is a Unicode string with some text from the book. In addition to separation by spaces and punctuation, words also start and end on string boundaries.
• "position" is the position number associated with the first character of the string. Each subsequent character is at the next highest position number. Each Unicode character occupies one position even if it is encoded by multiple bytes. There may be gaps in position numbering between strings.

For chunks representing an image:

• "type" is 2.
• "content" is the resource name of the image. The same image resource may appear multiple times within a book.
• "position" is the position number associated with that image. Each image occupies one position.

Other "type" values are reserved for future use and should be ignored.

Dealing with conversion errors:

Spoiler:

When importing a book from Kindle for PC/Mac the original file name will have an ".azw" extension. If the import process fails it can result in the book showing AZW format within your calibre library. calibre will treat that as being in a different Kindle e-book format than KFX, leading the error "Failed to convert: (name) This is an Amazon KFX book. It cannot be processed...." when attempting to convert it to another format.

This plugin will fail to convert files that contain unexpected data, such as new or uncommon KFX features. It will also fail for books that use a fixed layout, such as comics, children's books, and print replica books. Use the KFX Input CLI for those books instead.

You can force conversion to proceed despite these errors by selecting the option "Allow conversion to complete even if errors are detected" in the KFX Input tab of the conversion dialog. However, in that case the results of conversion may be incomplete or incorrect.

If the option to ignore errors is selected or if the results of a conversion appear to be incorrect, it is recommended that the conversion job log be checked for errors and warnings. Please report problems in this MobileRead thread.

You can view the conversion job log after an error occurs to see additional information on why the conversion failed. To access the log, either click on the "Jobs: 0" label in the lower right corner of the calibre window or use the Alt-Shift-J keyboard shortcut to activate the Jobs dialog. Select the failed conversion job from the list (the most recent will be first) and click the "Show job details" button. To copy the log to the clipboard under Windows, click within the log text then press Ctrl-A (select all) and Ctrl-C (copy). You can then paste it wherever you like.

Reading book metadata:

Spoiler:

The KFX Input plugin also contains a Metadata Reader plugin, which will extract book metadata and cover images on import.

It is also activated by the "Set metadata for the book from the selected format" and "Set the cover for the book from the selected format" buttons in the single Edit Metadata dialog. ("Set metadata from the e-book files" and "Change cover - Set from e-book files" in bulk Edit Metadata.)

The supported fields are: title, author, language, publisher, publication date, description, cover, and "mobi-asin" identifier (ASIN).

Command Line Interface:

Spoiler:

The KFX Input plugin command line interface (CLI) bypasses the calibre conversion pipeline and can result in an EPUB that more closely matches the formatting of the original book. Conversion using this method is required for books with special formatting, such as fixed layout comics, children's books, and print replica books.

usage: calibre-debug -r "KFX Input" -- [-h] [-e] [-2] [-p] [-u] [-j]
                                       infile [outfile]

Convert KFX e-book to EPUB or extract its resources

positional arguments:
  infile              Pathname of the .azw8, .kfx, .kfx-zip, or .kpf file to
                      be processed
  outfile             Optional pathname of the resulting .epub, .pdf, or .zip
                      file

optional arguments:
  -h, --help          show this help message and exit
  -e, --epub          Convert to EPUB (default action)
  -2, --epub2         Convert to EPUB 2 instead of EPUB 3
  -p, --pdf           Extract PDF from print replica, create PDF from comics & children
  -u, --unpack        Create a ZIP file with extracted resources
  -j, --json-content  Create a JSON content/position file

Documentation for calibre-debug can be found here. The full path for the command may be needed under MacOS: /Applications/calibre.app/Contents/MacOS/calibre-debug

The default action is to convert to EPUB version 3. If the --epub2 option is selected the result will still be EPUB version 3 if required by the book's content.

Instead of converting to EPUB, the resources contained within a book can be extracted as a ZIP archive. The archive will also contain a file named "book.ion" with the content of the book in ion text format.

The "book.ion" file can be produced containing readable names, instead of numeric symbols, by use of an optional symbol table. The plugin will make use of a catalog file named "kfx_symbol_catalog.ion" containing the definition of the YJ_symbols shared symbol table if it is present in the calibre plugin configuration folder. (This file is not included with the plugin.)

Limitations:

Spoiler:

The KFX format is undocumented and has changed over time. Books containing new or unknown features may fail to convert or result in incorrect content or formatting.

KFX is more highly processed than Amazon's KF8 (azw3) format, causing some details in the markup of the original e-book provided by the publisher to be lost during conversion. The exact HTML tags and styles originally used in the book are not retained in KFX format. However, all of the text and most of the formatting should be preserved. Images may have been reduced in resolution or converted to black and white to match the capabilities of the device to which they were delivered.

PDF resources in KFX print replica books may have been split into multiple files. Links between pages across files will be broken if the PDF content is extracted.

These limitations make KFX less suitable for archival and conversion when compared with KF8.

This plugin only supports KFX files without DRM. Removal of DRM is not within the scope of this plugin.

Some specialized variants of KFX cannot be converted using this plugin, including Kindle in Motion and dictionaries.

This plugin does not contain a copy of the KFX symbol table. Unpacked book content and error messages may refer to KFX symbols in numeric form (such as $123). Interpreting these is not within the scope of this plugin, however the plugin can make use of a symbol table if provided by the user.

This plugin requires calibre version 2.0 or later. Version 2.74 or later is required for automatic import of KFX format. Version 2.80 or later is required to import books in KFX format from the device view. Version 3.24 or later is required for correct metadata when importing KFX books using the "calibredb add" command.

Version History:

Spoiler:

Version 1.50.0 - 16 Nov 2022

Produce semantic element tags for paragraphs (<p>), headings (<h1>, <h2>, <h3>), and figures (<figure>) when possible.

Allow conversion of fixed-layout KFX children's books using the plugin CLI.

Use high resolution tiled images if available when using the CLI to produce PDFs from comics.

Version 1.49.0 - 30 May 2022

Change the content/position information collected by this plugin to replace footnote references with spaces to better support the WordDumb plugin.

Fix occasionally missing text in the content/position information collected by this plugin when inline images are present.

Allow combining palette mode tiles into a single image.

Version 1.48.0 - 17 Apr 2022

Allow conversion of of books that use KFX Illustrated Layout such as Amazon Original Stories and Kindle in Motion books. (Fixes "KeyError('$176')" and "book fragment used multiple times: $259")

Allow conversion of table cells that contain only a horizontal rule element. (Fixes "unexpected hr found in table_row")

Support additional KFX capability (yj_fixed_layout-2) and features (yj_arabic_fixed_format-1 and yj_direction_rtl-1).

Retain the original image mode when combining grayscale tiles into a single image.

Version 1.47.0 - 15 Feb 2022

Support new KFX features. (Fixes "Unknown com.amazon.yjconversion feature: yj_mixed_writing_mode-2")

Prevent unused table thumbnail images from being included in the output EPUB when a book contains large tables.

Do not produce an empty CSS rule when a class has no styling.

Version 1.46.0 - 20 Dec 2021

Change the format of the file produced by the "--json-content" CLI option to include the position and resource names of images. See "Generating content/position information" in the plugin documentation for more information.

Support new KFX features. (Fixes "Unknown com.amazon.yjconversion feature: yj_table-11" and "Unexpected Ion symbols used: $736, $741, $742, $821, $822, $823, $824, $825.")

Version 1.45.0 - 08 Nov 2021

Handle SVG wrapped images in comics.

Version 1.44.0 - 04 Oct 2021

Fix optimization that was incorrectly removing nested div elements in some rare cases.

Version 1.43.0 - 20 Sep 2021

Support new KFX features. (Fixes "Unknown com.amazon.yjconversion feature: reflow-style-14" and "Unexpected Ion symbols used: $790, $791, $792, $793, $794, $795, $796.")

Version 1.42.0 - 25 Aug 2021

Support new KFX feature, tiled images greater than 1920 pixels high or wide (yj_hdv-2).

Support new KFX feature, text-shadow in books with vertical text (yj_vertical_text_shadow-1).

Version 1.41.0 - 06 Aug 2021

Support new KFX feature, float and clear in vertical Japanese books (jpvertical-reflow-language-7).

Support KFX feature, writing-mode:vertical-lr.

Simplify CSS by consolidating heritable properties that were split across child elements during conversion to KFX.

Version 1.40.0 - 14 Jun 2021

Support new KFX feature, vertical tables in horizontal books and vice versa (yj_table-10).

Fix forced left alignment of text not being retained during conversion.

Version 1.39.0 - 23 Apr 2021

Support new KFX feature. (Fixes "Unexpected Ion symbols used: $788" and "Unexpected Ion symbols used: $789")

Version 1.38.0 - 14 Apr 2021

Fix occasional incorrect image centering.

Internal kfxlib API change to return asset_id metadata.

Version 1.37.0 - 07 Mar 2021

Include DOCTYPE declaration when producing EPUB 2 output using the plugin command line interface.

Support nested tables and lists in KFX format.

Version 1.36.0 - 18 Nov 2020

Add an option to the plugin command line interface, "--json-content", that will produce a JSON file containing a mapping between the book's textual content and a numbering scheme that refers to it. See "Generating content/position information" in the plugin documentation for more information.

Version 1.35.0 - 23 Oct 2020

Support additional KFX features. (Fixes "Unexpected Ion symbols used: $432", "Unknown com.amazon.yjconversion feature: reflow-style-13", and "Unknown com.amazon.yjconversion feature: yj_mixed_writing_mode-1")

Version 1.34.0 - 12 Sep 2020

By default books that use a fixed layout, such as comics and print replica books, will no longer convert within calibre. The option to ignore errors can be used to force conversion, but will yield poor results. Use the KFX Input CLI for those books instead.

Add option to plugin CLI to extract PDF resources from KFX print replica books and to create a PDF file from KFX comic books.

Fix scaling of pixels and handle rotated pages when print replica to fixed-layout EPUB. (Fixes "Unexpected Ion symbols used: $549" and "Unexpected style definition: -webkit-transform")

Support additional KFX features. (Fixes "Unexpected Ion symbols used: $765, $766")

Internal code restructuring.

Version 1.33.0 - 02 Jul 2020

Handle recent changes to KFX format as produced by Kindle Previewer version 3.41.

Version 1.32.0 - 12 Jun 2020

Support additional KFX features. (Fixes "Unexpected style definition: float: $786")

Fix non-unique HTML ids occasionally generated for books having non-numeric page numbers.

Version 1.31.0 - 06 Mar 2020

Support additional KFX features. (Fixes "Unexpected style definition: page-break-after: $383" and "Unexpected style definition: page-break-before: $383")

Prevent failure if a book contains duplicated fonts. (Fixes "YJFragmentList get has multiple matches for...$418")

Version 1.30.0 - 25 Feb 2020

Support additional KFX features. (Fixes "Unexpected Ion symbols used: $134", "Unexpected style definition: page-break-after: avoid", "Unexpected style definition: page-break-before: avoid")

Version 1.29.0 - 13 Feb 2020

Support additional KFX features. (Fixes "Unexpected Ion symbols used: $73")

Version 1.28.0 - 30 Jan 2020

Support additional KFX features. (Fixes "KeyError(u'$758',)")

Ignore extraneous data in KPF files produced by Kindle Create version 1.35. (Fixes "$269 has extra data: (yj.semantics.page_entity:...")

Version 1.27.0 - 10 Jan 2020

Support additional KFX features. (Fixes "Invalid local symbol created: crop_bleed" and "Unexpected Ion symbols used: $35")

Fix handling of very small property values. (Fixes "Unexpected style definition: margin-top: -3.97364E-8lh")

Version 1.26.0 - 23 Dec 2019

Compatibility fixes for calibre beta versions using Python 3.

Version 1.25.0 - 07 Dec 2019

Support additional KFX features. (Fixes "Unexpected Ion symbols used: $46", "Conflicting margin properties: ...", and "Property margin-xxx has multiple values: ...")

Fix occasional error converting books with a mix of horizontal and vertical writing modes. (Fixes "Incorrect div content for SVG wrapper:...")

Convert line separator characters into line breaks.

Version 1.24.0 - 16 Nov 2019

Support additional KFX features. (Fixes "Unexpected Ion symbols used: $784" and "Unexpected style definition: outline-style: $330")

Version 1.23.0 - 12 Nov 2019

Support new local symbol format produced by Kindle Previewer version 3.35.

Fix occasional error converting books containing widows/orphans properties. (Fixes "TypeError('%%d format: a number is required, not unicode',)")

Version 1.22.0 - 03 Nov 2019

Ignore extraneous data in KPF files produced by Kindle Create version 1.32. (Fixes "Referenced fragment is missing from book" and "$538 has extra data: (yj.print.settings...")

Support additional KFX features. (Fixes "Unexpected Ion symbols used: $64, $762, $763" and "Unexpected ruby element type")

Version 1.21.0 - 18 Oct 2019

Support additional KFX features. (Fixes "Unexpected Ion symbols used: $131, $132, $785...")

Version 1.20.0 - 21 Sep 2019

Support new KFX local symbol format. (Fixes "$538 has extra data: (max_id: ...)")

Version 1.19.0 - 18 Sep 2019

Support additional KFX feature. (Fixes "Unexpected style definition: box-shadow:...")

Version 1.18.0 - 20 Aug 2019

Replace "--epubversion" command line option with "--epub2". The default has been changed to produce EPUB version 3.

Prevent failure to convert books that contain duplicated resources. (Fixes "KeyError(u'YJFragmentList get has multiple matches for resource...")

Prevent failure to convert some books with vertical-align in table cells (Fixes "Conflicting div -kfx-vertical-align and vertical-align in same style")

Support additional KFX features. (Fixes "Unknown com.amazon.yjconversion feature: yj_audio-2")

Internal restructuring.

Version 1.17.1 - 10 Jun 2019

Fix incorrect handling of units leading to conversion failure for some books. (Fixes "TypeError(\"unsupported operand type(s) for *: 'NoneType' and 'Decimal'\",)")

Version 1.17.0 - 03 Jun 2019

Support additional KFX features. (Fixes "Unexpected Ion symbols used: $780, $782. Unexpected style definition: line-break: strict")

In addition to the webkit properties supported by Amazon also emit the equivalent standard properties when possible.

Emit an SVG image wrapper for cover pages. (Only relevant when using the command line interface or when the "Use cover from source file" conversion option is selected.)

Internal changes in preparation for the planned switch in calibre from Python 2 to Python 3.

Version 1.16.0 - 19 Apr 2019

Handle images containing links. (Fixes "story ... content type $271 has extra data: {$179: ...}")

Fix link colors sometimes being applied to the wrong element.

Version 1.15.0 - 16 Apr 2019

Handle transparent color found in some books. (Fixes "Unexpected style definition: color: $349")

Handle new local symbol format used by Kindle Create version 1.24. (Fixes "Invalid local symbol created: ...")

Handle special case of drop cap sometimes produced by Kindle Create. (Fixes "story xxx has dropcap_chars 1 with dropcap_lines 0")

Improve conversion of complex floated content.

Improve conversion of dimensions presented as points in KFX that were originally specified as pixels.

Improve determination of default font and language.

Use KFX "layout hints" to provide semantic information in some class names.

Version 1.14.0 - 08 Mar 2019

Improve handling of font names and fallbacks for the default font of a book. (Fixes poor rendering of some books in calibre under Windows.)

Improve conversion when the book metadata specifies a language code that is less specific than that found in the book content. For example language "en" vs. "en-US".

In the command line interface the EPUBVERSION value must now be either "2" or "3". Values "2.0" and "3.0" are no longer accepted. Choosing 3 attempts to produce an EPUB that conforms to the preliminary EPUB 3.2 specification.

Version 1.13.0 - 02 Feb 2019

Handle unexpected data found in some print replica textbooks. (Fixes "TypeError: %d format: a number is required, not IonStruct")

Ignore unexpected control characters found in some print replica textbooks.

Support KPF files produced by Kindle Create version 1.20. (Fixes "$538 has extra data: ('yj.semantics.page_number_begin': ...")

Support additional KFX features. (reflow-language-2, reflow-language-expansion-1, reflow-style-12, Ion symbols $136 $137)

Version 1.12.1 - 30 Dec 2018

Support additional KFX feature. (Fixes "Unexpected Ion symbols used: $282")

Version 1.12.0 - 20 Dec 2018

Change the default value of the "kfx_input_set_format_virtual" plugin tweak to False so that by default calibre will not import books in KFX format from e-ink Kindle devices. (Books in KFX format can no longer be imported from the latest Kindle firmware, version 5.10.2.)

Fix incorrect font size used for drop caps in some cases.

Allow combinations of text-decoration line values.

Support additional KFX features. (Fixes "ERROR: Unexpected Ion symbols used: $135, $353", "ERROR: unexpected style events in $270", and "ERROR: Unknown com.amazon.yjconversion feature: yj_mathml-1".)

Version 1.11.0 - 08 Dec 2018

Add plugin tweak "kfx_input_set_format_virtual" that when set to True will prevent import of books in KFX format from e-ink Kindle devices in calibre "Device" view.

Support additional KFX features. (Fixes "ERROR: Unknown com.amazon.yjconversion feature: yj_table-8", "ERROR: Unexpected Ion symbols used: $292, $293, $749, $750, $751, $752", and "ERROR: Unexpected Ion symbols used: $760, $761".)

Ignore extraneous data produced by Kindle Previewer version 3.28.1.

Version 1.10.0 - 11 Nov 2018

Allow books in KFX format to be imported from a Kindle using the device view. Calibre (version 2.80 or higher) will no longer treat KFX as a "virtual format" when this plugin is installed.

Improve handling of properties that combine several others in an equivalent shorthand. (Fixes "ERROR: Setting conflicting property: border-top-style...".)

Support additional KFX features. (Fixes "ERROR: Unknown com.amazon.yjconversion feature: reflow-style-11".)

Version 1.9.0 - 11 Oct 2018

Fix extra text-indent properties from being produced for some books.

Fix incorrect placement of "colspan" in table cells containing only an image. (Fixes "ERROR: Unexpected class_attribute in img: colspan")

Fix uninformative error details provided if a conversion error occurs while attempting to use the calibre viewer on a book in KFX format.

Improve handling of long path names when importing books under Windows.

Support additional KFX features. (Fixes "ERROR: Unknown com.amazon.yjconversion feature: reflow-style-8".)

Version 1.8.0 - 30 Aug 2018

Prevent occasional failures when converting some books from e-ink Kindles that contain problematic JPEG-XR images.

Support additional KFX features produced by Kindle Create version 1.12.

Version 1.7.0 - 13 Aug 2018

Fix incorrect image type sometimes being detected for books that were published containing SVG images.

Prevent conversion failure caused by unexpected content in some KFX books. (Fixes "Unexpected box-align value: justify")

Prevent conversion failure caused by alignment of lists in KFX books. (Fixes "Unexpected box-align found in ol element")

Prevent conversion failure caused by drop caps with zero length, sometimes produced by Kindle Create.

Fix incompatibility with calibre versions before 3.0 introduced in plugin version 1.2.0. (Fixes "AttributeError: 'apsw.Cursor' object has no attribute 'fetchone'" when converting from KPF format.)

Support additional KFX features.

Version 1.6.1 - 20 Jul 2018

Prevent conversion failure when a book contains an unexpected symbol count.

Version 1.6.0 - 20 Jul 2018

When converting to EPUB report in the log if the book contains any PDF resources. This can occur for print replica books.

Allow book content and resources to be unpacked and extracted as a ZIP archive using the command line interface.

Fix bug introduced in version 1.4.0 that causes incorrect styles to be produced for drop caps.

Support additional KFX features.

Version 1.5.0 - 25 Jun 2018

Create KFX-ZIP format during import even if additional files that make up the book cannot be located. This prevents potential confusion when books from Kindle for PC/Mac in KFX format fail to import, previously resulting in AZW format.

Improve error messages associated with unsuccessful import of books in KFX format.

Support books containing unicode characters with code points outside of the basic multilingual plane, such as emoji.

Retain original image file names when converting KPF files produced by Kindle Create.

Prevent conversion failure when unexpected quoting of font names is encountered. Fixes conversion error: "Incorrectly quoted font name: 'arial"

Prevent conversion failure when the OS platform on which the plugin is running cannot be properly determined.

Version 1.4.0 - 21 May 2018

Fix error when converting KPF files produced by Kindle Create version 1.6: CorruptError: malformed database schema (?)

Do not include extra metadata in OPF that could lead to duplicate cover pages when converting comics.

Support additional KFX features.

Version 1.3.0 - 09 Apr 2018

Fix error during conversion of books with referenced fonts that are not present: "'unicode' object has no attribute 'list_symbols'"

Support additional KFX features.

Version 1.2.0 - 02 Apr 2018

Added an option to proceed with conversion even if unknown or incorrect content is detected.

Handle additional features of KPF format produced by Kindle Create.

Fix KFX-ZIP archives not being created for e-book samples.

Fix debug message logging when creating KFX-ZIP archives.

Internal restructuring and performance improvements.

Version 1.1.0 - 31 Jan 2018

Added a built-in File Type Plugin to automatically gather the component files of a book and produce KFX-ZIP format during book import. This eliminates the need for the user to produce KFX-ZIP files manually, simplifying the process of importing KFX books into calibre. (DRM removal must still be accomplished by other means, prior to import.)

Added another built-in File Type Plugin to combine the components of a KFX-ZIP into monolithic KFX format. This process will only succeed if all of the components of the book are present in the KFX-ZIP and do not contain DRM. This makes it easier to determine whether on not the import was successful since a successful import will result in KFX format and an unsuccessful one will result in KFX-ZIP format.

Added a Metadata Reader Plugin to extract KFX book metadata. This allows the title, author, and other metadata to appear in the calibre book entry when importing a KFX book.

Ignore approximate page numbers created by the KFX Output plugin.

Handle changes to KPF format produced by Kindle Create 1.3.30.0 and Kindle Previewer 3.17.1.

Fix KPF format not being accepted for conversion using the GUI interface.

Allow conversion of additional KFX features.

Improve detection of unsupported KFX features.

Version 1.0.1 - 16 Oct 2017

Remove extraneous files that were inadvertently included in the initial release.

Version 1.0.0 - 16 Oct 2017

Initial release.

[jhowell]

Reserved post

[odamizu]

Awesome! Thank you, jhowell, for all your work unraveling KFX and making it more accessible to all!

[jhowell]

Originally Posted by odamizu

Awesome! Thank you, jhowell, for all your work unraveling KFX and making it more accessible to all!

You're welcome.

I know that this plugin isn't especially useful right now given the continued availability of KF8 and the lack of automatic DRM removal for KFX. But I thought it would be best to be prepared in case this capability is needed in the future.

[odamizu]

Originally Posted by jhowell

... I thought it would be best to be prepared in case this capability is needed in the future.

Absolutely! Hopefully KF8 and EPUB will continue to be available and accessible for a long time to come, but it gives me great comfort knowing your plug-in (and the Pascal decrypter) exists, just in case. for all your time and effort.

[Terisa de morgan]

Thank you very much, perhaps some of the things which you say they limit its usefulness will disappear in a near future.

[mbovenka]

Originally Posted by odamizu

Absolutely! Hopefully KF8 and EPUB will continue to be available and accessible for a long time to come, but it gives me great comfort knowing your plug-in (and the Pascal decrypter) exists, just in case. for all your time and effort.

+1. Not that I expect EPUB to go anywhere, as Amazon has no control over it, but KF8 might.

[odamizu]

Originally Posted by mbovenka

+1. Not that I expect EPUB to go anywhere, as Amazon has no control over it, but KF8 might.

Amazon isn't the only one we have to worry about. Adobe has their hardened DRM. So far ADE 2.0.1 still works for most Adobe epubs, but that could change.

[pdurrant]

Thank you for this most impressive bit of work.

[Iris-maybe]

I do not think this is unuseful right now since Epubor has update this software to remove Kindle KFX DRM yesterday. Unfortunately, they can not convert KFX. This is really what I need. Thanks for your contribution.

[JSWolf]

epubor is a rip-off of Calibre (overall).

[odamizu]

Originally Posted by JSWolf

epubor is a rip-off of Calibre (overall).

And epubor will likely soon rip-off jhowell's awesome KFX conversion plug-in

[JSWolf]

Originally Posted by odamizu

And epubor will likely soon rip-off jhowell's awesome KFX conversion plug-in

I would not be surprised if they do.

[issybird]

The rest of this conversation has been moved to a new thread here, per request. Please continue the conversation along those lines there; this thread is for the support of the KFX Input plugin only.

[luisdent]

I have decent luck so far, but one book "inferno" by dan brown won't convert to kfx. It seems to be any image files from what I can tell. If I removed the first graphic symbol on one of the pages, the next time i convert it errors on a further page... Any way to keep the graphic elements and still convert?