[Conversion Plugin] KFX Input

[madelonw]

Of late I have seen no real updates to the KFX/KFX-ZIP situation. Every Kindle book added to Calibre (I am currently using Calibre 9.2.1, and I update every time an update is announced.) is now in KFX-ZIP format.

I use Calibre, to which I have added a number of columns, for tracking the books I read and especially those I read for review. The onboard reader allows me to open, read, and search books. I can't do any of those things if everything is in an unreadable format.

I am also noticing the new Pages feature cranks away on the KFX-ZIP books until it gives up and just puts 'error' in the column. What a waste of CPU time.

The issue is KFX-ZIP.

The issue is not counting pages, or any other function of Calibre.

I repeat—The issue is KFX-ZIP.

Not everyone who reads this thread is a programmer. I am not a programmer. I do have enough experience with computers to follow instructions on configurations and the like.

A clear explanation of the status of deDRMing KFX-ZIP is what is needed here.

[DNSB]

The current status?

DRM is not removable from the current K4MAC app.

DRM is removable from K4PC (2.7.1 and older) however a lot more hoop jumping is needed. For K4PC 2.8.0 I've never bothered to try since from what I've read, even more hoop jumping is needed


DRM is not removable from a Kindle ereader with a firmware version higher that 5.18.5 which would be the 11th generation and newer devices that have updated.

For Kindles from the 5th generation PW1 and older, you can remove DRM and get a azw3 file or mobi for those that do not support azw3/KF8. However many of the older Kindles did not have WiFi and can't be used to download books so this is pretty much limited to the 4th and 5th generation.

For Kindles from the 6th to 10th generation, you can remove DRM and get a KFX file.

For many people on MobileRead, the answer seems to be making any book purchases that are available from other vendors from those vendors.

Please note that I suspect this is close to the limit of the information you can receive on this topic at MobileRead.

[texasnightowl]

What DNSB said. ^

Plus refer to the Dealing with Kindle ebooks sticky thread: https://www.mobileread.com/forums/sh...d.php?t=361503

[jhowell]

Originally Posted by madelonw

Of late I have seen no real updates to the KFX/KFX-ZIP situation. Every Kindle book added to Calibre (I am currently using Calibre 9.2.1, and I update every time an update is announced.) is now in KFX-ZIP format.

An import of a Kindle book resulting in KFX-ZIP is usually indicative of a DRM removal failure. This plugin does not perform DRM removal. Help with dealing with those failures should be requested at the site where you obtained the DRM removal plugin.

Originally Posted by madelonw

I am also noticing the new Pages feature cranks away on the KFX-ZIP books until it gives up and just puts 'error' in the column. What a waste of CPU time.

I doubt the cost is very high, but if you are concerned you should raise the issue with the developer of calibre.

Originally Posted by madelonw

Not everyone who reads this thread is a programmer. I am not a programmer. I do have enough experience with computers to follow instructions on configurations and the like.

A clear explanation of the status of deDRMing KFX-ZIP is what is needed here.

Unfortunately DRM removal has potential legal ramifications, especially for US residents due to the DMCA. Because of this MobileRead's policy is to not allow detailed instructions or assistance with DRM removal. The sites hosting DRM removal plugins are the places for that.


Some general comments:

DRM removal is a moving target, especially for Kindle books. Amazon is aggressive in updating their DRM in order to block its removal. Over time the process has become more constrained and complex.

Producing DRM removal techniques takes work. The developers of DRM removal software who make it freely available are under no obligation to put in the extra time and effort needed to make the process as simple as possible. Those who want this capability have to meet them part way and make the effort to learn how to do it. Ask for help at the hosting site if you do not understand something.

As an alternative some sites sell DRM removal software. Those are often repackaged versions of free software with a nicer user interface. Because they are commercial products there is often more emphasis on making them easy to use for beginners.

[mascalwer]

Hi,
I have a problem with a book I purchased on Amazon.
I was able to import it to Calibre (kfx), so my understanding is it was DeDRMed without issues.
But now when i try to view it after import, or convert it to any other format, I'm getting:

calibre, version 9.3.1
ERROR: Loading book failed: Failed to open the book at \Calibre Library\Angelo Bobak\Connecting the Data_ Data Integrati (1280)\Connecting the Data_ Data Integ - Angelo Bobak.kfx. Click "Show details" for more info.

Failed to convert book: \Calibre Library\Angelo Bobak\Connecting the Data_ Data Integrati (1280)\Connecting the Data_ Data Integ - Angelo Bobak.kfx with error:
KFX metadata reader (from KFX Input) activated for \Calibre Library\Angelo Bobak\Connecting the Data_ Data Integrati (1280)\Connecting the Data_ Data Integ - Angelo Bobak.kfx
DeDRM v10.0.9: Trying to decrypt Connecting the Data_ Data Integ - Angelo Bobak.kfx
Unknown booktype kfx. Passing back to calibre unchanged
InputFormatPlugin: KFX Input running
on \Calibre Library\Angelo Bobak\Connecting the Data_ Data Integrati (1280)\Connecting the Data_ Data Integ - Angelo Bobak.kfx
Software versions: KFX Input 2.29.0, calibre 9.3.1, Windows-11-10.0.26100-SP0
KFX Input plugin help is available at https://www.mobileread.com/forums/sh...d.php?t=291290
Converting \Calibre Library\Angelo Bobak\Connecting the Data_ Data Integrati (1280)\Connecting the Data_ Data Integ - Angelo Bobak.kfx
Processing container: \Calibre Library\Angelo Bobak\Connecting the Data_ Data Integrati (1280)\Connecting the Data_ Data Integ - Angelo Bobak.kfx
Features: CanonicalFormat-2, kfxgen.pidMapWithOffset-1, kfxgen.positionMaps-2, kfxgen.textBlock-1, reflow-style-3, yj_jpegxr_sd-1, yj_table-6, yj_table_viewer-1
Metadata: ASIN=B00NES7TMM, asset_id=CR!SNCK7807SH0T5511QNJ0ERDW4KBK, author="Bobak, Angelo", book_id=i9Z9z5j-TUmBZryJLT4MeQ0, cde_content_type=EBOK, content_id=B00NES7TMM, cover_image=1166x1458, creator_version=2.15.0, description=..., file_creator=YJConversionTools, is_sample=False, issue_date=2014-09-07, kfxgen=20.12.238.0, language=en, max_id=786, min_kindle_version=5.9.4, override_kindle_font=True, pages=251, publisher="Technics Publications", reading_orders=1, selection=enabled, title="Connecting the Data: Data Integration Techniques for Building an Operational Data Store (ODS)"
Converting book to EPUB 3
first: b'<span>INDE</span>'
last: b'<span>A</span>'
ERROR: section cDU7, $269 15273, story lDUB, $269 15276 style event first and last have different parents: offset 0 len 6: b'<div><span style="-kfx-style-name: sE18; font-family: Arial; font-weight: bold"><img src="image_rsrcE7J.jpg" alt="" id="page_243" style="-kfx-render: inline; -kfx-style-name: sDUG; box-sizing: content-box; height: 3.25875em; width: 3.35491em"/><span>INDE</span></span><span style="-kfx-style-name: sE19; font-family: Arial; font-weight: bold; letter-spacing: -15em">X</span><span>A</span><span>I. See Artificial Intelligence</span></div>'
Failed to run pipe worker with command: from calibre.srv.render_book import viewer_main; viewer_main()
Traceback (most recent call last):
File "calibre_plugins.kfx_input.__init__", line 113, in convert
Exception: section cDU7, $269 15273, story lDUB, $269 15276 style event first and last have different parents: offset 0 len 6: b'<div><span style="-kfx-style-name: sE18; font-family: Arial; font-weight: bold"><img src="image_rsrcE7J.jpg" alt="" id="page_243" style="-kfx-render: inline; -kfx-style-name: sDUG; box-sizing: content-box; height: 3.25875em; width: 3.35491em"/><span>INDE</span></span><span style="-kfx-style-name: sE19; font-family: Arial; font-weight: bold; letter-spacing: -15em">X</span><span>A</span><span>I. See Artificial Intelligence</span></div>'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "runpy.py", line 198, in _run_module_as_main
File "runpy.py", line 88, in _run_code
File "site.py", line 83, in <module>
File "site.py", line 78, in main
File "site.py", line 50, in run_entry_point
File "calibre\utils\ipc\worker.py", line 194, in main
File "<string>", line 1, in <module>
File "calibre\srv\render_book.py", line 828, in viewer_main
File "calibre\srv\render_book.py", line 819, in render_for_viewer
File "calibre\srv\render_book.py", line 796, in render
File "calibre\ebooks\oeb\iterator\book.py", line 64, in extract_book
File "calibre\customize\conversion.py", line 241, in __call__
File "calibre_plugins.kfx_input.__init__", line 119, in convert
calibre.ebooks.conversion.ConversionUserFeedBack: {"msg": "<b>Cannot convert ?</b><br/><br/>Exception('section cDU7, $269 15273, story lDUB, $269 15276 style event first and last have different parents: offset 0 len 6: b\\'&lt;div&gt;&lt;span style=\"-kfx-style-name: sE18; font-family: Arial; font-weight: bold\"&gt;&lt;img src=\"image_rsrcE7J.jpg\" alt=\"\" id=\"page_243\" style=\"-kfx-render: inline; -kfx-style-name: sDUG; box-sizing: content-box; height: 3.25875em; width: 3.35491em\"/&gt;&lt;span&gt;INDE&lt;/span&gt;&lt;/span&gt;&lt;span style=\"-kfx-style-name: sE19; font-family: Arial; font-weight: bold; letter-spacing: -15em\"&gt;X&lt;/span&gt;&lt;span&gt;A&lt;/span&gt;&lt;span&gt;I. See Artificial Intelligence&lt;/span&gt;&lt;/div&gt;\\'')", "level": "error", "det_msg": "", "title": "KFX conversion failed"}

I am able though to force conversion by checking "Allow conversion to proceed even if errors are detected" in KFX Input - thank you for this option. So it's not end of the world for me, but it would be nice for this plugin to be able to convert this book 'cleanly'.

[JSWolf]

delete post

[mascalwer]

JSWolf I would gladly do what you suggest, but the book being in KFX format is not editable in Calibre editor - it can only edit AZW3 or EPUB or KEPUB formats (or am i missing something??

I can edit the epub version I created by selecting 'Allow conversion to proceed even if errors are detected' option while converting from KFX.
So I can fix errors there and thus get the properly formatted epub - which is a valid way for me to get properly formatted epub, thanks.

Still, It would be nice if the pluging could handle this formatting without any additonal actions.

[jhowell]

Originally Posted by mascalwer

I have a problem with a book I purchased on Amazon.
I was able to import it to Calibre (kfx), so my understanding is it was DeDRMed without issues.
But now when i try to view it after import, or convert it to any other format, I'm getting:

calibre, version 9.3.1
ERROR: Loading book failed: Failed to open the book at \Calibre Library\Angelo Bobak\Connecting the Data_ Data Integrati (1280)\Connecting the Data_ Data Integ - Angelo Bobak.kfx. Click "Show details" for more info.

Thank you for the error report.

That error crops up occasionally in books that use a small image as part of the chapter headings, in this case the heading of the Index at the end of the book. In the end it does not affect the result of conversion in any meaningful way.

Originally Posted by mascalwer

I am able though to force conversion by checking "Allow conversion to proceed even if errors are detected" in KFX Input - thank you for this option. So it's not end of the world for me, but it would be nice for this plugin to be able to convert this book 'cleanly'.

That is the best thing to do for now. I will change the plugin to not generate an error in this situation. But the result produced will be the same as what you get now by just ignoring the error.

[mascalwer]

Absolutely correct. Thanks for your quick response.

[ozonepot]

Ok, I got a book DeDRMed and saved in KFX-Zip Format using the command line DeDRM tools method.
Imported successfully to Calibre, it shows as KFX and I can view it on the internal Viewer.
BUT BUT BUT, the book is now all rasterized to images? What was a nice text-searchable and selectable Kindle e-book, is now a book of images that would need text recognition?
And of course a conversion to any format in Calibre will result in a EPUB or PDF file which is still a bunch of images.
Am I missing something? And if I did, which part of the process and how to fix it?
Thanks

[jhowell]

Originally Posted by ozonepot

Ok, I got a book DeDRMed and saved in KFX-Zip Format using the command line DeDRM tools method.
Imported successfully to Calibre, it shows as KFX and I can view it on the internal Viewer.
BUT BUT BUT, the book is now all rasterized to images? What was a nice text-searchable and selectable Kindle e-book, is now a book of images that would need text recognition?
And of course a conversion to any format in Calibre will result in a EPUB or PDF file which is still a bunch of images.
Am I missing something? And if I did, which part of the process and how to fix it?
Thanks

It sounds like your book is probably a Print Replica. You should be able to tell from the Amazon web page for the book. Look for “Format ‏: ‎Print Replica” in the Product Details section.

If so then the best results can be obtained by using the “From KFX” toolbar action to convert it to PDF. See the first post of this thread for more information.

[ozonepot]

Originally Posted by jhowell

It sounds like your book is probably a Print Replica. You should be able to tell from the Amazon web page for the book. Look for “Format ‏: ‎Print Replica” in the Product Details section.

If so then the best results can be obtained by using the “From KFX” toolbar action to convert it to PDF. See the first post of this thread for more information.

Indeed, thanks! I am going to look into that.

[ozonepot]

Well this worked like magic! PDF is now back to text selectable, and I must say one of the best conversion I've seen on calibre, the book looks identical to the amazon version. Thanks!
One question though: why is the EPUB still made of images? The pagination is the same, I thought if PDF can be done this way why not EPUB? And would a conversion from the converted text-selectable PDF to EPUB work to maintain it?

[jhowell]

Originally Posted by ozonepot

Well this worked like magic! PDF is now back to text selectable, and I must say one of the best conversion I've seen on calibre, the book looks identical to the amazon version. Thanks!

Print replica books are actually PDF wrapped inside Kindle formats. This plugin is just extracting that content.

Originally Posted by ozonepot

One question though: why is the EPUB still made of images? The pagination is the same, I thought if PDF can be done this way why not EPUB? And would a conversion from the converted text-selectable PDF to EPUB work to maintain it?

To make an EPUB from a fixed layout book the plugin extracts the pages from the PDF as images. It adds any text present as an invisible overlay on top of each image but that part ends up being ignored by most reading software.

The extraction of PDF is a far better option. You can explore other methods to convert that to a more readable EPUB. That is beyond the scope of this plugin.