[Conversion Plugin] KFX Input

[BetterRed]

Originally Posted by Ma'am-I-Am

. . .

Any tips on settings/conversion options, etc.?

I tend to leave options that control conversion etc at factory settings, I never apply 'book' specific options. If I can't get a presentable EPUB from a PDF using Word and calibre I leave it be. If they're fixed-layout PDFs I don't even bother trying. Most of the PDF's I do convert are produced by technical writers/documentation professionals using Word or LO Writer.

I used to use the long defunct MobiPocket Creator tool, but it won't install on Windows 10, and I'm not minded to unravel the msi. But it was never as good as PDF->Word (docx)->calibre (epub) is today.

If you're dealing with OCR scans and you have a recent edition of MS Word I suggest you have a look at the Epub Tools add-on for Word ==>> https://www.mobileread.com/forums/sh...d.php?t=213372. It has a range of features that help deal with the vagaries of OCR scans - including a more than decent dialogue checker. And it can produce a well formed EPUB, including mapping of Word Styles to HTML CSS.

BR

[Ma'am-I-Am]

I'll definitely take a look at that.
I sometimes download scanned-to-PDF books at archive.org. They are usually huge and take a relatively long time to load. Is there a fix for those things, i.e reduce file size and load quicker?

[DNSB]

Originally Posted by Ma'am-I-Am

I'll definitely take a look at that.
I sometimes download scanned-to-PDF books at archive.org. They are usually huge and take a relatively long time to load. Is there a fix for those things, i.e reduce file size and load quicker?

Not really any way. All the archive.org .PDF files I've looked at have been scanned images of the pages plus a very poorly OCRed text layer to allow searching.

The images make the files large but their quality is not the greatest so when I tried running various OCR packages on the images, the results needed way too much work to be useful.

[jhowell]

Version 2.0.0 - 01 Jun 2023

Allow conversion of Kindle Scribe notebooks. See the plugin documentation for instructions. (This is a work in progress. Please report any incorrect rendering or errors encountered in the KFX Input plugin thread.)

Handle the PDF backed fixed layout variant of KFX format.

Improve handling of margins expressed as a percentage in cases where the width of the element is unknown.

Remove support for calibre versions prior to 5.0. (Older versions of this plugin that run under calibre 2.x through 4.x are available in this post.)

[tomsem]

Looks good!

I tried one of my notebooks and Sticky Notes for one of my books.

- Apple Books expects a cover image and result is it truncates the first page of the notebook.
- calibre ebook-viewer (and ebook-edit) are applying a theme (dark white etc) and they probably should not be. So if they are set to dark theme you cannot see the black strokes. I think these just don't handle FXL rendering correctly?

Only suggestion so far is to fill in some placeholder metadata for title/author (e.g. "{filename} - {creation date}" , "KFX input plugin v{version}", and maybe include a simple cover for it with these.

Would make it easier, for example, to bring into calibre if someone wants to manage them there.

[jhowell]

Originally Posted by tomsem

Apple Books expects a cover image and result is it truncates the first page of the notebook.

I will add an option to the plugin to create a dummy cover page for notebooks. (Without that the calibre commands to replace an existing cover will not work properly.)

Originally Posted by tomsem

calibre ebook-viewer (and ebook-edit) are applying a theme (dark white etc) and they probably should not be. So if they are set to dark theme you cannot see the black strokes. I think these just don't handle FXL rendering correctly?

I will add a white background to notebook pages. That should fix this problem.

Calibre does not handle fixed layout EPUBs well, however the EPUBs created from notebooks are simple enough that they should work in calibre.

Originally Posted by tomsem

Only suggestion so far is to fill in some placeholder metadata for title/author (e.g. "{filename} - {creation date}" , "KFX input plugin v{version}", and maybe include a simple cover for it with these.

Unfortunately the file name is not readily available at the point where the title and author are set. For now I will change it to use the title "Notebook" followed by the 23-character internal book ID.

You can always change the title, author, and cover of the EPUB to whatever you want using the calibre CLI:

ebook-meta notebookname.epub --authors="New Author" --title="New Title" --cover=newcover.jpg

[tomsem]

Originally Posted by jhowell

I will add an option to the plugin to create a dummy cover page for notebooks. (Without that the calibre commands to replace an existing cover will not work properly.)



I will add a white background to notebook pages. That should fix this problem.

Calibre does not handle fixed layout EPUBs well, however the EPUBs created from notebooks are simple enough that they should work in calibre.



Unfortunately the file name is not readily available at the point where the title and author are set. For now I will change it to use the title "Notebook" followed by the 23-character internal book ID.

You can always change the title, author, and cover of the EPUB to whatever you want using the calibre CLI:

ebook-meta notebookname.epub --authors="New Author" --title="New Title" --cover=newcover.jpg

Great, thank you!

The placeholder title should work, but would be inclined to add timestamp as well so if imported to calibre there won't be duplicate warnings. (e.g. someone wants to store versions of a notebook in calibre)

Time allowing I'm planning to work on a script to help manage the steps, and orchestrate KFX Input and calibre CLI calls. Gathering requirements, use cases.

Eventually perhaps this becomes some Scribe Notebook manager plugin, if there's sufficient interest.

[jhowell]

Originally Posted by tomsem

The placeholder title should work, but would be inclined to add timestamp as well so if imported to calibre there won't be duplicate warnings. (e.g. someone wants to store versions of a notebook in calibre).

I will add the date and time to the placeholder title.

[George96]

Dear all!
I have an issue using the KFX-Input Plugin. "Issue" means it simply doesn't work.

I think I followed the (brilliant and detailled - thx for that!) instructions in this threat, but it simply doesn't work. :-( It always ends in a conversion error....

Where do I enter the serial-number of the device? I didn't find that field to enter in Calibre...

Once I added that, it might be working. Let's see...

Thanks for your help in advance!
Regards George.

[jhowell]

Originally Posted by George96

I have an issue using the KFX-Input Plugin. "Issue" means it simply doesn't work.

I think I followed the (brilliant and detailled - thx for that!) instructions in this threat, but it simply doesn't work. :-( It always ends in a conversion error....

What specific error message are you receiving from the plugin?

Originally Posted by George96

Where do I enter the serial-number of the device? I didn't find that field to enter in Calibre...

Once I added that, it might be working. Let's see...

The KFX Input plugin can only perform conversion once DRM removal has been accomplished. The serial number is need by the DeDRM plugin in order to remove DRM from books that are copied from a Kindle device. Detailed assistance with DRM removal is not allowed here on MobileRead. There is a FAQ and you can ask for help with that on GitHub.

You should also read the "DRM Concerns" section in the first post of this thread. If your Kindle is running a firmware version greater than 5.12.1 (which is true of most Kindles) then the DeDRM plugin will not be able to remove KFX DRM. If that is the case for you then see this thread for alternative methods of obtaining Kindle books in a format compatible with DeDRM.

[jhowell]

Version 2.1.0 - 18 Jun 2023

Miscellaneous improvements in the conversion of Scribe notebooks.

Handle Scribe notebooks containing multiple page templates. (Only one will be active.)

Add a white background to Scribe notebook pages. This should allow them to render better in dark mode.

Change the title of EPUBs created from Scribe notebooks from "Unknown" to "Notebook" followed by the internal book ID and current date. The calibre ebook-meta command can be used to change the title or author if something else is desired.

Add an option to the plugin command line interface to create a generic EPUB cover page showing the title of the book if the book does not already have a cover.

[tomsem]

Finally had a chance to try 2.1.0. Getting this error when trying to convert nbk (renamed to 'test.kpf)':

(base) ➜  test /Applications/calibre.app/Contents/MacOS/calibre-debug -r "KFX Input" -- test.kpf
Building icons.rcc
Software versions: KFX Input 2.1.0, calibre 6.22*, macOS-13.4.1-x86_64-i386-64bit
KFX Input plugin help is available at https://www.mobileread.com/forums/showthread.php?t=291290

Processing test.kpf
Processing container: test.kpf
Traceback (most recent call last):
  File "runpy.py", line 196, in _run_module_as_main
  File "runpy.py", line 86, in _run_code
  File "site.py", line 41, in <module>
  File "site.py", line 37, in main
  File "calibre/debug.py", line 282, in main
  File "calibre_plugins.kfx_input.__init__", line 194, in cli_main
  File "calibre_plugins.kfx_input.kfxlib.yj_book", line 211, in decode_book
  File "calibre_plugins.kfx_input.kfxlib.kpf_container", line 51, in deserialize
  File "calibre_plugins.kfx_input.kfxlib.utilities", line 586, in as_ZipFile
  File "zipfile.py", line 1258, in __init__
  File "zipfile.py", line 1325, in _RealGetContents
zipfile.BadZipFile: File is not a zip file
(base) ➜  test

No doubt I'm doing something wrong.

[jhowell]

Originally Posted by tomsem

Finally had a chance to try 2.1.0. Getting this error when trying to convert nbk (renamed to 'test.kpf)':

No doubt I'm doing something wrong.

You need to pass the name of the notebook folder to the KFX Input plugin. See the section "Converting Scribe notebooks" for details.

(KPF is a variant of KFX format in a ZIP file. That is different from the format used by Scribe notebooks which is why it fails.)

[tomsem]

Originally Posted by jhowell

You need to pass the name of the notebook folder to the KFX Input plugin. See the section "Converting Scribe notebooks" for details.

(KPF is a variant of KFX format in a ZIP file. That is different from the format used by Scribe notebooks which is why it fails.)

Perfect, thank you!

FYI I am seeing issues with a number of notebooks: some show warnings/errors and generate .ePub, others error out before generating one. Will supply details after I collect examples.

[tomsem]

Failures:

• Key Error
• Insufficient data

Success, with errors/warnings ('cosmetic'):

• Book has short symbol format without document_data max_id (virtually every !!EBOK!! & !!PDOC!! nbk) - I think most of these are 'empty' e.g. no pen annotations
• Missing sections
• repeats previous location

Unfortunately I no longer have the PDOC corresponding to the missing_sections nbk (it was orphaned somehow when I deleted the KFX-PR document). I'll see if I can find another example.