[GUI Plugin] K2pdfopt Plugin

[capink]

This plugin used k2pdfopt program to convert PDFs to an optimized version that can be read on ereaders


How the plugin works

• Select book(s) you want to convert and click on the plugin icon in the main toolbar. It should open a dialog with conversion options.
• Choose the options you want and then press preview button to see how it would look on a sample page.
• To convert the book(s), press the convert button.
• Calibre job(s) will be launched to carry out the conversion in the background, after which it will prompt the user to add the converted books to calibre.
• The newly converted PDFs will be added to calibre, depending on the user settings as follows:
  1. [Default option] The converted book is added to the same book record as a PDF format, while the original file is renamed into an ORIGINAL_PDF format.
  2. The converted books is added to a new book record.

Modifying metadata of added books

• When the converted books are added to calibre as a new book record (as opposed to the default option of adding it to the same book record), an identifier in the form of hex_library_id/source_book_id is inserted into the new record. You can use this in conjunction with calibre's identifier rules to create a link to the original book (using calibre's URL scheme)
• The plugin provides an action that will be available in Action Chains plugin if you have it installed. Using the two plugins together, you can modify the title, tags or any of the metadata of the newly created books to your liking.

Notes

• This plugin uses a third party tool called k2pdfopt. For the plugin to work, you must have the k2pdfop binary available on your system path. If not, you will have to use the plugin configuration to specify the path to the k2pdfopt binary. Note that you must make sure the binary have all permissions necessary to run, otherwise the plugin will not be able to function.
• This plugin is implemented as User Interface Plugin, and not as an output (conversion) plugin. Calibre conversion pipeline necessitates converting the input format into HTML, which does not work on this case where the conversion is from PDF to PDF without any intermediary format.
• The plugin interface tries to adhere to the GUI of the original program whenever possible, but there are still some differences and omissions.
• The plugin comes with a preview window to examine the settings on a sample page of the PDF file before conversion.
• The plugin comes with the ability to create and store custom presets with different settings.
• For options that does not have a GUI element, you can specify the corresponding command-line argument in the additional options text box.
• To get a help on a any option hover over it to display the associated tooltip.
• Action Chains plugin have an action called (save/restore formats), that gives users the ability to change format extension, in case you want to restore ORIGINAL_PDF to PDF.

Running the plugin outside calibre

• You can run the plugin outside calibre to convert PDFs on your filesystem (outside calibre) in much the same way the original k2pdfopt can be used on Windows. This can be handy for users on linux and osx as both OSs do not have an official GUI. To run it use the following command:
  
  

  calibre-debug -r "K2pdfopt Plugin"

Warning

• Converting large image based PDFs using this plugin, might take a long time. This is why this plugin uses the calibre job manager to run the conversion process in the background without blocking the calibre.

Credits:

• Willus: The creator of fabulous k2pdfopt software.
• Kovid Goyal: Besides his well known work on calibre and the plugin's infrastructre, the plugin's preview window is based on calibre's Image Viewer class.
• Kiwidude: The jobs part of this plugin is based on his work in the Extract ISBN plugin.

Installation Notes:

• Download the attached zip file/install the plugin/add to toolbar or menu as described in the Introduction to plugins thread.

Special Notes:

• Requires Calibre 3.0.0 or later.

Paypal Donations:

• If you find this useful please feel free to show your appreciation.
  

Version History:

Spoiler:

Version 0.2.0 - 5 February 2024
Update: New option (Now the default option) to add the converted book to the same book record, while moving the original book to an ORIGINAL_PDF format
Update: Various modifications to page preview.

Version 0.1.1 - 2 February 2024
Update: Preview: Remove timeout and make preview button toggle into a cancel button instead.

Version 0.1.0 - 1 February 2024
Initial Release.

[capink]

reserved.

[capink]

Version 0.2.0

• Add another option (will be the default for first time users), to add the converted pdf to the same book record, while renaming the original pdf into ORIGINAL_PDF format.
• Various modifications to page preview.

Notes:

• People who already have the plugin installed and want to change to the new option, have to change the corresponding setting in the plugin's configuration dialog.
• Action Chains plugin have an action called (save/restore formats), that gives users the ability to change format extension, in case you want to restore ORIGINAL_PDF to PDF.

[capink]

Version 0.2.0

• Jobs overhaul. Shift all to threaded jobs. Jobs are invoked individually for each book so that they can be aborted mid-conversion and show progress properly.
• Page count by querying k2pdfopt without having to use any third party utils.
• Stand alone version: Double progress bar to show overall progress as well as each book progress. Also enable abort mid-conversion instead of waiting for the current book to finish.

[correoparaappzz]

hi, i'm having problems.

i don't the binary part. i downloaded the .exe form the web and put it in "Choose binary".
Now every time i click the plugin icon it opens 2 windows

[KindleAZWorKFX]

Originally Posted by correoparaappzz

hi, i'm having problems.

i don't the binary part. i downloaded the .exe form the web and put it in "Choose binary".
Now every time i click the plugin icon it opens 2 windows

it does that to me too, plus it doesn't seem to use OCR option in the k2pdfopt program 2.55 version of the program I have downloaded of the x64 binary type. it was at first opening it two times, if you click on the plugin, it opens it then once that window, click on the convert button it reopens it again. so that is two times.

its missing the plugin does seem to be missing settings that the program has like that OCR (Tesseract) feature it has within the program itself the plugin is missing that and its missing Fixed output font size, which I am not sure that even works or not, I don't really know if it affects pdf fonts in the images at all, but without OCR on the pdf files cannot use in edge and read aloud feature doesn't work for pdf without having OCRing the pdf , which does work well with pdf and text to speech, for me, it's one of the few OCR apps that seem to work well, it's a nice idea this plugin but I do think it needs work I guess.

I have x64 version of Calibre 7.5.1 and the 32x bit version calibre-5.44.0 for the text to speech voices that don't show up in Calibre's Ebook Viewer on the x64 app, both seem to work fine together, this plugin was added in 7.5.1 build of Calibre software, I am guessing the person who also has it installed is using the current version too on their pc.

I had to manually setup a save for Presets: Default profile, as it didn't come with a save value and each time I open k2pdfopt Plugin I have to reopen the Default profile, otherwise it doesn't remember my settings when the plugin is opened, that might need to be fixed in future builds, again I like the idea of this plugin it just needs some updates, looking forward to seeing its progress, and thanks for the plugin

https://www.willus.com/k2pdfopt/

https://www.willus.com/k2pdfopt/download/?-1

https://willus.com/k2pdfopt/contrib/

So, I tried that k2pdfoptgui 2.0 from k2pdfoptgui2_1_0_0,zip from contrib website it seems to get this message with the official 64bit downloads, it doesn't seem to want to output a pdf from that software? ok I think it doesn't work since it was last updated Last update: v2.1.0.0, Jan 2014 which is older than k2pdfopt.exe 2.55 version, I guess that makes sense why it doesn't work anymore, not sure why there is still a link to it on that page?

[01:20:59] WARNING: k2pdfopt.exe file size mismatch.
Conversion might not work properly.
[01:20:59] Conversion process started...

- I just found this from the link in the original post, anyway 2.55 is current but second mirror site only has 2.54 app, so its downloads I guess are a bit older since current version is not on the mirrored site for this app. this link has alternative gui I just found this, I been only using the one gui, I have to try this out, again thanks for that link I didn't know their was a post here about it too.

https://www.mobileread.com/forums/sh...d.php?t=144711

v2.55 (December 26, 2023) windows 64bit app, and it has the gui in it, not sure if their a console only version as I don't see it on Willus.com's K2pdfopt Download Page, I am pretty sure this is the correct homepage for K2pdfopt app, I don't see a console only version, I am not sure if this plugin is looking for a console only version, is their a github page for K2pdfopt that might have a console only version of it? anyway again thanks for the plugin

Also, this is a good read below on OCR and tesseract, which might be helpful.

https://www.willus.com/blog.shtml?tesseract_accuracy

one thing I am not sure about when it comes to tesseract is done k2pdfopt 2.55 default to using Tesseract v5.3 "best" or is there a setting somewhere you need to tell it to use, also is the newest version the best when converting pdf files that don't have OCR in them? since the click on box doesn't have much of the way of settings for OCR besides 50% CPUs and Fast button, I noticed that clicking on Fast button seems to make it better for pdf files and edge's read aloud feature, otherwise if Fast is not clicked on its not better at reading back words, I am not sure why this is, since I would think Fast is a lower quality but it seems to help it to read some words in some pdfs files, for me at least I had noticed that.

I am running this software on a 12th Gen Intel® Core™ i9-12900K processor, that what's in my pc if that info helps at all.

The latest installer can be downloaded here: for this I just found this windows installer, I don't know if k2pdfop will use the tesseract if installed or if it only uses its own built in one, since I didn't have it installed when using the OCR option in the app, ok I might of installed it but I forgot that I did already have a version installed does k2pdfop use this directly or does it use its own version?

tesseract-ocr-w64-setup-5.3.3.20231005.exe (64 bit)
https://digi.bib.uni-mannheim.de/tes...3.20231005.exe

https://github.com/UB-Mannheim/tesseract/wiki

https://valid.x86.fr/sgnydq

[capink]

Originally Posted by correoparaappzz

hi, i'm having problems.

i don't the binary part. i downloaded the .exe form the web and put it in "Choose binary".
Now every time i click the plugin icon it opens 2 windows

When I tested on Windows this did not happen to me. Anyway I uploaded a version that adds an additional switch to suppress GUI (-gui-). Please test and report whether you still have the same problem.

Originally Posted by KindleAZWorKFX

its missing the plugin does seem to be missing settings that the program has like that OCR (Tesseract) ...

You can use the ocr option by adding the following arguments in the Additional options box (see attached screenshot):

-ocr m -ocrlan eng

This will be saved along other options in your preset, so you will not need to re-type it every-time. Make sure to change the language according to your needs.

You can use any other command line option you wish in the same way.

I explained in another thread why I did not include the OCR option, so instead of repeating myself here, I quote what I said there:

Originally Posted by capink

I opted for now to leave out the OCR option (users can still it manually in the additional options box). The reason is that it has many languages and I am not sure if they are going to be available on every system where k2pdfopt is installed, and what kind of verification I might do to ensure the user settings can be honored. My question is does output of -ocrlang ? differ for system to system? or does the fact that k2pdfopt is installed ensures all these languages are available on all systems?

Originally Posted by KindleAZWorKFX

I had to manually setup a save for Presets: Default profile, as it didn't come with a save value and each time I open k2pdfopt Plugin ..

The new version I just uploaded does remember you last settings. Test and see whether it works for you.

Originally Posted by KindleAZWorKFX

one thing I am not sure about when it comes to tesseract is done k2pdfopt 2.55 default to using Tesseract v5.3 "best" or is there a setting somewhere you need to tell it to use, also is the newest version the best when converting pdf files that don't have OCR in them? since the click on box doesn't have much of the way of settings for OCR besides 50% CPUs and Fast button, I noticed that clicking on Fast button seems to make it better for pdf files and edge's read aloud feature, otherwise if Fast is not clicked on its not better at reading back words, I am not sure why this is, since I would think Fast is a lower quality but it seems to help it to read some words in some pdfs files, for me at least I had noticed that.

These are questions pertaning to the original program. Willus has a page for k2pdfopt in this forum where you can ask questions. I encourage you to read k2pdfopt help page before asking any questions.

I can give you a couple of pointers, regarding the fast option here is the relevant part from k2pdfopt help page:

k2pdfopt will download training files from github as needed. You can append -fast to the training file name to get the fast version, otherwise the best version will be downloaded.

As for the CPU option, you can specify it also in the additional opts box as follows:

-nt -50

Which would use 50% of the available threads, you can read more on this in the k2pdfopt help page.

Finally note, that as of now, the plugin is not tied to any particular version of k2pdfopt. You can use whatever version you want and it will be called by the plugin. Any missing options you want, just add as illustrated above in the OCR bit.

[CardamomDragon]

Does this plugin work on Mac OS? I am trying to use it on my 2020 Macbook Pro running Sonoma 14.2.1 and Calibre 6.15.1 (I haven't updated for fear of breaking other important plugins, ha).

I downloaded the MacOSX x86 64-bit file from the K2pdfopt webpage. When I get the initial dialogue on running the plugin where it asks me to choose binary, I select that file, and then receive the following error. Any support would be greatly appreciated as this seems very useful.

calibre, version 6.15.1
ERROR: Unhandled exception: <b>NameError</b>:name 'iswindows' is not defined

calibre 6.15.1 embedded-python: True
macOS-14.2.1-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '23.2.0', 'Darwin Kernel Version 23.2.0: Wed Nov 15 21:54:10 PST 2023; root:xnu-10002.61.3~2/RELEASE_X86_64')
Python 3.10.1
OSX: ('14.2.1', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (2, 9, 0) && DeDRM (10, 0, 3) && Package KFX (from KFX Input) (2, 9, 0) && K2pdfopt Plugin (0, 5, 2) && KFX metadata reader (from KFX Input) (2, 9, 0) && From KFX (2, 9, 0) && KFX Input (2, 9, 0) && Obok DeDRM (10, 0, 3)
Traceback (most recent call last):
File "calibre_plugins.k2pdfopt_plugin.gui.config", line 101, in _choose_file
if iswindows:
NameError: name 'iswindows' is not defined

[capink]

Originally Posted by CardamomDragon

Does this plugin work on Mac OS? I am trying to use it on my 2020 Macbook Pro running Sonoma 14.2.1 and Calibre 6.15.1 (I haven't updated for fear of breaking other important plugins, ha).

I downloaded the MacOSX x86 64-bit file from the K2pdfopt webpage. When I get the initial dialogue on running the plugin where it asks me to choose binary, I select that file, and then receive the following error. Any support would be greatly appreciated as this seems very useful.

Yes it works on Mac OS. I pushed a fix for the error you reported. Calibre usually shows the updates every hour. Alternatively you can directly download the new version from the first post.

[CardamomDragon]

Originally Posted by capink

Yes it works on Mac OS. I pushed a fix for the error you reported. Calibre usually shows the updates every hour. Alternatively you can directly download the new version from the first post.

Thank you. I decided to update Calibre and I downloaded the new version of the K2pdfopt plugin. When I tried again I get this error instead. I have tried moving the binary file to several different locations including my Applications and Downloads folders and the Calibre Library folder and they all give me the same result.

calibre, version 7.6.0
ERROR: Path not found: Unable to run k2pdfopt from specified location. Make sure you enter the right path and that the file has execution permissions

[capink]

Originally Posted by capink

Note that you must make sure the binary have all permissions necessary to run, otherwise the plugin will not be able to function.

Originally Posted by CardamomDragon

ERROR: Path not found: Unable to run k2pdfopt from specified location. Make sure you enter the right path and that the file has execution permissions

You need to make sure the binary has execution permissions. In the terminal:

chmod +x /path/to/k2pdfopt/binary

Or whatever the equivalent command (or method) in Mac OS is.

Edit: An alternative approach is to copy the binary to any directory in your system $PATH, in which case you do not need to specify its location. You can know the directories in you system $PATH using the following command:

echo $PATH

Note that MacOS will by default disables copying to /usr/bin. A better location would be /usr/local/bin provided it is in your system $PATH

[CardamomDragon]

Thank you, that information was super helpful and it let me get it up and running. I hope all of this helps other users too!

I ran a test job and noticed a few things.

One is that I couldn't check some of the options boxes, specifically Rotate output to landscape, Native PDF output, and color output.

Another is that I tried adding "-om 0.2" in the additional options box to get a little bit of a margin, and in the preview it didn't change anything at all.

I then ran the file through the plugin, and it correctly took the old file and changed its extension and created a new PDF file, but when I tried to open the new file I got the "could not be opened" error from Preview. "It may be damaged or use a file format that Preview doesn’t recognize."

[capink]

Originally Posted by CardamomDragon

One is that I couldn't check some of the options boxes, specifically Rotate output to landscape, Native PDF output, and color output.

That is because some options are mutually exclusive. To enable that option, you need to to set conversion mode option to null (the option is at the very bottom). The plugin follows the Windows GUI of the original program in this respect.

Originally Posted by CardamomDragon

Another is that I tried adding "-om 0.2" in the additional options box to get a little bit of a margin, and in the preview it didn't change anything at all.

The plugin function is to pass the option to the k2pdfopt program. Anything the program does or does not after this is out of the scope of the plugin.

Originally Posted by CardamomDragon

I then ran the file through the plugin, and it correctly took the old file and changed its extension and created a new PDF file, but when I tried to open the new file I got the "could not be opened" error from Preview. "It may be damaged or use a file format that Preview doesn’t recognize."

Cannot comment on this unless I test the pdf file with the same options you chose. The main thing I would be looking for is to make sure that the plugin is passing the options to the k2pdfopt binary correctly.

If there is anything related to the original program, you can ask in the k2pdfopt thread, but I recommend you first consult the k2pdfopt help page before posting there.

[Ma'am-I-Am]

I haven't installed this yet because it looks a little technical. Will it help give a better result when converting the finished PDF file to epub?

[capink]

Originally Posted by Ma'am-I-Am

I haven't installed this yet because it looks a little technical. Will it help give a better result when converting the finished PDF file to epub?

No, it will not.