k2pdfopt: optimizes PDFs for viewing on e-readers

[willus]

This version is compiled on the latest OS's with the latest C compilers and the latest third-party libraries. There is one bug fix (bookmarks work again). See details at the web site.

[VladimirS]

K2pdfopt v2.55 works perfectly !
Happy New 2024. to all K2pdfopt users

[willus]

Thank you for confirming, Vladimir.

[Yasuro]

Originally Posted by willus

This version is compiled on the latest OS's with the latest C compilers and the latest third-party libraries. There is one bug fix (bookmarks work again). See details at the web site.

I have just put ver. 2.55 to test and I am happy to report it got rid of the TOC issue! Thank you very much, Willus, for so swiftly addressing it!!! Much, much appreciated!

Yasuro

[Shohreh]

Hello,

I'm going through the options in k2pdfopt in order to read PDFs on a 6" e-reader, and have a few questions:

1. Are "Native PDF output" and "Re-flow text" related? I notice they are mutually exclusive, and sometimes are only available in certain modes

2. Under the hood, what is "reflow text", really? Does it mean that k2pdfopt will move text blocks leftward + downward, combine them into a single page before turning it from native PDF to a bitmap?

3. Would I have more chance converting a PDF into EPUB if I first run it through k2optpdf before opening its simplified output into Calibre and export as EPUB?

Thank you.
--
Edit: For some reason, -title "Some title" → "Some title.pdf"

k2pdfopt.exe -author "Some author" -title "Some title" -mode fw -p 1-5 -ls- intput.pdf

--
Edit: cropping doesn't remove data, it only hides them, and they show again when using Calibre to convert the cropped PDF to EPUB.

Is there a way to really remove the cropped data so it's gone in the output PDF?

k2pdfopt.exe -mode crop -cbox 0in,0.7in,-1in,-1in -o output.pdf input.pdf
ebook-convert.exe output.pdf output.epub --enable-heuristics

[Quoth]

if PDF is processed as an image and really cropped and converted back to a PDF, then it's really cropped.

[Shohreh]

Is there no way to hard-crop native PDF without first turning them into bitmaps?

Redaction annotation works as a way to remove text from a PDF.

[capink]

Hi, I developed a calibre K2pdfopt plugin that uses k2pdfopt to optimize PDFs inside calibre. It can also be used as standalone—It still need calibre to be installed—GUI for Linux and OSX. It can work on Windows too, but the main program already has a GUI there. For more details you can see this post.

I have some questions to I hope you can provide the answers for:

• I implemented a additional options box similar to the one in k2pdfopt Windows GUI. I want to give the options there precedence over any GUI setting, as this box supports calibre template language, which means users can customize the options per book inside calibre. In my testing it seems that whenever two options are in conflict, the last one wins. Is that reliable way to go about things? or are there situations where this assumption might not hold?
• In the Windows GUI, there is an option called "Fast Preview". On examination I saw it associated with the -rt option. The help page mentions that this option controls the degrees of rotation of the source page. I can't see how this can make a "Fast Preview", so I opted to leave it out until I understand more about it.
• I cannot see an option to control where to put the output for the -bmp option. Since the plugin also works inside the calibre, it is imperative to not give any binary direct access to files instead the library. It is not a big deal as I circumvented this by copying the source file to temporary location before previewing. Just wondering if I missed something here?
• Is there a way to validate any supplied args inside the optional arguments. Like running the command on a simulation basis to make it spit out error codes and messages without actually doing the conversion.
• I opted for now to leave out the OCR option (users can still it manually in the additional options box). The reason is that it has many languages and I am not sure if they are going to be available on every system where k2pdfopt is installed, and what kind of verification I might do to ensure the user settings can be honored. My question is does output of -ocrlang ? differ for system to system? or does the fact that k2pdfopt is installed ensures all these languages are available on all systems?
• The one option I wanted to include in the GUI and couldn't is the -ci option to reinsert the cover of the source pdf into the optimized version. Problem is it always returns the following error:
  

  Segmentation fault (core dumped)

  The error code for the error is 139. Here is a link for a book if you are interested in examining what happens.
  Note to moderators: The book linked to is legally available from hindawy.org based on an agreement with copyright holders.
• I am having problems with the preview option, when I run the following command on the book linked above:
  

  k2pdfopt -x -ui- -bmp 31 harafeesh.pdf

  Instead of previewing page 31 it actually previews page 19. How to rectify this?
  
  Edit: I am currently getting around this by extracting the target preview page to a separate pdf file using the -mode copy and then running -bmp 1 on the extracted pdf.
• Also would like to know if possible to use k2pdfopt to get information about pdf and djvu files. I am specially interested in page count. I can see an info button in the Windows screenshots and I suppose that's what they do.

Last but not least, thanks for the efforts and the great program.

[willus]

Originally Posted by Shohreh

1. Are "Native PDF output" and "Re-flow text" related? I notice they are mutually exclusive, and sometimes are only available in certain modes

Sorry for the late reply. "Native PDF" means the output consists of the original PDF source and then each page is just a sequence of cropped regions from that source material. To do that at the re-flow level is not practical--too many cropboxes and would overload most readers. The other option is that k2pdfopt simply builds a simple bitmap of each output page, so in that case re-flow works fine. That's why they are mutually exclusive.

Originally Posted by Shohreh

2. Under the hood, what is "reflow text", really? Does it mean that k2pdfopt will move text blocks leftward + downward, combine them into a single page before turning it from native PDF to a bitmap?

That's right. k2pdfopt determines word locations and then re-organizes the words on each bitmapped page to fit the page size best. If you use the -sm option, you can get more of an idea of what it is doing.

Originally Posted by Shohreh

3. Would I have more chance converting a PDF into EPUB if I first run it through k2optpdf before opening its simplified output into Calibre and export as EPUB?

These are my (dated) comments on epub conversion.

Originally Posted by Shohreh

Is there a way to really remove the cropped data so it's gone in the output PDF?

You started a thread on hard trimming PDFs not long ago.

[willus]

Bummer--I had responded to all of your questions and hit "Submit Reply" and mobileread decided to log me out and lose all my changes. Ugh. I'll start again...

Originally Posted by capink

I implemented a additional options box similar to the one in k2pdfopt Windows GUI. I want to give the options there precedence over any GUI setting, as this box supports calibre template language, which means users can customize the options per book inside calibre. In my testing it seems that whenever two options are in conflict, the last one wins. Is that reliable way to go about things? or are there situations where this assumption might not hold?

Yes, that should work. K2pdfopt processes command-line options left to right so the ones at the end should over-write the ones from before.

Originally Posted by capink

In the Windows GUI, there is an option called "Fast Preview". On examination I saw it associated with the -rt option. The help page mentions that this option controls the degrees of rotation of the source page. I can't see how this can make a "Fast Preview", so I opted to leave it out until I understand more about it.

When k2pdfopt first processes a source PDF, it tries to figure out how it is generally oriented, so it looks at 10 pages within the document (evenly spaced throughout) and for each page does analysis to see if it should rotate the page 90 degrees to get the text oriented horizontally. This takes some time, so using the -rt option in the right way tells it not to do this, hence speeding up the preview.

Originally Posted by capink

I cannot see an option to control where to put the output for the -bmp option. Since the plugin also works inside the calibre, it is imperative to not give any binary direct access to files instead the library. It is not a big deal as I circumvented this by copying the source file to temporary location before previewing. Just wondering if I missed something here?

You did not miss anything, and you came up with a good solution, but it would be easy to add an option to specify the file name of the -bmp output.

Originally Posted by capink

Is there a way to validate any supplied args inside the optional arguments. Like running the command on a simulation basis to make it spit out error codes and messages without actually doing the conversion.

No. I'm not even sure how I would do that.

Originally Posted by capink

I opted for now to leave out the OCR option (users can still it manually in the additional options box). The reason is that it has many languages and I am not sure if they are going to be available on every system where k2pdfopt is installed, and what kind of verification I might do to ensure the user settings can be honored. My question is does output of -ocrlang ? differ for system to system? or does the fact that k2pdfopt is installed ensures all these languages are available on all systems?

The output from -ocrlang will vary depending on the environment, e.g. what language training files have been downloaded. Note that k2pdfopt will try to download any needed training file per my command-line option help on -ocrlang.

Originally Posted by capink

The one option I wanted to include in the GUI and couldn't is the -ci option to reinsert the cover of the source pdf into the optimized version. Problem is it always returns the following error:

I was able to reproduce this issue. I'll look into it. I have not used that option in a long time.

Originally Posted by capink

I am having problems with the preview option, when I run the following command on the book linked above. Instead of previewing page 31 it actually previews page 19. How to rectify this? Edit: I am currently getting around this by extracting the target preview page to a separate pdf file using the -mode copy and then running -bmp 1 on the extracted pdf.

The page number specifies the output file page number, not the source file page number. But it looks like you've come up with a good work-around.

Originally Posted by capink

Also would like to know if possible to use k2pdfopt to get information about pdf and djvu files. I am specially interested in page count. I can see an info button in the Windows screenshots and I suppose that's what they do.

Use the -i option.

[willus]

Originally Posted by capink

Hi, I developed a calibre K2pdfopt plugin that uses k2pdfopt to optimize PDFs inside calibre. It can also be used as standalone—It still need calibre to be installed—GUI for Linux and OSX. It can work on Windows too, but the main program already has a GUI there. For more details you can see this post.

BTW thank you for doing this, and very nice work. I'll post a link to it on my k2pdfopt user contributions page hopefully sometime soon.

[capink]

Originally Posted by willus

BTW thank you for doing this, and very nice work. I'll post a link to it on my k2pdfopt user contributions page hopefully sometime soon.

Thanks for you detailed reply and your work.

[braxman]

Hi there! I've been working in a custom way to read papers on a pdf-based e-ink reader and annotate with a stylus, but I also wanted to use cropped pdf files in the device. Came to use pdf-crop-margins because I can crop the file, send to e-reader, anotate, and then also and crucially undo the cropping while retaining the annotations in the file now in its orginal size for greater glory of PC screen display.

But the cropping tool is quite limited compared with the smart cropping that k2pdfopt does (no splitting of pages, for instance).

TL;DR: Is there a way to undo the k2pdfopt process recovering the original file from the transformed (and annotated) one? If not, can I suggest that as a feature request?

[willus]

Originally Posted by braxman

TL;DR: Is there a way to undo the k2pdfopt process recovering the original file from the transformed (and annotated) one? If not, can I suggest that as a feature request?

Sorry but this would involve more thinking and new programming than I am up for. I never intended k2pdfopt to have "undo" functionality. I figured people would keep the original source file.

[braxman]

Originally Posted by willus

Sorry but this would involve more thinking and new programming than I am up for. I never intended k2pdfopt to have "undo" functionality. I figured people would keep the original source file.

Ok no problem, great software, writing native pdf output is a quantum leap. Thanks for all.