k2pdfopt: optimizes PDFs for viewing on e-readers

[willus]

Originally Posted by famfam

Is tesseract integrated in the k2pdfopt-gui and if so version 3 or version 4? Do the traindata files have to be version 3 or version 4? In which folder must the traindata files be in Windows 10: 'Programs' (for 64 bit) or 'Programs (x86)' (for 32 bit)?
In my k2pdfopt-gui I get the error message,

Initializing OCR for 2 threads x x
Could not find Tesseract data (env var TESSDATA_PREFIX = (not assigned)).
Using GOCR v0.50.

What am I doing wrong?
Is my entry in the input window 'Env. var: TESSDATA_PREFIX = c: \ program files \ tesseract-ocr \ tessdata 'not correct?

http://willus.com/k2pdfopt/help/ocr.shtml

Though looking at it now, I see this help page needs some updating. The latest version uses Tesseract 4.0 and can use either Tesseract 3 or 4 training files (I recommend the v4 training files). See also the command-line help page (search for “Tesseract”).

Also, in the GUI “help” menu, select “Tesseract Training File Info”.

[slide13]

Hello there!
I am trying to find the optimal options for k2pdfopt for my Kindle Paperwhite 2, unfortunately I can't..
The pdf is attached, the language is Greek and after some search, I have used the following options: -mode fp -dev kp2.
Can you please help me?
Thank you very much for your time!

[willus]

Originally Posted by slide13

Hello there!
I am trying to find the optimal options for k2pdfopt for my Kindle Paperwhite 2, unfortunately I can't..
The pdf is attached, the language is Greek and after some search, I have used the following options: -mode fp -dev kp2.
Can you please help me?
Thank you very much for your time!

What do you not like about the conversion? Is the text too small?
Option 1:
k2pdfopt -mode fw -ls- -dev kp2 myfile.pdf
If the text is too small for you, then:
Option 2:
k2pdfopt -dev kp2 myfile.pdf

[slide13]

Originally Posted by willus

What do you not like about the conversion? Is the text too small?
Option 1:
k2pdfopt -mode fw -ls- -dev kp2 myfile.pdf
If the text is too small for you, then:
Option 2:
k2pdfopt -dev kp2 myfile.pdf

Thank you so much for your immediate response and support!
Yes, the text was too small, the 2nd option helped me, even though the letters are somehow "dirty" now, and the chapters don't start from a new page, or the header/footer seem to be put in the middle of the page (screenshots attached).
But it's ok, it is more than readable, thank you very much dear willus!
Your program rocks and you're so helpful!
Congratulations!

[willus]

Originally Posted by slide13

Thank you so much for your immediate response and support!
Yes, the text was too small, the 2nd option helped me, even though the letters are somehow "dirty" now, and the chapters don't start from a new page, or the header/footer seem to be put in the middle of the page (screenshots attached).
But it's ok, it is more than readable, thank you very much dear willus!
Your program rocks and you're so helpful!
Congratulations!

If the page numbers and headings bother you, you can crop them out pretty reliably like so for your particular document:

k2pdfopt -dev kp2 -m 0.29in,0.73in,0.32in,0.33in nesbo.pdf

The "dirty" look is not really the fault of k2pdfopt. Your source file, if you magnify it enough, has a very pixelated look to it.

[slide13]

It is perfect!
Thank you so much, it's unbelievable that you respond to each one of us separately..
I really really appreciate it, have a nice day!

[meem]

Very hard to use.
So slow to preview the result on good PCs.
I can not get a correct result for a simple two-column book.

[willus]

Originally Posted by meem

Very hard to use.
So slow to preview the result on good PCs.
I can not get a correct result for a simple two-column book.

Did you use the help pages/videos at all? Did you try 2-column mode? Sometimes the compression type of the PDF can slow things down for previewing. JPX images are CPU intensive. OCR can also be CPU intensive.

Can you post or PM me a sample of your source PDF?

[pandaeye]

Hello, I am very new to this app but it helped do what I expected, almost. Please do not mind if I am asking stupid question.

As far as I understand, text reflow cannot be done together with native pdf, am I right? I am curious why they cannot be done together. Because I am converting a book which is mostly regularly formatted text, converting it to bitmap does seem silly and a very big file indeed. Is there any way I can do to do the text reflow while keeping a small file size?

[DaleDe]

You need to read PDF#Reflow. You should also know that there are two types of PDF documents. One has text while the other has images of text. A PDF that has image of text must use an OCR to extract the text from an image.

Dale

[willus]

Originally Posted by pandaeye

Hello, I am very new to this app but it helped do what I expected, almost. Please do not mind if I am asking stupid question.

As far as I understand, text reflow cannot be done together with native pdf, am I right? I am curious why they cannot be done together. Because I am converting a book which is mostly regularly formatted text, converting it to bitmap does seem silly and a very big file indeed. Is there any way I can do to do the text reflow while keeping a small file size?

I agree 100%. It does seem silly. But k2pdfopt is not the right app to reflow text from a "native" / non-scanned PDF. At its core, k2pdfopt is an image processing program, and it simply rearranges "crop boxes" from the source PDF into new positions. If those crop boxes are large--one or two per page--it can spit out a converted PDF by simply adding a few cropping instructions into the source PDF file--effectively telling it to display its contents in a modified way. But if those crop boxes are numerous and small--i.e. one for each word or row of text, which is how k2pdfopt does reflow--then this method doesn't work well. The resultant PDF won't display well in most readers. It's better to bitmap it.

If you have access to MS Word, I recommend you try loading your PDF into Word. See my PDF conversion tips.

[pandaeye]

Originally Posted by willus

I agree 100%. It does seem silly. But k2pdfopt is not the right app to reflow text from a "native" / non-scanned PDF. At its core, k2pdfopt is an image processing program, and it simply rearranges "crop boxes" from the source PDF into new positions. If those crop boxes are large--one or two per page--it can spit out a converted PDF by simply adding a few cropping instructions into the source PDF file--effectively telling it to display its contents in a modified way. But if those crop boxes are numerous and small--i.e. one for each word or row of text, which is how k2pdfopt does reflow--then this method doesn't work well. The resultant PDF won't display well in most readers. It's better to bitmap it.

If you have access to MS Word, I recommend you try loading your PDF into Word. See my PDF conversion tips.

Thank you very much. I think I know more about your work now. I am very satisfied with it.

[Flumine]

I wish I saw your application before, it is really unique of its kind (at least I have not yet found anything better).
Actually, I had a similar idea about a two years ago - to re-flow wide scanned documents on a glyph level to fit into mobile screen, so spent some time with my friend writing an android application. The result we got is working fine but with some limitations - it could not recognize complex formulas or multi-column layout.
Here is a good page sample:
https://slack-files.com/T9YDZ38JY-FUG1J0AKA-40bce696bf
Here is a sample of original page which failed to re-flow properly - with glyphs recoginzed - https://glyphs.flum.app/image?id=448&mode=glyphs.
To recognize glyphs we are using OpenCV library and it mostly works fine but it is hard to get formulas to be recognized as a single image. Your application is working much better with them so I wonder what algorithm you are using for that?

[solanoctes]

What settings (screen size,font,margins.etc) should i use for the paperwhite 10th edition? I tried using the paperwhite 4 prest but the font was too big (like the bottom of the title page spilled over to the next page.

[willus]

Originally Posted by Flumine

I wish I saw your application before, it is really unique of its kind (at least I have not yet found anything better).
Actually, I had a similar idea about a two years ago - to re-flow wide scanned documents on a glyph level to fit into mobile screen, so spent some time with my friend writing an android application. The result we got is working fine but with some limitations - it could not recognize complex formulas or multi-column layout.
Here is a good page sample:
https://slack-files.com/T9YDZ38JY-FUG1J0AKA-40bce696bf
Here is a sample of original page which failed to re-flow properly - with glyphs recoginzed - https://glyphs.flum.app/image?id=448&mode=glyphs.
To recognize glyphs we are using OpenCV library and it mostly works fine but it is hard to get formulas to be recognized as a single image. Your application is working much better with them so I wonder what algorithm you are using for that?

If you look at the comments at the top of the main source file, k2pdfopt.c, it outlines the high level process used by k2pdfopt and points out some of the key C functions. The algorithms for detection are just my own inventions, with a lot of trial and error for what works well and what doesn't. The basic concept is to first look for columnar regions / large blocks of the page by scanning for horizontal and vertical blank (white) areas between the regions, and then to break those columns/regions into rows of text, and then the rows of text into words. The process has given me a deep appreciation for how easily the human brain can visually parse a page (and instantly know "that is text" and "that is an image", etc.) compared to how hard it is to write a reliable algorithm to do the same thing.