k2pdfopt: optimizes PDFs for viewing on e-readers

[willus]

Originally Posted by dhdurgee

That did the trick in most cases, but I am finding that a one page PDF is being processed twice! Looking at these cases it seems that one page PDFs are first processed using the -cbox1 setting and then again with the -cbox2- setting.

Any idea why this is happening? I guess I can work around it with a two pass process, but it is strange. Is this a bug that needs fixing?

Dave

It's a bug. I'll fix it in the next release.

[dhdurgee]

Originally Posted by willus

It's a bug. I'll fix it in the next release.

Great. How soon will the fixed release be available?

Dave

[willus]

Originally Posted by dhdurgee

Great. How soon will the fixed release be available?

Dave

I can't make any commitment. The time I can spend on k2pdfopt is limited and not always predictable, but I'll do my best to get it out sometime in the next month or so. I've got a few other bugs and new features I'm working on as well.

[willus]

Originally Posted by dhdurgee

Great. How soon will the fixed release be available?

Dave

I found the bug. Actually, if you use -cbox2-9999 in your script instead of -cbox2- then it should work correctly (9999 just has to be higher than the max number of pages you expect in any document).

[dhdurgee]

Originally Posted by willus

I found the bug. Actually, if you use -cbox2-9999 in your script instead of -cbox2- then it should work correctly (9999 just has to be higher than the max number of pages you expect in any document).

I just confirmed this worked in my case as you indicated. Just wondering, should the -cbox2-9999 be required given that 0,0 is the default -cbox value?

Dave

[willus]

Originally Posted by dhdurgee

I just confirmed this worked in my case as you indicated. Just wondering, should the -cbox2-9999 be required given that 0,0 is the default -cbox value?

Dave

The idea is that once you specify -cbox even once, k2pdfopt uses that paradigm, and only converts the specified crop boxes over their specified page ranges. I suppose that may not be the most natural way to think about it, but once you know that, it's not difficult to use the option correctly. I will clarify this in the usage.

[dhdurgee]

Originally Posted by willus

The idea is that once you specify -cbox even once, k2pdfopt uses that paradigm, and only converts the specified crop boxes over their specified page ranges. I suppose that may not be the most natural way to think about it, but once you know that, it's not difficult to use the option correctly. I will clarify this in the usage.

That should do the trick. As long as it is clear how this works it can be dealt with.

On a totally separate subject, I know that your tool does OCR for the purpose of enabling searches to work on a scanned document. Is it possible now, or would an enhancement make sense, to offer an option to create a mobi or epub document with the OCR text and the graphics/images from the source as opposed to a PDF document? I am aware that no OCR is perfect, so there will certainly be errors in the output, but the ebook format will be a much more flexible fit to a dedicated reader than even the current output of the package.

This might be beyond your design goals for this tool, but I think there might be others beside myself interested in a native format document for a dedicated reader from less than perfect sources. Your tool does well with adapting a scanned PDF document for a dedicated reader. This might be a logical further step along the path.

Dave

[willus]

Originally Posted by dhdurgee

That should do the trick. As long as it is clear how this works it can be dealt with.

On a totally separate subject, I know that your tool does OCR for the purpose of enabling searches to work on a scanned document. Is it possible now, or would an enhancement make sense, to offer an option to create a mobi or epub document with the OCR text and the graphics/images from the source as opposed to a PDF document? I am aware that no OCR is perfect, so there will certainly be errors in the output, but the ebook format will be a much more flexible fit to a dedicated reader than even the current output of the package.

This might be beyond your design goals for this tool, but I think there might be others beside myself interested in a native format document for a dedicated reader from less than perfect sources. Your tool does well with adapting a scanned PDF document for a dedicated reader. This might be a logical further step along the path.

Dave

I have thought about other output formats off and on, but there are several other tools that will do that kind of thing (calibre, for one), and probably better than I could, so I'm trying to keep k2pdfopt focused on its core functionality. You could use k2pdfopt (or any number of other tools) to OCR the text in a file and then do the conversion in calibre after the OCR is created. This is fraught with errors, though, and inevitably requires a lot of hand editing. See some of the threads listed in my pdf conversion tips.

[wdq4587]

Is there are any way to prevent the k2pdfopt scale the font?

What I mean is no scale, no zoom, no rotate, not increase the font size, no pixel resampling. I have struggle with this program for 2 days and no luck until now. For other people they may only want 2 words have same size in source also display same size in output PDF, but what I want is the scale factor always equal to 1 exactly so I can convert it to mobi format losslessly and without filesize increasement.Yes, what I mean is want k2pdfopt only doing mark region, crop and offset operation, no zoom, no rotate.

I have tried trim mode and native PDF output, it can works as I said. But when I try split and merge the page, I can not find out a option to prevent the k2pdfopt doing scale.

[willus]

Originally Posted by wdq4587

Is there are any way to prevent the k2pdfopt scale the font?

What I mean is no scale, no zoom, no rotate, not increase the font size, no pixel resampling. I have struggle with this program for 2 days and no luck until now. For other people they may only want 2 words have same size in source also display same size in output PDF, but what I want is the scale factor always equal to 1 exactly so I can convert it to mobi format losslessly and without filesize increasement.Yes, what I mean is want k2pdfopt only doing mark region, crop and offset operation, no zoom, no rotate.

I have tried trim mode and native PDF output, it can works as I said. But when I try split and merge the page, I can not find out a option to prevent the k2pdfopt doing scale.

Yes--I'd have told you to use -mode trim. It works pretty much exactly as you said you want--each page simply has the margins trimmed away and is then saved as a converted page, in native mode, the size of which matches the trimmed source page, so each converted page may be a different size. So I guess I'm not sure exactly what you want. What do you mean by "when I try to split and merge the page"? Maybe you want to attach a specific PDF example and the commands you are running on it?

[wdq4587]

I know I need explain.

First is what I want: I want view a scanned PDF using mobi format in my Kindle paperwhite 2. The reason is I dislike kinlde display a border around the PDF content, and also kindle may be not stable for PDF file. Then I want the convert process is lossless to avoid scale image twice (kinlde will scale while display if resolution not match the best resolution, although I don't know which is but the PDF file I've tested is small than that), and do not want increase the output filesize (the test file image have 5 color and resample will have 32 color and double the filesize).

Then what I have done. After many test and adjustment, I found using trim mode and native PDF output, by using a tool to render the pages, the result image can match the source exactly pixel by pixel. But since I view it in landscape mode, I need k2pdfopt split the pages to about 40% original height and trim and merge with next page. Then I found the I can not render out page exactly as the original. Even using native PDF mode, the transform matrix of Xobj image can not always be like "1 0 0 1 -12 -405", so it will resample pixels and color number will be extended and finale mobi filesize will double.

By now I just think I almost have made it, by using such parameter (I am using GUI):

-m 74p,145p,74p,81p -idpi 72 -bp-- 0 -vb -2
-de 1.5 -wrap- -n -c -col 1 -odpi 72 -h 0.37s -w 0.8s -ws 0

but there are a few pages still have 32 color and big size, I checked then I found these are combined by two pages, I guess there are a gap between page. But even I use -bp-- 0, it's still there, 2 matrix look like "0.99643 0 0 0.99643 -71.74332 220.49945" and "0.99643 0 0 0.99643 -60.78253 -531.09894".

[wdq4587]

So, what I want?

For other people, prevent font size change too much:
may be a minimum scale factor and max scale factor setting for all pages,
or a page zoom factor = max scale factor / minimum scale factor within a output page,
or both.

For me, a lossless option which can avoid pixel resampling:
scale factor always = 1
(x,y) offset always be integer by using pixel as unit.

To check does it lossless or not is easy, using 4 or 5 color images as source, if total output filesize not increase much (compare by total image filesize), and check the biggest output image file to see how many color?

[willus]

Originally Posted by wdq4587

I know I need explain.

First is what I want: I want view a scanned PDF using mobi format in my Kindle paperwhite 2. The reason is I dislike kinlde display a border around the PDF content, and also kindle may be not stable for PDF file. Then I want the convert process is lossless to avoid scale image twice (kinlde will scale while display if resolution not match the best resolution, although I don't know which is but the PDF file I've tested is small than that), and do not want increase the output filesize (the test file image have 5 color and resample will have 32 color and double the filesize).

Then what I have done. After many test and adjustment, I found using trim mode and native PDF output, by using a tool to render the pages, the result image can match the source exactly pixel by pixel. But since I view it in landscape mode, I need k2pdfopt split the pages to about 40% original height and trim and merge with next page. Then I found the I can not render out page exactly as the original. Even using native PDF mode, the transform matrix of Xobj image can not always be like "1 0 0 1 -12 -405", so it will resample pixels and color number will be extended and finale mobi filesize will double.

By now I just think I almost have made it, by using such parameter (I am using GUI):

-m 74p,145p,74p,81p -idpi 72 -bp-- 0 -vb -2
-de 1.5 -wrap- -n -c -col 1 -odpi 72 -h 0.37s -w 0.8s -ws 0

but there are a few pages still have 32 color and big size, I checked then I found these are combined by two pages, I guess there are a gap between page. But even I use -bp-- 0, it's still there, 2 matrix look like "0.99643 0 0 0.99643 -71.74332 220.49945" and "0.99643 0 0 0.99643 -60.78253 -531.09894".

Any chance you can post the source and conversion (or PM to me) so I can see what you are talking about? I kind of get it, but not completely. BTW, -mode fw turns on a lot of the options you're using. It's the same as:

-n -wrap- -col 1 -vb -2 -f2p -2 -t

If you don't want it to include -f2p -2 or -t, you can turn those off after you specify it, e.g.

-mode fw -f2p 0 -t-

Also, since you've turned off re-flow (-wrap-), you don't need to specify the -ws 0 option. It won't be used unless re-flow is turned on.

[willus]

Originally Posted by wdq4587

So, what I want?

For other people, prevent font size change too much:
may be a minimum scale factor and max scale factor setting for all pages,
or a page zoom factor = max scale factor / minimum scale factor within a output page,
or both.

For me, a lossless option which can avoid pixel resampling:
scale factor always = 1
(x,y) offset always be integer by using pixel as unit.

To check does it lossless or not is easy, using 4 or 5 color images as source, if total output filesize not increase much (compare by total image filesize), and check the biggest output image file to see how many color?

I guess I'm still not understanding you. If you are using native PDF output, any issues with scale factors being pixel-for-pixel with your display, or with file size or lossless images, become moot. With native PDF output, k2pdfopt just reassembles the source using cropping, rotation, and translation. It does not have any further control over losslessness or file size. It leaves the source PDF's internal bitmap encoding or resolutions completely unchanged.

[wdq4587]

Originally Posted by willus

I guess I'm still not understanding you. If you are using native PDF output, any issues with scale factors being pixel-for-pixel with your display, or with file size or lossless images, become moot. With native PDF output, k2pdfopt just reassembles the source using cropping, rotation, and translation. It does not have any further control over losslessness or file size. It leaves the source PDF's internal bitmap encoding or resolutions completely unchanged.

Let me explain:

First is I want export k2pdfopt processed PDF file to images and then convert to mobi file. Then I found I can not let even the native PDF output render pixel-for-pixel image result because the transformation matrix not equal to (1 0 0 1 x y). May be I can adjust let it look like 1.001 but not exactly to 1. That will cause pixel resampling while rending the page.

I know if I view the native PDF mode processed PDF in kindle directly, it will only scale once to screen. But in this situation I mean convert to other file format, extract images form the PDF file will not help, I need render the pages to get the split page and merge page result.

Secondary I also want in text reflow mode can has a option pixel-for-pixel, so the result filesize will not increase. I know there are some limitation, for example can not use Autostraighteen and so may be cause bad result.

But in other meaning, if the output file resolution not match the read device best resolution, the read device will do scale. And since the text reflow process have scaled once, then when reader view it will scale twice. I think scale twice can not have better quality then only scale once by read device, even it use bilinear interpolation and text reflow process use some advanced algorithm like bicubic.

Only way can get better quality now is not let the read device scale, only the text reflow process scale. That need output resolution just match the read device best display resolution which I don't know exactly. And this resolution will also change if some user using landscape mode and others using portrait mode.

My test file here: http://pan.baidu.com/s/1jGtGXpW password: 1h91

BTW, I also suggest add using percent option in GUI for -m option.