k2pdfopt: optimizes PDFs for viewing on e-readers

[Fanhoso]

Hey, thanks a lot for your program. However, there seems to be a problem with the latest version (1.63 32-bit). It keeps on rising its memory usage on both Windows and Linux until there isn't any more memory left. Here's the final part of the output:

SOURCE PAGE 123 of 360 (8.5 x 11.0 in) ... 2 new pages saved.

SOURCE PAGE 124 of 360 (8.5 x 11.0 in) ... 3 new pages saved.
error: malloc of array (4664 x 7208 bytes) failed

 ** ERROR reading from /home/pedro/Dropbox/IST/2011 Fall/PE/2010-09-12-NotasApoioPE-EdRevista-SeccaoFolhas.pdfusing MuPDF.  Trying Ghostscript...


 ** ERROR reading page 125 from /home/pedro/Dropbox/IST/2011 Fall/PE/2010-09-12-NotasApoioPE-EdRevista-SeccaoFolhas.pdf.


 ** ERROR reading page 126 from /home/pedro/Dropbox/IST/2011 Fall/PE/2010-09-12-NotasApoioPE-EdRevista-SeccaoFolhas.pdf.


 ** ERROR reading page 127 from /home/pedro/Dropbox/IST/2011 Fall/PE/2010-09-12-NotasApoioPE-EdRevista-SeccaoFolhas.pdf.


 ** ERROR reading page 128 from /home/pedro/Dropbox/IST/2011 Fall/PE/2010-09-12-NotasApoioPE-EdRevista-SeccaoFolhas.pdf.


 ** ERROR reading page 129 from /home/pedro/Dropbox/IST/2011 Fall/PE/2010-09-12-NotasApoioPE-EdRevista-SeccaoFolhas.pdf.


 ** ERROR reading page 130 from /home/pedro/Dropbox/IST/2011 Fall/PE/2010-09-12-NotasApoioPE-EdRevista-SeccaoFolhas.pdf.


 ** ERROR reading page 131 from /home/pedro/Dropbox/IST/2011 Fall/PE/2010-09-12-NotasApoioPE-EdRevista-SeccaoFolhas.pdf.


 ** ERROR reading page 132 from /home/pedro/Dropbox/IST/2011 Fall/PE/2010-09-12-NotasApoioPE-EdRevista-SeccaoFolhas.pdf.


 ** ERROR reading page 133 from /home/pedro/Dropbox/IST/2011 Fall/PE/2010-09-12-NotasApoioPE-EdRevista-SeccaoFolhas.pdf.


 ** ERROR reading page 134 from /home/pedro/Dropbox/IST/2011 Fall/PE/2010-09-12-NotasApoioPE-EdRevista-SeccaoFolhas.pdf.


 ** (No more read errors will be echoed for file /home/pedro/Dropbox/IST/2011 Fall/PE/2010-09-12-NotasApoioPE-EdRevista-SeccaoFolhas.pdf.)


275 pages written to /home/pedro/Dropbox/IST/2011 Fall/PE/2010-09-12-NotasApoioPE-EdRevista-SeccaoFolhas_k2opt.pdf (8.9 MB).

Press <ENTER> to exit.

[willus]

Originally Posted by Fanhoso

Hey, thanks a lot for your program. However, there seems to be a problem with the latest version (1.63 32-bit). It keeps on rising its memory usage on both Windows and Linux until there isn't any more memory left. [/CODE]

Thanks. I've looked into this and fixed it for the next release. It was a memory leak in the new code that I added to make multi-column detection faster. I'll try and get a new release out reasonably soon. Meanwhile, you should be able to get around this by using the -p option to limit the number of pages converted.

[willus]

Originally Posted by eleaticus

I had k2pdfopt on this Windows 7 Home Premium laptop but crashes lost it. Loved it. But now both the 32 and 64-bit latest versions say "cannot open" no matter what I throw at it. Including files I had converted previously.

Can you post a screen shot of the error? Are you sure you have write permission to the folder the PDF files are in? What version of k2pdfopt were you running before--do you know?

[Fanhoso]

Originally Posted by willus

Thanks. I've looked into this and fixed it for the next release. It was a memory leak in the new code that I added to make multi-column detection faster. I'll try and get a new release out reasonably soon. Meanwhile, you should be able to get around this by using the -p option to limit the number of pages converted.

Great! Glad you found the leak so quickly

Now, is it possible to make k2pdfopt cut every page in a file in two approximate halves so as not to cut through any text?

For example, by telling it to output exactly 2 pages for each input page and to assume device height of half the input page height?

[willus]

Originally Posted by Fanhoso

Now, is it possible to make k2pdfopt cut every page in a file in two approximate halves so as not to cut through any text?

For example, by telling it to output exactly 2 pages for each input page and to assume device height of half the input page height?

I don't have quite what you want. The closest is to use the -grid option, e.g.

k2pdfopt -grid 1x2x5 ...

will split each source page into a 1 x 2 grid with 5% overlap. It will not try to break between lines, though. You also have to manually specify the device width and height with -w and -h. You could also try:

k2pdfopt -mode fw -ls- ...

This will fit the (cropped) width of the source page to the device width and put as much of each source page as it can on each destination page, breaking at intelligent places. Use -bp to force a destination page break wherever there is a source page break. If you don't want the source page margins trimmed, add -t- to your options.

[eleaticus]

All error messages when running from all (few) folders and all the various pdfs are the same: "cannot open <pathname>" and all the folders are ones I'm in and out of all the time, reading, 'riting, and 'rithmaticing.

Plenty of disk space.

The previous version? Don't know but prolly 'bout 3 years ago.

The 32 bit now up says 1.34a (I thought it said 1.63 on the download page?)

I don't recall - didn't even notice - what the top line of the 64-bit version said, the "1.63" I downloaded.

Please notice that the "quote message in reply?" line is broken.

[willus]

Originally Posted by eleaticus

All error messages when running from all (few) folders and all the various pdfs are the same: "cannot open <pathname>" and all the folders are ones I'm in and out of all the time, reading, 'riting, and 'rithmaticing.

Plenty of disk space.

The previous version? Don't know but prolly 'bout 3 years ago.

The 32 bit now up says 1.34a (I thought it said 1.63 on the download page?)

I don't recall - didn't even notice - what the top line of the 64-bit version said, the "1.63" I downloaded.

Please notice that the "quote message in reply?" line is broken.

You might be having the same issue GGMerlin had. See post #195. He had a non-standard ASCII character in his path (e.g. letters with accent marks over them), but I did implement a fix which seemed to work for him. An attached screen shot would really help--you can send it to me in a private message if you want. Or at least post or send me a verbatim copy of the complete path name. Also, is it that k2pdfopt can't read the source file or can't write the destination file? Try moving the PDF file to the root folder in C: or D: and seeing if that works.

BTW, seeing as k2pdfopt has only experienced significant downloading activity since July 2011, I don't think your last version was 3 years old.

[Fanhoso]

Originally Posted by willus

I don't have quite what you want. The closest is to use the -grid option, e.g.

k2pdfopt -grid 1x2x5 ...

will split each source page into a 1 x 2 grid with 5% overlap. It will not try to break between lines, though. You also have to manually specify the device width and height with -w and -h. You could also try:

k2pdfopt -mode fw -ls- ...

This will fit the (cropped) width of the source page to the device width and put as much of each source page as it can on each destination page, breaking at intelligent places. Use -bp to force a destination page break wherever there is a source page break. If you don't want the source page margins trimmed, add -t- to your options.

Awesome. Grid option is about what I wanted. And it doesn't leak memory! Which leads me to another question: setting an option via the interactive menu (either by e.g l or -ls) takes precedence over the default mode, right? So if I set any option (even if it's not set in the default mode), the mode is ignored and only the option that I set is used?

[willus]

Originally Posted by Fanhoso

Awesome. Grid option is about what I wanted. And it doesn't leak memory! Which leads me to another question: setting an option via the interactive menu (either by e.g l or -ls) takes precedence over the default mode, right? So if I set any option (even if it's not set in the default mode), the mode is ignored and only the option that I set is used?

Very good question. You're close, but not quite right. The -mode options are actually a combination of settings. See the usage page (under -mode). Anything you put into the interactive menu will get parsed after the default options, and the later options overwrite previous ones, which is why the -ls- must be after the -mode fw option in my previous example so that it flips the destination orientation back to portrait. The easiest way to be sure to turn off all the options that are turned on by -mode fw is to put -mode def into the interactive menu (or select "mo" then "d") and then follow that with whatever other options you want instead.

BTW, the next release will be able to automatically size the output pages to the trimmed source page sizes, as you requested.

[Fanhoso]

Originally Posted by willus

Very good question. You're close, but not quite right. The -mode options are actually a combination of settings. See the usage page (under -mode). Anything you put into the interactive menu will get parsed after the default options, and the later options overwrite previous ones, which is why the -ls- must be after the -mode fw option in my previous example so that it flips the destination orientation back to portrait. The easiest way to be sure to turn off all the options that are turned on by -mode fw is to put -mode def into the interactive menu (or select "mo" then "d") and then follow that with whatever other options you want instead.

BTW, the next release will be able to automatically size the output pages to the trimmed source page sizes, as you requested.

Hm, but, e.g. -col's default value is 4, but on the -mode def it's 2. Which one is used then?

And why is it that if I use -grid it doesn't leak memory, if, from what I understood, all the default options are still used? Is it because it sets -col 1?

And thanks for the added functionality - look forward to use it

[willus]

Originally Posted by Fanhoso

Hm, but, e.g. -col's default value is 4, but on the -mode def it's 2. Which one is used then?

And why is it that if I use -grid it doesn't leak memory, if, from what I understood, all the default options are still used? Is it because it sets -col 1?

And thanks for the added functionality - look forward to use it

Whoops--the usage list is wrong. The default for -col is 2. I'll fix that in the next release. The memory leak only occurs for -col 2 (or 4) because that forces the call to the function that scans the source pages for multiple columns, which has the leak. The -grid option does not call the column finding function since it brute-force grids the source page instead.

[willus]

Originally Posted by willus

The most reliable way for you to convert your PDF file (if you want native PDF output) is with the -bp option. This ensures that every converted page will have content from only a single source page. Most of the issues you are having are because of the way k2pdfopt renders multiple source document pages onto one destination page. If the two source pages use the same macro name for different things, then the result is that the macro from the earlier page gets overwritten by the macro from the later page, and the displayed converted page is then incorrect. With some PDF files you can get away with it because each page has different macro names, but with your particular PDF file, you cannot (this is why examples are so helpful). There is probably a better way to combine two source PDF pages into a single destination page, but I haven't figured it out yet (still learning about PDF).

Unfortunately, that means you have to live with the gaps due to the page breaks. There's not much I can do about that given that -bp has to be used to render the converted file correctly (unless you use non-native mode by adding -n-). Sorry I can't do better for you for now. I'll try and investigate more about how to render multiple source pages into one destination page. I think there are other programs that do it that I can try and learn from. ....

This issue with content from two different source pages being put onto a single destination page has been resolved. The next release will not have this issue.

[willus]

An OSX GUI for k2pdfopt has been contributed by Adrian Zandberg.

[willus]

K2pdfopt v1.64 is released. This version improves the native PDF output so that contents from multiple source pages can reliably be displayed on one output page. It also has some other minor new features and a memory-leak bug fix. See the web site for details.

[willus]

K2pdfopt v1.64a is released. This version fixes a native PDF output bug in v1.64. See the web site for details.