k2pdfopt: optimizes PDFs for viewing on e-readers

[onlinespending]

I noticed that k2pdfopt doesn't deal with drop caps (those large capital letters that begin a chapter that occupy two or more lines of text) particularly well. Are there particular settings that might improve its handling of those? thanks

[willus]

Originally Posted by onlinespending

I noticed that k2pdfopt doesn't deal with drop caps (those large capital letters that begin a chapter that occupy two or more lines of text) particularly well. Are there particular settings that might improve its handling of those? thanks

Not at the moment. I've tried to get k2pdfopt to be better about splitting lines that are spanned by a drop-cap when wrapping text (i.e. effectively ignoring the drop-cap and breaking it in two). It's not ideal but I figured it's preferable to leaving the lines joined together and being unwrapped. Correct drop-cap handling has been low on my list because it's a fair amount of effort for a relatively small gain, but I'll keep it on my radar.

[ephestione]

Originally Posted by willus

Interesting. Do you know if it is the joining technique or just the sheer size of the PDF that reduces performance? You used pdfsam to join the files? Could you try jpdftweak also? Or could you try an overnight conversion that does it all in one piece (though as I recall this segfaulted k2pfdopt--so maybe just half of it?) to see how that compares? What would be best would be an A-B comparison between something produced only with k2pdfopt vs. something put together with various joining tools.

Sorry for the late reply, I get mail notifications about new posts each week.
Soooo.. the path I took to process the pdf was kind of the only one I could follow, I was not able to use k2pdfopt on the whole PDF without the crash, and even at half of the file the final PDF would still be a fraction of the whole size so it wouldn't tell us much even if the pdf worked better on the kindle, maybe the device is just not "enough" to process PDF's bigger than a certain size.
Also, pdfsam was the only one that worked for me to join the files, as jpdftweak didn't do anything, even after allowing up to 2GB of RAM to java applications, I waited eons and had no output.

Anyway, out of curiosity I am still converting the pdf with the latest version of k2pdfopt, let's see out it fares.

...played around, apparently by enabling "text reflow" disables "native pdf output", I realized it only later. By keeping text reflow disabled I could convert in native mode (albeit slow since only on one core) and the final size not only is very similar to the original file (200mb vs 250mb of your splitter command line batch some pages ago) but I retain color and text quality is better.
Have yet to test on the kindle tho, I just started assembling my 3d printer which arrived just now :P

[ephestione]

Ok, this time the coversion went fine, but... well... the output file is 860MB compared to the original 200mb, quite overkill.
Used native pdf output, text reflow, and autocrop... I don't understand.

[willus]

Originally Posted by ephestione

Ok, this time the coversion went fine, but... well... the output file is 860MB compared to the original 200mb, quite overkill.
Used native pdf output, text reflow, and autocrop... I don't understand.

Glad to hear it worked, at least. If it crashes again, PM me the source file and your settings. Text re-flow overrides native PDF and turns it off. The two are incompatible for the way I convert the documents. That's why the converted document is larger than the source document--it's all bitmaps now. But it may process faster that way on your kindle. There's not a lot of processing or memory usage the kindle has to deal with to display bitmaps.

I've used jpdftweak on files up to 750 pages and 50 MB, but I had to increase the java memory size to > 1 GB. I like it better than PDFSAM for what I do. It's got a much smaller install footprint, and I can't get PDFSAM to work from the command line, even after spending 15 minutes trying to get it to work. But PDFSAM does seem to be able to handle larger files than jpdftweak (from the GUI).

[ephestione]

(pasting here what I added in my previous message, as you already replied)


...played around, apparently by enabling "text reflow" disables "native pdf output", I realized it only later. By keeping text reflow disabled I could convert in native mode (albeit slow since only on one core) and the final size not only is very similar to the original file (200mb vs 250mb of your splitter command line batch some pages ago) but I retain color and text quality is better.
Have yet to test on the kindle tho, I just started assembling my 3d printer which arrived just now :P

[oren64]

How can I crop/remove the bottom of the page by 0.3in , I am using kobo glo 942px-4.42in height, i tried few options but it didn't work, the top works well.
and how can skip the first 2 pages.

-cbox- -ibox- -cbox 0in,0.3in,0in,0in

[willus]

Originally Posted by oren64

How con I crop/remove the bottom of the page by 0.3in , I am using kobo glo 942px-4.42in height, i tried few options but it didn't work, the top works wall.
and how can skip the first 2 pages.

-cbox- -ibox- -cbox 0in,0.3in,0in,0in

I didn't make -cbox work the same way as the -m (crop margins) option, unfortunately. See the command-line usage page. You want this:

-p 3- -mb 0.3in

-p 3- to convert page 3 and up (skip first two page).
-mb 0.3in to crop the bottom 0.3 inches out.

[ephestione]

Just copied the new native pdf job on the kindle and tested it. Globally smaller than the pdf's made with your multitasking suggestion, and text is top quality, on par with the native ebooks quality, plus page turning is as fast the smaller, splitted pixelated pdf's... images are clear and better... if only I started reading the book in this quality instead of picking up just now
Apparently pdfsam made somehow the smaller joined pdf's slower to read on the kindle, and since I couldn't succesfully run jpdftweak on those I wouldn't know if that's better.

[willus]

Originally Posted by ephestione

Just copied the new native pdf job on the kindle and tested it. Globally smaller than the pdf's made with your multitasking suggestion, and text is top quality, on par with the native ebooks quality, plus page turning is as fast the smaller, splitted pixelated pdf's... images are clear and better... if only I started reading the book in this quality instead of picking up just now
Apparently pdfsam made somehow the smaller joined pdf's slower to read on the kindle, and since I couldn't succesfully run jpdftweak on those I wouldn't know if that's better.

Yes, native PDF output does have all those advantages, though I don't think it's always faster to process than a bitmapped file--it depends on the PDF. Plus the text of the source file has to be laid out so that you can still read it without text re-flow, which isn't always the case. I'm surprised that pdfsam would be the culprit. I think one would really have to do more tests to confirm that for sure. My hunch would be that it's a combination of pdfsam and the size of the file.

[ephestione]

I still am convinced pdfsam is to "blame". Separate 30mb-ish segments: text and images look like sh*t but I can turn pages relatively fast; same exact files merged together with pdfsam, 240mb-ish file, I may have to wait up to 3 seconds to view the next page. 200mb-ish native pdf conversion, text and images are brilliant, practically as fast as 30mb-ish sh*tty segments

[willus]

Originally Posted by ephestione

I still am convinced pdfsam is to "blame". Separate 30mb-ish segments: text and images look like sh*t but I can turn pages relatively fast; same exact files merged together with pdfsam, 240mb-ish file, I may have to wait up to 3 seconds to view the next page. 200mb-ish native pdf conversion, text and images are brilliant, practically as fast as 30mb-ish sh*tty segments

The 30 MB segments are bitmapped, or native conversions? What if you just join two of them with pdfsam to make a smaller file? If they are bitmapped, what if they are native instead and then you join them? If you have the time to run those tests, I'd be curious to know (or PM me the file and I'll run my own tests).

[sailco12]

The "k2pdfopt" software at http://willus.com/k2pdfopt/ is a real lifesaver for those wanting to read PDF files on their e-reader, especially those with 2 columns. It converts them to an e_pub file. However, apparently there is no traditional documentation section prominently linked from the present website.

The real instructions for usage seem to exist only in two videos. Unless people know that, they may give up on this fine software, so it would be better if the help page made this clear.

The video for Windows users is
"GETTING STARTED WITH THE WINDOWS GUI" at
http://www.willus.com/k2pdfopt/help/at.

For other users, see the video
"K2PDFOPT NATIVE PDF OUTPUT MODESK2PDFOPT NATIVE PDF OUTPUT MODES" at
http://www.willus.com/k2pdfopt/help/native.shtml

In particular, there is a help system which is displayed by running "k2pdfopt" in a terminal. It is shown beginning 214 seconds into the video.


It would also be great if the web page could link to instructions for Nook users. Their seems to be no mention of the Nook on the web site. As someone using Linux on the desktop and `Nook Simple Touch^ as a tablet, my method was to

download the correct file from http://www.willus.com/k2pdfopt/download/

install it by running sudo cp k2pdfopt /usr/local/bin

convert my 2 column PDF by running

k2pdfopt my.pdf -mode 2col -dev nookst -x

Until I realized that I needed to specify `native pdf output mode^ (the `-mode 2col^ includes this mode), all I saw on
my e-reader were question marks.

[ephestione]

Originally Posted by willus

The 30 MB segments are bitmapped, or native conversions? What if you just join two of them with pdfsam to make a smaller file? If they are bitmapped, what if they are native instead and then you join them? If you have the time to run those tests, I'd be curious to know (or PM me the file and I'll run my own tests).

I will go with the second option, I MAY find the time to do that but probably you will receive a response in more than a month's time.
So sending you a link in PM.
The first 30mb blocks were done with the sample commands suggested by you to me a few pages ago to split the load of k2pdfopt over multiple instances hence multiple cpus by using the command start.

[willus]

Thank you for your post.

Originally Posted by sailco12

The "k2pdfopt" software at http://willus.com/k2pdfopt/ is a real lifesaver for those wanting to read PDF files on their e-reader, especially those with 2 columns. It converts them to an e_pub file....

This is not quite true. K2pdfopt converts to a PDF file (not an e-pub) tailored for the size of the device specified by the user.

Originally Posted by sailco12

The real instructions for usage seem to exist only in two videos. Unless people know that, they may give up on this fine software, so it would be better if the help page made this clear.

The video for Windows users is
"GETTING STARTED WITH THE WINDOWS GUI" at
http://www.willus.com/k2pdfopt/help/at.

For other users, see the video
"K2PDFOPT NATIVE PDF OUTPUT MODESK2PDFOPT NATIVE PDF OUTPUT MODES" at
http://www.willus.com/k2pdfopt/help/native.shtml

In particular, there is a help system which is displayed by running "k2pdfopt" in a terminal. It is shown beginning 214 seconds into the video.

There are actually four videos on the web site (go to help page and look for "with video" in the left column):
1. MS Windows GUI (your link to this video doesn't work)
2. Text-based menu
3. Using a shortcut
4. Using Native output
Maybe I could highlight them better on the main page. Also, do you mean 2 minutes and 14 seconds rather than 214 seconds (which would be 3 minutes and 34 seconds)? It is not entirely clear to me what "help system" you refer to, unless it's the text-based menu from the 2nd video above.

Originally Posted by sailco12

It would also be great if the web page could link to instructions for Nook users. Their seems to be no mention of the Nook on the web site.

I could put some more obvious tips on the main page and help pages about selecting the device.

Originally Posted by sailco12

...Until I realized that I needed to specify `native pdf output mode^ (the `-mode 2col^ includes this mode), all I saw on
my e-reader were question marks.

Can you please post or PM me the source file and the options you used to convert it that resulted in all question marks?