A4 PDFs - tips, tricks, and a call to the community

[alex_d]

hmm, that c&p into Word method isn't bad. I'll give it a try. I'll see if I can convert it to something other than PDF, though, since using even medium-sized fonts (such as the Reader's 'small' BBeB setting) comes out too faint (@#$@ sony).

kovidgoyl, thanks a lot for the htm to BBeB suggestion. That sounds perfect.

Henkvdg, I have a feeling that the reason SciAm looks good on the Iliad is that it renders the fonts right.* If I put the Reader into landscape, it effectively has more resolution than the Iliad but the characters are still very hard to read. Also, The Reader has only very rudimentary presets for zooming/rotating PDFs (not that panning is very enjoyable with eInk).

*In fact, if Iliad didn't want to pay Adobe for its software, they are probably using the same Ghostscript engine I am.


Anyway, I've noticed that inverting colors makes glare worse. Another thing I've found to work well is using the -unsharp 2 parameter in imagemagick. It essentially makes the letters bolder but messes with the antialiasing.
convert -unsharp 2 infile.*.png outfile.%03d.png

PNG - example 2e

I think even better image filters can be created. I know very little about this, but maybe somebody else does?

[alex_d]

HTML2LRF isn't quite working. The image won't fill up the whole page, even after editing the template lrf.

In fact, I even tried writing my own .lrs and then compiling it with lrs2lrf. Same results. The sample books have cover images that fill up the page, so there's gotta be a way...
source LRS
result LRF

[alex_d]

HTML2LRF isn't working right. The image won't fill up the whole page, even after editing the template lrf.

In fact, I even tried writing my own .lrs and then compiling it with lrs2lrf. Same results. The sample books have cover images that fill up the page, so there's gotta be a way...
source LRS
result LRF

[kovidgoyal]

You'll probably have to read the bbeb spec, a link to it is in one of the threads by igor in the developers section

[wadood]

I think Sony should really fix pdf problem. It has little to do withs creen size just bad rendering. I have Sony UX-280P which has a smaller screen size but PDFs are readable in all sizes and under all resolutions.

[William Moates]

Originally Posted by wadood

I have Sony UX-280P which has a smaller screen size but PDFs are readable in all sizes and under all resolutions.

I looked it up, and the UX-280P runs Windows XP, on a 4.5” LCD screen. This differs from the PRS-500, which runs Linux on an EPD (ElectroPhoretic Display) screen. The comparison is apples to oranges.

Saying that, I agree with you. I think it's one of the top 5 features that needs fixing.

[alex_d]

Thanks a lot, kovidgoyal. Alright, I've combined all of this into a .cmd script that turns PDFs into pre-rastered LRFs.

The advantages of converting PDFs are:

PDF text has much greater boldness
Page turns take approx 2s (including eInk refresh) (regardless of the pdf)
Opportunity to set zoom region
All formatting, graphics, tables, etc. are 100% preserved
Non-english fonts displayed correctly

Disadvantage:

Large file size
Since images are at a specific resolution, you must optimize either for portrait or landscape.


PDFrasterFarian has a nice interface and explains everything. There is no learning curve or command prompts, just extract and double-click.
You can also call it using PDFrasterFarian.cmd inputfile.pdf outputfile.lrf


Strongly Recommended: Use a program like Adobe Acrobat to edit the PDF's cropbox and select the specific region you'd like to zoom into. (The cropped region should be the same size for each page.) Tips on integrating software to do this are appreciated.



The script reads a PDF, rasterizes it using Ghostscript, enhances the text using ImageMagick, writes a .lrs file, and finally compiles a full LRF BBeB using lrs2lrf. (These programs are called from directories in the zip file. They do not get installed.)

Still TODO: Option to create landscape-mode LRFs. Write a better image-enhancement algorithm to improve text boldness (current one is really clumsy). Extract the ToC from the pdf. Allow full command-line parameters (promptless operation). Reduce size of outputted lrfs.

PDFrasterFarian will hit 1.0 when I feel a good image algorithm is found. If any image expert out there knows a decent one.. please share. I'm using this framework. Right now I'm just using the imagemagick unsharp filter, which is silly.

[William Moates]

alex_d, kudos for your hard work.

[kovidgoyal]

Sounds good alex, I will give it a try next time i'm browsing a paper. Meanwhile, have you seen the PDF hacks website?
http://www.accesspdf.com/article.php/20050825112121686

[alex_d]

man, you always come in with these great little pearls. They seem small at first, but they're always exactly what i'm missing. That link'll be great to get the ToC running.

One cool thing I noticed: If you press the page forward key multiple times in one of my BBeBs, the pages turn at exactly 1s/page (ie the refresh rate), but the first turn takes 2s. I wonder if I can make every page turn in 1s.


Ok, version 0.9.5 brings these improvements:

Support for landscape resolution. The actual orientation, however, is chosen on the Reader so you always have a little flexibility.
About 4x smaller file size (sometimes even smaller than the original pdf).
Somewhat better image algorithm. Still not perfect, but it's an improvement esp for very small letters. (All i'm really doing is applying the old one at higher res and then downsampling... this adds the antialiasing that was missing)
Lastly, there's some fixes and polishings (imagemagick wants to load all images into memory before processing, which can use up gigabytes. therefore, i have to feed it one image at a time)


some more musings:
The 2s to start up, 1s to turn is a bit puzzling. I guess maybe there's 1s of processing that must be done and when flipping through pages it's overlapped with eInk. I tried using uncompressed bitmaps to reduce processing but couldn't get lrs2lrf to accept them. Also, seeing as the framebuffer for the 800x600 2-bit screen is only 120KB, sony really has NO excuse for not pre-rendering the page ahead. $@#$ sony. what kind of brain dead developer would not think to do that.... &^$%! sony.

Also, the render-at-high-res-then-downsample isn't really a huge leap and wastes a lot of time (especially in landscape). Ugh... what kind of filter could I use? Maybe compute a distance-weighed average of surrounding pixels and then use that value for the current pixel only if it's darker than the current pixel?

UPDATE:
the version i posted had a small change i forgot to undo. (i mean bug). So use 0.9.6.

[kovidgoyal]

Glad to be of service ;-)

So the major todos left are:
1) Automate setting the crop box.
2) Somebody needs to write a dll independent version of lrs2lrf.exe that implements a sufficient subset of its functionality so that we can make PDFrasterfarian platform independent

I'd do that myself, except that I'm burning all my time on making the linux GUI.

[cbookrun]

I registered just to write this note. Thank you for doing this and sharing it with us. I've had this reader for a couple months but it has mostly been collecting dust because I can't read PDFs on it. I started playing with it again last night with renewed enthusiasm. There is hope!

Thanks again.

[alex_d]

cbookrun, I'm glad you've found this useful!

Ok, ToC is up. Turned out pretty easy with the help of pdftk and the PDF Hacks book. (there's some concern about pdftk hanging... let me know if that happens)

Also, I found out what image filter i'm looking for. It's called dilate, and it's used precisely for this sort of thing. I haven't yet figured out how to do a proper implementation of it, though, but shouldn't be too hard.

Autocropping, however, is turning out a bit frustrating. I used perl2exe to convert pdfcrop.pl, and then xpdf's pdfinfo to read pagesize and adjust the dpi automatically. That's fine, but I can't for the love of god configure miktex's paths so that it doesn't have to be installed. It's required by pdfcrop.pl and I haven't found anything besides pdfcrop other than the ImageXpress c++ library.

Incidentally, ImageXpress is designed specifically for grayscale text and can do dilate. So maybe it's just easier to make an executable out of that. Any takers?

[kovidgoyal]

I'm not sure I understand. You're saying miktex is a dependency of pdfcrop.pl? Looking at the source it seems like it should only need ghostscript?

EDIT: I see that it uses pdftex to calculate margins

So why do you want to make a monolothic distribution anyway, Let people who want to use it install MikTeX.

EDIT2: You can compile pdftex as a standalone
http://www.tug.org/applications/pdftex/

[alex_d]

It's not that pdftex doesn't run, it's that it asks for ini files and I can't find any parameters or variables to tell it where to look. But yeah, better to just get it working and either leave it as an optional feature or fix it later.