An algorithm to render PDF in small devices

[vinniet]

I took IceHand script along with the PI compile under windows. I am using a bash command from cygwin to run this under Windows. I have been able to replace all Unix commands with Windows equivalent. The script runs until I get a error under PI.

Converting file "Lotus Domino Administrator 6.pdf". Please wait ...
Error: No display font for 'Symbol'
Error: No display font for 'ZapfDingbats'
pi: Error reading row. Short read of 1514 bytes instead of 1530

I have included the PDF that is just a simple 2 page tech manual. Please let me know what I am doing wrong. Maybe I am trying too hard to get this to work under windows.

Thanks!

[IceHand]

I think it's because the fonts are not embedded in the PDF file and pdftoppm seems to be unable to find the missing fonts in the fonts folder. Try again with embedded fonts or try to install the missing fonts (the first would be easier I guess).
I've attached the converted PDF how it would look if you had the required fonts installed.

[caritas]

Most items in wish list is reasonable. Although I may have not enough ability to finish all of them. I have a new version now and hopes it accomplish some wishes.

[nrapallo]

Originally Posted by caritas

Most items in wish list is reasonable. Although I may have not enough ability to finish all of them. I have a new version now and hopes it accomplish some wishes.

Thank you for all your efforts!

Wishlists are guides to implement changes. Please do start with the easy ones and (eventually) work up to the hard ones.

No pressure to do so, as we all should be grateful for (free) software that works as advertised!

I have been thinking about your methods without actually following your code. These are some ideas you may choose to incorporate:

1. When the (halfway) split point falls on a word, the decision to add the flex to find the end of that word should take into account 'how much of that word is to the right'. In particular, if more of the word falls to the right of the split point, then the split should occur at the *previous* word; not the word where the split falls.

2. If the main pdf text appears justified, then the (halfway) split point will usually fall at the same point on each subsequent line. However, if the text appears left-aligned, then the split point should be calculated on each line's actual width. This will avoid having the first line segment always longer than the split next line segment.

[nrapallo]

Originally Posted by caritas

I have a new version now

Great sample .pdf (chap6.pdf). BTW, I was in the same university undergrad program as one of the authors (Alfred J. Menezes ), though I do not know him personally!

Yeah, Bachelor of Mathematics (1986) from the University of Waterloo!!!
How about that for school spirit!

[IceHand]

Thanks a lot for your hard work, caritas!

The new version doesn't work on my laptop, any advice? When I try to run the pi executable directly I get the error:

bash: ./pi: cannot execute binary file

When I try the method described in the first post (. env.sh; pi_format.py chap6.conf) I get the error:

Traceback (most recent call last):
  File "/home/icehand/Downloads/pi/bin/pi_format.py", line 16, in <module>
    pi_lib.page_divide_all(tmpl_fn)
  File "/home/icehand/Downloads/pi/bin/pi_lib.py", line 174, in page_divide_all
    doc_conf.gen(file(tmpl_fn, 'r'))
  File "/home/icehand/Downloads/pi/bin/pi_lib.py", line 38, in gen
    page_param = get_page_param(self)
  File "/home/icehand/Downloads/pi/bin/pi_lib.py", line 128, in get_page_param
    page_info = get_page_info(pt_fn)
  File "/home/icehand/Downloads/pi/bin/pi_lib.py", line 97, in get_page_info
    p = Popen(['pi_page_info', fn], stdout = PIPE)
  File "/usr/lib/python2.5/subprocess.py", line 594, in __init__
    errread, errwrite)
  File "/usr/lib/python2.5/subprocess.py", line 1091, in _execute_child
    raise child_exception
OSError: [Errno 8] Exec format error

And finally when I try to compile the pi executable myself I get the error:

gcc -o pi -g pi.o -lnetpbm
pi.o: In function `pi_line_divide':
/home/icehand/Downloads/pi/pi/pi.c:546: undefined reference to `max'
/home/icehand/Downloads/pi/pi/pi.c:547: undefined reference to `min'
/home/icehand/Downloads/pi/pi/pi.c:561: undefined reference to `min'
pi.o: In function `pi_page_divide':
/home/icehand/Downloads/pi/pi/pi.c:824: undefined reference to `max'
/home/icehand/Downloads/pi/pi/pi.c:827: undefined reference to `min'
/home/icehand/Downloads/pi/pi/pi.c:828: undefined reference to `min'
/home/icehand/Downloads/pi/pi/pi.c:830: undefined reference to `min'
collect2: ld returned 1 exit status
make: *** [pi] Error 1

[caritas]

> bash: ./pi: cannot execute binary file

This time, I use a x86_64 to develop, so the binary is for x86_64.

The compiling error seems comes from libnetpbm. The max and min is defined in libnetpbm. Make sure there is /usr/include/pm.h available. I use libnetpbm10 package in debian.

[IceHand]

I have the equivalent of libnetpbm installed and /usr/include/pm.h is present (I use Arch Linux where packages are not split into a normal package and a lib package). Is a specific version of libnetpbm needed? I have version 10.35 installed.

Btw:

cat /usr/include/pm.h | grep max
pm_maxvaltobits(int const maxval);
pm_bitstomaxval(int const bits);

It doesn't look like "max" and "min" are defined in my version of pm.h

[nrapallo]

Originally Posted by IceHand

I have the equivalent of libnetpbm installed and /usr/include/pm.h is present (I use Arch Linux where packages are not split into a normal package and a lib package). Is a specific version of libnetpbm needed? I have version 10.35 installed.

Btw:

It doesn't look like "max" and "min" are defined in my version of pm.h

In pi.c, just add:

#define max(a,b) ((a) > (b) ? (a) : (b))
#define min(a,b) ((a) < (b) ? (a) : (b))

This addition allowed me to compile pi.c under cygwin/windows, but that is as far as I could get.

Sorry, I'm not on Linux so I'm adapting to your setup as best as I can.

[nrapallo]

Originally Posted by caritas

Basic Usage for version 0.2:

tar -xjf pi_0.2.tar.bz2
cd pi
. env.sh
cd test
pi_format.py chap.conf
/* output goes in out directory */

Caritas:

The 'pi_0.2.tar.bz2' unarchives as incomplete. In the resulting bin directory I get zero length files for 'pi_image_bbox', 'pi_image_crop', 'pi_page_divide' and 'pi_page_info'. Is this normal? I have used three different unarchivers (including 7-zip) and got the same results.

Please fix version 0.2 so we can all start to use your (wonderful) program!

[IceHand]

Originally Posted by nrapallo

The 'pi_0.2.tar.bz2' unarchives as incomplete. In the resulting bin directory I get zero length files for 'pi_image_bbox', 'pi_image_crop', 'pi_page_divide' and 'pi_page_info'. Is this normal?

It's normal, they are just links to the pi executable, I don't know the reason they exist though.

Btw, I was able to compile pi with the code you posted in your previous post, but it's not working as it should. When running the test chapter I get the error:

Error open pgm image file (null): Bad address.
pi_page_divide: pi.c:330: pi_image_save: Assertion `0' failed.

[rmanasa]

Greetings -

Idiot in Da House time. First, thanks for all the effort to address this long standing, complex and frustrating issue. Greatly appreciated.

Now to business. I've extracted the Windows version, copied a pdf into that directory, opened a command prompt, cd'd over to that directory and typed "pi old.pdf new.pdf". As soon as I said ".pdf", most of you started shaking your heads, I know - pi is looking for pgm or pbm files.

I'm reasonably intelligent, but not a programmer. I haven't been able to figure out how to create either of these file types from reading the thread. Is there some kind soul out there who can show me how to learn and take advantage of this fine conversion program?

Looking forward to your reply. Thank you!

[nrapallo]

Originally Posted by rmanasa

Greetings -

Idiot in Da House time. First, thanks for all the effort to address this long standing, complex and frustrating issue. Greatly appreciated.

Don't worry, when it comes to using new (breaking edge) software, we all are at a disadvantage and feel frustrated when things don't work and that makes us fell a bit sheepish about asking for help (myself included).

Now to business. I've extracted the Windows version, copied a pdf into that directory, opened a command prompt, cd'd over to that directory and typed "pi old.pdf new.pdf". As soon as I said ".pdf", most of you started shaking your heads, I know - pi is looking for pgm or pbm files.

Good that you caught yourself on that one!

I was able to find a compiled 'pdftoppm.exe' for windows users at http://www.foolabs.com/xpdf/download.html .

Just get Win32 (built with MSVC): xpdf-3.02pl2-win32.zip and extract it; therein you will find the .exe and help .txt.

I think you already have pi-exe.zip.

That should do it. Just issue:

pdftoppm -r 180 -f 1 -l 1 -gray -aa yes a.pdf a

where you can replace "-f 1 -l 1" with your first and last page numbers to convert, 'a.pdf' with your .pdf filename and the following 'a' with any prefix for the resulting .pgm filename you want.

p.s. don't venture onto version 0.2 just yet as no-one has been able to make that work yet on Linux/Mac or Windows. Stay tuned!

I'm reasonably intelligent, but not a programmer. I haven't been able to figure out how to create either of these file types from reading the thread. Is there some kind soul out there who can show me how to learn and take advantage of this fine conversion program?

Looking forward to your reply. Thank you!

You're welcome!

[rmanasa]

Thanks for the consideration, Nick. I don't know why everyone says you're so mean.


I extracted the pdftoppm program from your link, set it up as best I can, changed one default parameter - so it would create a pgm file instead of a ppm file - and gave it a go. That might be the source of subsequent problems with pi-exe, but it didn't prevent pdftoppm from doing it's thing.

The program created 52 pgm files from the pdf I'm using for testing purposes. While that's a lot, it's the one type of pdf I know I'm gonna need to convert monthly, so I figured I'd see what happened.

Took one of those 52 files, and did the "pi X.pgm Y" thing, which produced the following messages:

C:\Documents and Settings\Rick\My Documents\Unzipped\pi-exe\pi> pi CT0408-000001 .pgm ct0408-01
6 [main] pi 1832 _cygtls::handle_exceptions: Exception: STATUS_ACCESS_VIOLATION
1524 [main] pi 1832 open_stackdumpfile: Dumping stack trace to pi.exe.stackdump
1029497 [main] pi 1832 _cygtls::handle_exceptions: Exception: STATUS_ACCESS_VIOLATION
1062966 [main] pi 1832 _cygtls::handle_exceptions: Error while dumping state (probably corrupted stack)

The stackdump file look slike this:

Exception: STATUS_ACCESS_VIOLATION at eip=004014C5
eax=000000FF ebx=0000002B ecx=7FF13198 edx=00000262 esi=00000004 edi=0066423C
ebp=0023CC38 esp=0023CC20 program=C:\Documents and Settings\Rick\My Documents\Unzipped\pi-exe\pi\pi.exe, pid 3676, thread main
cs=001B ds=0023 es=0023 fs=003B gs=0000 ss=0023
Stack trace:
Frame Function Args
0023CC38 004014C5 (00661350, 00660210, 00000000, 000006D9)
0023CCB8 004026CD (00000003, 006601A8, 00660090, 610BEEB7)
0023CD98 610060D8 (00000000, 0023CDD0, 61005450, 0023CDD0)
61005450 61004416 (0000009C, A02404C7, E8611021, FFFFFF48)
1314401 [main] pi 3676 _cygtls::handle_exceptions: Exception: STATUS_ACCESS_VIOLATION
1356063 [main] pi 3676 _cygtls::handle_exceptions: Error while dumping state (probably corrupted stack)


As far as I know, I'm not using the 0.2 version, that you cautioned about (though who knows? Anything's possible wit Da Idiot in Da House.)

Looking forward to your reply. Thank you!

[nrapallo]

Originally Posted by rmanasa

Thanks for the consideration, Nick. I don't know why everyone says you're so mean.

Hersay!

I extracted the pdftoppm program from your link, set it up as best I can, changed one default parameter

OK now this forewarns me that something bad is going to happen...

- so it would create a pgm file instead of a ppm file

the '-gray' switch should take care of that, I think.

- and gave it a go. That might be the source of subsequent problems with pi-exe, but it didn't prevent pdftoppm from doing it's thing.

The program created 52 pgm files from the pdf I'm using for testing purposes. While that's a lot, it's the one type of pdf I know I'm gonna need to convert monthly, so I figured I'd see what happened.

Took one of those 52 files, and did the "pi X.pgm Y" thing,

you did mean 'pi X.pgm Y.pgm'?

which produced the following messages:

C:\Documents and Settings\Rick\My Documents\Unzipped\pi-exe\pi> pi CT0408-000001 .pgm ct0408-01
6 [main] pi 1832 _cygtls::handle_exceptions: Exception: STATUS_ACCESS_VIOLATION
1524 [main] pi 1832 open_stackdumpfile: Dumping stack trace to pi.exe.stackdump
1029497 [main] pi 1832 _cygtls::handle_exceptions: Exception: STATUS_ACCESS_VIOLATION
1062966 [main] pi 1832 _cygtls::handle_exceptions: Error while dumping state (probably corrupted stack)

Yes, this is what is referred to in this thread as a segmentation fault. It basically is the program's way of crashing.

The stackdump file look slike this:

Exception: STATUS_ACCESS_VIOLATION at eip=004014C5
eax=000000FF ebx=0000002B ecx=7FF13198 edx=00000262 esi=00000004 edi=0066423C
ebp=0023CC38 esp=0023CC20 program=C:\Documents and Settings\Rick\My Documents\Unzipped\pi-exe\pi\pi.exe, pid 3676, thread main
cs=001B ds=0023 es=0023 fs=003B gs=0000 ss=0023
Stack trace:
Frame Function Args
0023CC38 004014C5 (00661350, 00660210, 00000000, 000006D9)
0023CCB8 004026CD (00000003, 006601A8, 00660090, 610BEEB7)
0023CD98 610060D8 (00000000, 0023CDD0, 61005450, 0023CDD0)
61005450 61004416 (0000009C, A02404C7, E8611021, FFFFFF48)
1314401 [main] pi 3676 _cygtls::handle_exceptions: Exception: STATUS_ACCESS_VIOLATION
1356063 [main] pi 3676 _cygtls::handle_exceptions: Error while dumping state (probably corrupted stack)

All this means is that the .pdf is too low in resolution, or has a non-white background, or the 'white-gaps' between lines are not easily detectable. In otherwords, your .pdf cannot be converted by version 0.1 "as is".

As far as I know, I'm not using the 0.2 version, that you cautioned about (though who knows? Anything's possible wit Da Idiot in Da House.)

Looking forward to your reply. Thank you!

Yes, you don't have version 0.2 as no-one has that one working yet!