All on the Mac: PDFs with Formatting!

myelinator · 02-09-2008, 10:11 AM

Hello, all.

I noticed that there were many people who were struggling to find a way to make formatted PDFs for their Kindle with their Mac without resorting to MobiPocket creator. I needed to do the same thing and was getting frustrated with being forced to use an emulator. Here is the method I came up with. It is rather technical and requires some UNIX terminal knowledge and is just my quick and dirty homebrew method. However, it took a signficant amount of time to develop and I wanted to make sure to share it with others.

Please understand that I am extremely busy and don't get to spend as much time doing this as I might like. (I've been using this for over a month and just now have had a moment to sit down and type this.) Therefore, I may or may not have time to answer questions, but I've tried to be as clear as I can.

This program relies on two other pieces of software:

Mobiperl (https://www.mobileread.com/forums/sho...&highlight=prc) and its dependencies (perl, etc.)
ImageMagick (http://www.imagemagick.org/)

After you have downloaded Mobiperl you need to make modifications to the source perl to optimize it for use with the Kindle. In the file [...your perl libraries...]/MobiPerl/Util.pm in the subroutine get_image_data() change the lines that read:

Code:

my $maxwidth = 480;
my $maxheight = 640;

to:

Code:

my $maxwidth = 525;
my $maxheight = 700;

Now you are ready for the shell script that puts it all together (I named it pdf2mobi):

Code:

#!/bin/bash

for FILE in "$@"; do
FILEPATH=${FILE%/*}
FILENAME=${FILE##*/}
FILEBASE=${FILENAME%%.*}
FILEEXT=${FILENAME#*.}
IMGEXT=jpg

# if my experiments are right then the size of the Kindle screen is 525x700
# (a 4:3 aspect ratio) as one might expect

# use imagemagick to do all the needed conversions:
# -density              166x166 to get the right dpi
# -trim                 off the margins (do it before you resize!)
# -resize 700           to get the width of the pages maximized
# -rotate -90           to maximize the use of the screen
# -colorspace gray      self explanatory
# -type Grayscale       self explanatory
# -level                darken the image
# +dither               turn off dither
# -level 34%,100%,1.2   darken the image
# -map kindle.gif       a four color gif with the allowed shades of gray
echo "**********************************************************************"
echo "* Converting to $FILE to $IMGEXT files... please be patient..."
echo "**********************************************************************"
convert -density 166x166  "${FILEBASE}.pdf" -trim +repage -resize 700 -rotate -90 -colorspace gray -type Grayscale -level 34%,100%,1.2 +dither -map kindle.gif "${FILEBASE}.${IMGEXT}"

echo "Dividing the $IMGEXT page images..."
for PAGE in "${FILEBASE}"*.${IMGEXT}
  do
    PAGEBASE=${PAGE%%.*}
    echo $PAGEBASE
    convert "${PAGEBASE}.${IMGEXT}" -crop 525x700+0 +repage "${PAGEBASE}-01.${IMGEXT}"
    convert "${PAGEBASE}.${IMGEXT}" -crop 525x700+450 +repage "${PAGEBASE}-02.${IMGEXT}"
    convert "${PAGEBASE}.${IMGEXT}" -crop 525x700+900 +repage "${PAGEBASE}-03.${IMGEXT}"
    rm "${PAGE}"
done

echo "Writing the HTML file..."
cat > "${FILEBASE}.html" <<HEAD
<html><head><guide></guide></head><body>
HEAD
for IMG in "${FILEBASE}"*.${IMGEXT}
  do
    echo "<img src='${IMG}'/><mbp:pagebreak/>" >> "${FILEBASE}.html"
  done
echo "</body></html>" >> "${FILEBASE}.html"

echo "Converting HTML to MOBI..."
html2mobi --title "${FILEBASE}" --author "ME" "${FILEBASE}.html"

#echo "Cleaning up..."
rm "${FILEBASE}.html" "${FILEBASE}"*.${IMGEXT}
done

Please share any further optimizations and thoughts... The main current issue with this program is that it can not handle very long PDF documents because ImageMagick chokes on 100s of pages to convert at once. I have not had time to make it read and convert to jpg one page at a time, but it should be relatively trivial.

Good luck and enjoy!

myelinator · 02-09-2008, 10:42 AM

Realized that I'd left out the four pixel image you need for mapping the greyscale: kindle.gif. Yes, it is that little speck you see in the Attached images box (use your magnifying glass if necessary!) - use your browser to download it to the same directory as the pdf2mobi script.

Moderator: edited to change attachment to ZIP to allow downloading

lovebeta · 02-09-2008, 01:50 PM

It's weird. From my experiment with Mobipocket creator, it seems that the largest img dimension is 500*640. Is it a restriction of Mobicreator or kindle?

tompe · 02-09-2008, 02:31 PM

Quote:

Originally Posted by myelinator

This program relies on two other pieces of software:

Mobiperl (https://www.mobileread.com/forums/sho...&highlight=prc) and its dependencies (perl, etc.)
ImageMagick (http://www.imagemagick.org/)

After you have downloaded Mobiperl you need to make modifications to the source perl to optimize it for use with the Kindle. In the file [...your perl libraries...]/MobiPerl/Util.pm in the subroutine get_image_data() change the lines that read:

Code:

my $maxwidth = 480;
my $maxheight = 640;

to:

Code:

my $maxwidth = 525;
my $maxheight = 700;

So you got Mobiperl to work on a Mac. Was there any problem with the Perl code? And could you write something about how to install Perl and the required modules on a Mac that I could put on the web page for Mobiperl?

This 480x640 limit is because of a bug in the Cybook. When I try to use larger images then sometimes the file cannot be read. So to be safe for all devices I have this limit until Bookeen fix their bug.

myelinator · 02-09-2008, 02:46 PM

Must be mobipocket - because I have checked screenshots that I got from the Kindle and am nearly certain that the readable area is 525 x 700. I have had no problem with the books I've made with Mobiperl (all of which are inserting the images at that dimension without scaling).

myelinator · 02-09-2008, 03:26 PM

tompe: thanks for making your MobiPerl collection - the perl code runs very smoothly - it was also very easy to find the necessary lines to change because your code is easily read (but maybe we could have a little switch to change the size from the default

) - Mac OS X comes with perl already installed (v5.8.6 under /usr/bin) - I don't know if you need v5.8.8 installed for your scripts to work because I already had v5.8.8 installed.

If someone needs to install perl 5.8.8 on their machine for it to work the following would need to be done:

Install macports http://www.macports.org/

Use macports to obtain perl in Terminal.app

Code:

$ sudo port -v autoupdate
$ sudo port -v install perl5.8

This should have placed perl in /opt/local/bin (and I believe changed the paths properly to find it) you can test by:
Code:
```
$ which perl        # tells you what directory it is in
$ perl -version     # tells you what version is active
```

Now download the latest tarball from the mobiperl website (currently 0.0.27), decompress it, and move the libraries into the distribution (on my machine using macports - /opt/local/lib/perl5/site_perl/5.8.8/)

Code:

# there maybe a friendlier way to do this but this works
# do this in the directory where you downloaded the mobiperl tar
$ tar xv mobiperl-0.0.27.tar
$ cd mobiperl-0.0.27
$ sudo cp Palm MobiPerl /opt/local/lib/perl5/site_perl/5.8.8

(optional) put the mobi2mobi, etc. in good places

Code:

$ sudo cp lit2mobi mobi2mobi html2mobi mobi2html opf2mobi /usr/bin

I typed this up pretty quick and wasn't actually doing the install at that moment - so everyone doing this should read it for themselves and make sure it looks good (esp the sudo commands)

Thanks again tompe!

lovebeta · 02-09-2008, 10:55 PM

Very interesting post, myelinator. I'll defintely give it a try.

tompe, so what's the largest img dimension in Cybook? I was baffled by the on screen controls in the kindle, which took so much precious real estate.

mateo · 02-09-2008, 11:30 PM

Interesting. I haven't tried to tackle this, so tell me if i'm wrong, but wouldn't it be easier just to convert your html to latex and the use pdflatex? Do those PDFs not show up correctly on the Kindle?

lovebeta · 02-09-2008, 11:54 PM

I just tried again and it still looked to be 500x640 to me.

The physical size of kindle is roughly 90mm x 120 mm. So in order to display an image of 525x700 natively, the picture has to appear roughly 78.75mm x 106.75mm. However the largest picture (in a prc file) I have ever seen is roughly 74mm x 97mm. That's why I believe the largest pic dimension on the kindle is 500x640. Please correct me if I am wrong.

wallcraft · 02-10-2008, 02:04 AM

The maximum image size in a MOBI file on the Kindle appears to be 640 by 520. I enclose an example set of images in a MOBI file, and Kindle screenshots. The 6 images are:

404x618
480x640
525x700
500x640
520x640
540x640

The original is 404x618 and normally the aspect ratio of the image would be conserved, but here I deliberately stretched it to the other listed sizes.

Note that the first five images are the same height (640 pixels) when displayed on the Kindle, which means that the 404x618 image has actually been rescaled to be 640 high. The 525x700 image has definitely been reduced in size. The last image is less high, because 540 pixels is rescaled to 520 which reduces the hieght to maintain the aspect ratio.

The .zip file contains the six images (last 5 generated from the 1st using IrfanView), a HTML wrapper and a .mobi file created with html2mobi.

tompe · 02-10-2008, 04:18 AM

Quote:

Originally Posted by lovebeta

tompe, so what's the largest img dimension in Cybook? I was baffled by the on screen controls in the kindle, which took so much precious real estate.

You can use any size and it is reduced when shown on the screen. A cover image covers the full screnn so I assume it is reduced to 600x800. I have not checked to what size an image inside the book is reduced to.

tompe · 02-10-2008, 04:21 AM

Quote:

Originally Posted by wallcraft

The .zip file contains the six images (last 5 generated from the 1st using IrfanView), a HTML wrapper and a .mobi file created with html2mobi.

Did you disable html2mobi's rescaling of the images in this procedure?

myelinator · 02-10-2008, 12:24 PM

Quote:

Originally Posted by wallcraft

The maximum image size in a MOBI file on the Kindle appears to be 640 by 520.

You are right, wallcraft, the maximum image is 640x520. I'm glad that we have the right number now (it makes my document conversion look even better, thanks!). What a waste of screenspace (and an odd aspect ratio 16:13)!

Apologies to lovebeta - I thought I was seeing something that I wasn't. I was confused by the margins on the articles I was converting using this technique (before I started trimming them off).

Anyway the correct script code would now be:

Code:

#!/bin/bash

for FILE in "$@"; do
FILEPATH=${FILE%/*}
FILENAME=${FILE##*/}
FILEBASE=${FILENAME%%.*}
FILEEXT=${FILENAME#*.}
IMGEXT=jpg

# if my experiments are right then the size of the Kindle screen is 525x700
# (a 4:3 aspect ratio) as one might expect

# use imagemagick to do all the needed conversions:
# -density              166x166 to get the right dpi
# -trim                 off the margins (do it before you resize!)
# -resize 700           to get the width of the pages maximized
# -rotate -90           to maximize the use of the screen
# -colorspace gray      self explanatory
# -type Grayscale       self explanatory
# -level                darken the image
# +dither               turn off dither
# -level 34%,100%,1.2   darken the image
# -map kindle.gif       a four color gif with the allowed shades of gray
echo "**********************************************************************"
echo "* Converting to $FILE to $IMGEXT files... please be patient..."
echo "**********************************************************************"
convert -density 166x166  "${FILEBASE}.pdf" -trim +repage -resize 640 -rotate -90 -colorspace gray -type Grayscale -level 34%,100%,1.2 +dither -map kindle.gif "${FILEBASE}.${IMGEXT}"

echo "Dividing the $IMGEXT page images..."
for PAGE in "${FILEBASE}"*.${IMGEXT}
  do
    PAGEBASE=${PAGE%%.*}
    echo $PAGEBASE
    convert "${PAGEBASE}.${IMGEXT}" -crop 520x640+0 +repage "${PAGEBASE}-01.${IMGEXT}"
    convert "${PAGEBASE}.${IMGEXT}" -crop 520x640+450 +repage "${PAGEBASE}-02.${IMGEXT}"
    convert "${PAGEBASE}.${IMGEXT}" -crop 520x640+900 +repage "${PAGEBASE}-03.${IMGEXT}"
    rm "${PAGE}"
done

echo "Writing the HTML file..."
cat > "${FILEBASE}.html" <<HEAD
<html><head><guide></guide></head><body>
HEAD
for IMG in "${FILEBASE}"*.${IMGEXT}
  do
    echo "<img src='${IMG}'/><mbp:pagebreak/>" >> "${FILEBASE}.html"
  done
echo "</body></html>" >> "${FILEBASE}.html"

echo "Converting HTML to MOBI..."
html2mobi --title "${FILEBASE}" --author "BBB" "${FILEBASE}.html"

#echo "Cleaning up..."
rm "${FILEBASE}.html" "${FILEBASE}"*.${IMGEXT}
done

You probably should also modify MobiPerl/Util.pm again to reflect the 640x520 dimension when you are converting other documents with html2mobi, etc.

myelinator · 02-10-2008, 01:00 PM

I should also probably add two points about the way the script currently works:

I attempted to optimize the ImageMagick command for text - it will not work well for images because the dither is turned off to keep the text from bleeding (especially on a screened background)
You will note that the minipages overlap - I did this on purpose because I can't cut the images like Amazon does (at a place where there is a horizontal blank area) because most of my documents don't have such an area (lines between columns, oddly placed figures) -- this was acutally one of my motivations for this because the amazon conversion didn't work as well for me on these types of documents. If you want to reduce the overlap for your personal tastes the crop commands, e.g., changing:
Code:
```
-crop 520x640+450
```
to
Code:
```
-crop 520x640+520
```
would produce no overlap between the first and second images (adjust the third accordingly as needed)

I hope some of these comments will help some of the tinkerers...

wallcraft · 02-10-2008, 01:58 PM

Quote:

Originally Posted by tompe

Did you disable html2mobi's rescaling of the images in this procedure?

No, and so the last 5 images in the original MOBI file are all 480 pixels wide. Fortunately, this error was self correcting (because html2mobi and the Kindle both maintain image aspect ratios) and the 520x640 limit seems to be right. I attach the last 2 images (520x640 and 540x640) from a MOBI created using mobigen - which does maintain the original image sizes. These end up the same size on the Kindle as the original versions. They should in principle have higher quality (one less rescaling step), although this isn't all that apparent.

02-09-2008, 03:26 PM	#6
myelinator Junior Member Posts: 6 Karma: 10 Join Date: Feb 2008 Device: kindle	tompe: thanks for making your MobiPerl collection - the perl code runs very smoothly - it was also very easy to find the necessary lines to change because your code is easily read (but maybe we could have a little switch to change the size from the default ) - Mac OS X comes with perl already installed (v5.8.6 under /usr/bin) - I don't know if you need v5.8.8 installed for your scripts to work because I already had v5.8.8 installed. If someone needs to install perl 5.8.8 on their machine for it to work the following would need to be done: Install macports http://www.macports.org/ Use macports to obtain perl in Terminal.app Code: $ sudo port -v autoupdate $ sudo port -v install perl5.8 This should have placed perl in /opt/local/bin (and I believe changed the paths properly to find it) you can test by: Code: $ which perl # tells you what directory it is in $ perl -version # tells you what version is active Now download the latest tarball from the mobiperl website (currently 0.0.27), decompress it, and move the libraries into the distribution (on my machine using macports - /opt/local/lib/perl5/site_perl/5.8.8/) Code: # there maybe a friendlier way to do this but this works # do this in the directory where you downloaded the mobiperl tar $ tar xv mobiperl-0.0.27.tar $ cd mobiperl-0.0.27 $ sudo cp Palm MobiPerl /opt/local/lib/perl5/site_perl/5.8.8 (optional) put the mobi2mobi, etc. in good places Code: $ sudo cp lit2mobi mobi2mobi html2mobi mobi2html opf2mobi /usr/bin I typed this up pretty quick and wasn't actually doing the install at that moment - so everyone doing this should read it for themselves and make sure it looks good (esp the sudo commands) Thanks again tompe!

02-10-2008, 01:00 PM	#14
myelinator Junior Member Posts: 6 Karma: 10 Join Date: Feb 2008 Device: kindle	I should also probably add two points about the way the script currently works: I attempted to optimize the ImageMagick command for text - it will not work well for images because the dither is turned off to keep the text from bleeding (especially on a screened background) You will note that the minipages overlap - I did this on purpose because I can't cut the images like Amazon does (at a place where there is a horizontal blank area) because most of my documents don't have such an area (lines between columns, oddly placed figures) -- this was acutally one of my motivations for this because the amazon conversion didn't work as well for me on these types of documents. If you want to reduce the overlap for your personal tastes the crop commands, e.g., changing: Code: -crop 520x640+450 to Code: -crop 520x640+520 would produce no overlap between the first and second images (adjust the third accordingly as needed) I hope some of these comments will help some of the tinkerers... Last edited by myelinator; 02-10-2008 at 01:01 PM. Reason: typos

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Formatting SD card for Boox on Mac OS X	pietvo	Onyx Boox	8	03-17-2010 10:30 AM
PDFs, metadata and conversion on a Mac	iain_benson	Sony Reader	2	01-19-2009 03:51 AM
Convert PDFs WITH formatting	njustn	Amazon Kindle	8	06-05-2008 01:10 PM
Mac OS X and PPDs/PDFs	mrdini	Workshop	2	03-27-2008 03:00 PM
Best formatting for PDF creation on a mac	TND	iRex	5	09-01-2006 02:11 AM

02-09-2008, 01:50 PM	#3
lovebeta Groupie Posts: 176 Karma: 406 Join Date: Jan 2008 Device: Amazon Kindle 2, Amazon Kindle, Sony PRS-505	It's weird. From my experiment with Mobipocket creator, it seems that the largest img dimension is 500*640. Is it a restriction of Mobicreator or kindle?

02-09-2008, 02:46 PM	#5
myelinator Junior Member Posts: 6 Karma: 10 Join Date: Feb 2008 Device: kindle	Must be mobipocket - because I have checked screenshots that I got from the Kindle and am nearly certain that the readable area is 525 x 700. I have had no problem with the books I've made with Mobiperl (all of which are inserting the images at that dimension without scaling).

02-09-2008, 10:55 PM	#7
lovebeta Groupie Posts: 176 Karma: 406 Join Date: Jan 2008 Device: Amazon Kindle 2, Amazon Kindle, Sony PRS-505	Very interesting post, myelinator. I'll defintely give it a try. tompe, so what's the largest img dimension in Cybook? I was baffled by the on screen controls in the kindle, which took so much precious real estate.

02-09-2008, 11:30 PM	#8
mateo Enthusiast Posts: 45 Karma: 10 Join Date: May 2005 Device: Palm Zire71	Interesting. I haven't tried to tackle this, so tell me if i'm wrong, but wouldn't it be easier just to convert your html to latex and the use pdflatex? Do those PDFs not show up correctly on the Kindle?

02-09-2008, 11:54 PM	#9
lovebeta Groupie Posts: 176 Karma: 406 Join Date: Jan 2008 Device: Amazon Kindle 2, Amazon Kindle, Sony PRS-505	I just tried again and it still looked to be 500x640 to me. The physical size of kindle is roughly 90mm x 120 mm. So in order to display an image of 525x700 natively, the picture has to appear roughly 78.75mm x 106.75mm. However the largest picture (in a prc file) I have ever seen is roughly 74mm x 97mm. That's why I believe the largest pic dimension on the kindle is 500x640. Please correct me if I am wrong.

Advert

Advert