Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Readers > Amazon Kindle

Notices

Reply
 
Thread Tools Search this Thread
Old 02-09-2008, 11:11 AM   #1
myelinator
Junior Member
myelinator began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Feb 2008
Device: kindle
Lightbulb All on the Mac: PDFs with Formatting!

Hello, all.

I noticed that there were many people who were struggling to find a way to make formatted PDFs for their Kindle with their Mac without resorting to MobiPocket creator. I needed to do the same thing and was getting frustrated with being forced to use an emulator. Here is the method I came up with. It is rather technical and requires some UNIX terminal knowledge and is just my quick and dirty homebrew method. However, it took a signficant amount of time to develop and I wanted to make sure to share it with others.

Please understand that I am extremely busy and don't get to spend as much time doing this as I might like. (I've been using this for over a month and just now have had a moment to sit down and type this.) Therefore, I may or may not have time to answer questions, but I've tried to be as clear as I can.

This program relies on two other pieces of software:
  1. Mobiperl (http://www.mobileread.com/forums/sho...&highlight=prc) and its dependencies (perl, etc.)
  2. ImageMagick (http://www.imagemagick.org/)

After you have downloaded Mobiperl you need to make modifications to the source perl to optimize it for use with the Kindle. In the file [...your perl libraries...]/MobiPerl/Util.pm in the subroutine get_image_data() change the lines that read:
Code:
my $maxwidth = 480;
my $maxheight = 640;
to:
Code:
my $maxwidth = 525;
my $maxheight = 700;
Now you are ready for the shell script that puts it all together (I named it pdf2mobi):
Code:
#!/bin/bash

for FILE in "$@"; do
FILEPATH=${FILE%/*}
FILENAME=${FILE##*/}
FILEBASE=${FILENAME%%.*}
FILEEXT=${FILENAME#*.}
IMGEXT=jpg

# if my experiments are right then the size of the Kindle screen is 525x700
# (a 4:3 aspect ratio) as one might expect

# use imagemagick to do all the needed conversions:
# -density              166x166 to get the right dpi
# -trim                 off the margins (do it before you resize!)
# -resize 700           to get the width of the pages maximized
# -rotate -90           to maximize the use of the screen
# -colorspace gray      self explanatory
# -type Grayscale       self explanatory
# -level                darken the image
# +dither               turn off dither
# -level 34%,100%,1.2   darken the image
# -map kindle.gif       a four color gif with the allowed shades of gray
echo "**********************************************************************"
echo "* Converting to $FILE to $IMGEXT files... please be patient..."
echo "**********************************************************************"
convert -density 166x166  "${FILEBASE}.pdf" -trim +repage -resize 700 -rotate -90 -colorspace gray -type Grayscale -level 34%,100%,1.2 +dither -map kindle.gif "${FILEBASE}.${IMGEXT}"

echo "Dividing the $IMGEXT page images..."
for PAGE in "${FILEBASE}"*.${IMGEXT}
  do
    PAGEBASE=${PAGE%%.*}
    echo $PAGEBASE
    convert "${PAGEBASE}.${IMGEXT}" -crop 525x700+0 +repage "${PAGEBASE}-01.${IMGEXT}"
    convert "${PAGEBASE}.${IMGEXT}" -crop 525x700+450 +repage "${PAGEBASE}-02.${IMGEXT}"
    convert "${PAGEBASE}.${IMGEXT}" -crop 525x700+900 +repage "${PAGEBASE}-03.${IMGEXT}"
    rm "${PAGE}"
done

echo "Writing the HTML file..."
cat > "${FILEBASE}.html" <<HEAD
<html><head><guide></guide></head><body>
HEAD
for IMG in "${FILEBASE}"*.${IMGEXT}
  do
    echo "<img src='${IMG}'/><mbp:pagebreak/>" >> "${FILEBASE}.html"
  done
echo "</body></html>" >> "${FILEBASE}.html"

echo "Converting HTML to MOBI..."
html2mobi --title "${FILEBASE}" --author "ME" "${FILEBASE}.html"

#echo "Cleaning up..."
rm "${FILEBASE}.html" "${FILEBASE}"*.${IMGEXT}
done
Please share any further optimizations and thoughts... The main current issue with this program is that it can not handle very long PDF documents because ImageMagick chokes on 100s of pages to convert at once. I have not had time to make it read and convert to jpg one page at a time, but it should be relatively trivial.

Good luck and enjoy!
myelinator is offline   Reply With Quote
Old 02-09-2008, 11:42 AM   #2
myelinator
Junior Member
myelinator began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Feb 2008
Device: kindle
kindle.gif

Realized that I'd left out the four pixel image you need for mapping the greyscale: kindle.gif. Yes, it is that little speck you see in the Attached images box (use your magnifying glass if necessary!) - use your browser to download it to the same directory as the pdf2mobi script.

Moderator: edited to change attachment to ZIP to allow downloading
Attached Files
File Type: zip kindle.zip (920 Bytes, 435 views)

Last edited by myelinator; 02-09-2008 at 11:43 AM. Reason: typo
myelinator is offline   Reply With Quote
Old 02-09-2008, 02:50 PM   #3
lovebeta
Groupie
lovebeta has a complete set of Star Wars action figures.lovebeta has a complete set of Star Wars action figures.lovebeta has a complete set of Star Wars action figures.lovebeta has a complete set of Star Wars action figures.lovebeta has a complete set of Star Wars action figures.
 
Posts: 176
Karma: 406
Join Date: Jan 2008
Device: Amazon Kindle 2, Amazon Kindle, Sony PRS-505
It's weird. From my experiment with Mobipocket creator, it seems that the largest img dimension is 500*640. Is it a restriction of Mobicreator or kindle?
lovebeta is offline   Reply With Quote
Old 02-09-2008, 03:31 PM   #4
tompe
Grand Sorcerer
tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.
 
Posts: 7,129
Karma: 4315826
Join Date: Oct 2007
Location: Link÷png, Sweden
Device: Nexus 7, Nexus 5, iPad 2, Kindle PW
Quote:
Originally Posted by myelinator View Post
This program relies on two other pieces of software:
  1. Mobiperl (http://www.mobileread.com/forums/sho...&highlight=prc) and its dependencies (perl, etc.)
  2. ImageMagick (http://www.imagemagick.org/)

After you have downloaded Mobiperl you need to make modifications to the source perl to optimize it for use with the Kindle. In the file [...your perl libraries...]/MobiPerl/Util.pm in the subroutine get_image_data() change the lines that read:
Code:
my $maxwidth = 480;
my $maxheight = 640;
to:
Code:
my $maxwidth = 525;
my $maxheight = 700;
So you got Mobiperl to work on a Mac. Was there any problem with the Perl code? And could you write something about how to install Perl and the required modules on a Mac that I could put on the web page for Mobiperl?

This 480x640 limit is because of a bug in the Cybook. When I try to use larger images then sometimes the file cannot be read. So to be safe for all devices I have this limit until Bookeen fix their bug.
tompe is offline   Reply With Quote
Old 02-09-2008, 03:46 PM   #5
myelinator
Junior Member
myelinator began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Feb 2008
Device: kindle
Must be mobipocket - because I have checked screenshots that I got from the Kindle and am nearly certain that the readable area is 525 x 700. I have had no problem with the books I've made with Mobiperl (all of which are inserting the images at that dimension without scaling).
myelinator is offline   Reply With Quote
Old 02-09-2008, 04:26 PM   #6
myelinator
Junior Member
myelinator began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Feb 2008
Device: kindle
tompe: thanks for making your MobiPerl collection - the perl code runs very smoothly - it was also very easy to find the necessary lines to change because your code is easily read (but maybe we could have a little switch to change the size from the default ) - Mac OS X comes with perl already installed (v5.8.6 under /usr/bin) - I don't know if you need v5.8.8 installed for your scripts to work because I already had v5.8.8 installed.

If someone needs to install perl 5.8.8 on their machine for it to work the following would need to be done:
  1. Install macports http://www.macports.org/
  2. Use macports to obtain perl in Terminal.app
    Code:
    $ sudo port -v autoupdate
    $ sudo port -v install perl5.8
  3. This should have placed perl in /opt/local/bin (and I believe changed the paths properly to find it) you can test by:
    Code:
    $ which perl        # tells you what directory it is in
    $ perl -version     # tells you what version is active
  4. Now download the latest tarball from the mobiperl website (currently 0.0.27), decompress it, and move the libraries into the distribution (on my machine using macports - /opt/local/lib/perl5/site_perl/5.8.8/)
    Code:
    # there maybe a friendlier way to do this but this works
    # do this in the directory where you downloaded the mobiperl tar
    $ tar xv mobiperl-0.0.27.tar
    $ cd mobiperl-0.0.27
    $ sudo cp Palm MobiPerl /opt/local/lib/perl5/site_perl/5.8.8
  5. (optional) put the mobi2mobi, etc. in good places
    Code:
    $ sudo cp lit2mobi mobi2mobi html2mobi mobi2html opf2mobi /usr/bin

I typed this up pretty quick and wasn't actually doing the install at that moment - so everyone doing this should read it for themselves and make sure it looks good (esp the sudo commands)

Thanks again tompe!
myelinator is offline   Reply With Quote
Old 02-09-2008, 11:55 PM   #7
lovebeta
Groupie
lovebeta has a complete set of Star Wars action figures.lovebeta has a complete set of Star Wars action figures.lovebeta has a complete set of Star Wars action figures.lovebeta has a complete set of Star Wars action figures.lovebeta has a complete set of Star Wars action figures.
 
Posts: 176
Karma: 406
Join Date: Jan 2008
Device: Amazon Kindle 2, Amazon Kindle, Sony PRS-505
Very interesting post, myelinator. I'll defintely give it a try.

tompe, so what's the largest img dimension in Cybook? I was baffled by the on screen controls in the kindle, which took so much precious real estate.
lovebeta is offline   Reply With Quote
Old 02-10-2008, 12:30 AM   #8
mateo
Enthusiast
mateo began at the beginning.
 
Posts: 45
Karma: 10
Join Date: May 2005
Device: Palm Zire71
Interesting. I haven't tried to tackle this, so tell me if i'm wrong, but wouldn't it be easier just to convert your html to latex and the use pdflatex? Do those PDFs not show up correctly on the Kindle?
mateo is offline   Reply With Quote
Old 02-10-2008, 12:54 AM   #9
lovebeta
Groupie
lovebeta has a complete set of Star Wars action figures.lovebeta has a complete set of Star Wars action figures.lovebeta has a complete set of Star Wars action figures.lovebeta has a complete set of Star Wars action figures.lovebeta has a complete set of Star Wars action figures.
 
Posts: 176
Karma: 406
Join Date: Jan 2008
Device: Amazon Kindle 2, Amazon Kindle, Sony PRS-505
I just tried again and it still looked to be 500x640 to me.

The physical size of kindle is roughly 90mm x 120 mm. So in order to display an image of 525x700 natively, the picture has to appear roughly 78.75mm x 106.75mm. However the largest picture (in a prc file) I have ever seen is roughly 74mm x 97mm. That's why I believe the largest pic dimension on the kindle is 500x640. Please correct me if I am wrong.
lovebeta is offline   Reply With Quote
Old 02-10-2008, 03:04 AM   #10
wallcraft
reader
wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.
 
wallcraft's Avatar
 
Posts: 6,979
Karma: 5183568
Join Date: Mar 2006
Location: Mississippi, USA
Device: Kindle 3 and Fire
The maximum image size in a MOBI file on the Kindle appears to be 640 by 520. I enclose an example set of images in a MOBI file, and Kindle screenshots. The 6 images are:
  1. 404x618
  2. 480x640
  3. 525x700
  4. 500x640
  5. 520x640
  6. 540x640
The original is 404x618 and normally the aspect ratio of the image would be conserved, but here I deliberately stretched it to the other listed sizes.

Note that the first five images are the same height (640 pixels) when displayed on the Kindle, which means that the 404x618 image has actually been rescaled to be 640 high. The 525x700 image has definitely been reduced in size. The last image is less high, because 540 pixels is rescaled to 520 which reduces the hieght to maintain the aspect ratio.

The .zip file contains the six images (last 5 generated from the 1st using IrfanView), a HTML wrapper and a .mobi file created with html2mobi.
Attached Thumbnails
Click image for larger version

Name:	KSS_PA404x618.gif
Views:	401
Size:	36.3 KB
ID:	10154   Click image for larger version

Name:	KSS_PA480x640.gif
Views:	382
Size:	40.0 KB
ID:	10155   Click image for larger version

Name:	KSS_PA525x700.gif
Views:	365
Size:	40.1 KB
ID:	10156   Click image for larger version

Name:	KSS_PA500x640.gif
Views:	374
Size:	40.3 KB
ID:	10157   Click image for larger version

Name:	KSS_PA520x640.gif
Views:	381
Size:	41.5 KB
ID:	10158   Click image for larger version

Name:	KSS_PA540x640.gif
Views:	384
Size:	40.4 KB
ID:	10159  
Attached Files
File Type: zip PineApple.zip (528.6 KB, 351 views)
wallcraft is offline   Reply With Quote
Old 02-10-2008, 05:18 AM   #11
tompe
Grand Sorcerer
tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.
 
Posts: 7,129
Karma: 4315826
Join Date: Oct 2007
Location: Link÷png, Sweden
Device: Nexus 7, Nexus 5, iPad 2, Kindle PW
Quote:
Originally Posted by lovebeta View Post
tompe, so what's the largest img dimension in Cybook? I was baffled by the on screen controls in the kindle, which took so much precious real estate.
You can use any size and it is reduced when shown on the screen. A cover image covers the full screnn so I assume it is reduced to 600x800. I have not checked to what size an image inside the book is reduced to.
tompe is offline   Reply With Quote
Old 02-10-2008, 05:21 AM   #12
tompe
Grand Sorcerer
tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.tompe ought to be getting tired of karma fortunes by now.
 
Posts: 7,129
Karma: 4315826
Join Date: Oct 2007
Location: Link÷png, Sweden
Device: Nexus 7, Nexus 5, iPad 2, Kindle PW
Quote:
Originally Posted by wallcraft View Post
The .zip file contains the six images (last 5 generated from the 1st using IrfanView), a HTML wrapper and a .mobi file created with html2mobi.
Did you disable html2mobi's rescaling of the images in this procedure?
tompe is offline   Reply With Quote
Old 02-10-2008, 01:24 PM   #13
myelinator
Junior Member
myelinator began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Feb 2008
Device: kindle
Quote:
Originally Posted by wallcraft View Post
The maximum image size in a MOBI file on the Kindle appears to be 640 by 520.
You are right, wallcraft, the maximum image is 640x520. I'm glad that we have the right number now (it makes my document conversion look even better, thanks!). What a waste of screenspace (and an odd aspect ratio 16:13)!

Apologies to lovebeta - I thought I was seeing something that I wasn't. I was confused by the margins on the articles I was converting using this technique (before I started trimming them off).

Anyway the correct script code would now be:
Code:
#!/bin/bash

for FILE in "$@"; do
FILEPATH=${FILE%/*}
FILENAME=${FILE##*/}
FILEBASE=${FILENAME%%.*}
FILEEXT=${FILENAME#*.}
IMGEXT=jpg

# if my experiments are right then the size of the Kindle screen is 525x700
# (a 4:3 aspect ratio) as one might expect

# use imagemagick to do all the needed conversions:
# -density              166x166 to get the right dpi
# -trim                 off the margins (do it before you resize!)
# -resize 700           to get the width of the pages maximized
# -rotate -90           to maximize the use of the screen
# -colorspace gray      self explanatory
# -type Grayscale       self explanatory
# -level                darken the image
# +dither               turn off dither
# -level 34%,100%,1.2   darken the image
# -map kindle.gif       a four color gif with the allowed shades of gray
echo "**********************************************************************"
echo "* Converting to $FILE to $IMGEXT files... please be patient..."
echo "**********************************************************************"
convert -density 166x166  "${FILEBASE}.pdf" -trim +repage -resize 640 -rotate -90 -colorspace gray -type Grayscale -level 34%,100%,1.2 +dither -map kindle.gif "${FILEBASE}.${IMGEXT}"

echo "Dividing the $IMGEXT page images..."
for PAGE in "${FILEBASE}"*.${IMGEXT}
  do
    PAGEBASE=${PAGE%%.*}
    echo $PAGEBASE
    convert "${PAGEBASE}.${IMGEXT}" -crop 520x640+0 +repage "${PAGEBASE}-01.${IMGEXT}"
    convert "${PAGEBASE}.${IMGEXT}" -crop 520x640+450 +repage "${PAGEBASE}-02.${IMGEXT}"
    convert "${PAGEBASE}.${IMGEXT}" -crop 520x640+900 +repage "${PAGEBASE}-03.${IMGEXT}"
    rm "${PAGE}"
done

echo "Writing the HTML file..."
cat > "${FILEBASE}.html" <<HEAD
<html><head><guide></guide></head><body>
HEAD
for IMG in "${FILEBASE}"*.${IMGEXT}
  do
    echo "<img src='${IMG}'/><mbp:pagebreak/>" >> "${FILEBASE}.html"
  done
echo "</body></html>" >> "${FILEBASE}.html"

echo "Converting HTML to MOBI..."
html2mobi --title "${FILEBASE}" --author "BBB" "${FILEBASE}.html"

#echo "Cleaning up..."
rm "${FILEBASE}.html" "${FILEBASE}"*.${IMGEXT}
done
You probably should also modify MobiPerl/Util.pm again to reflect the 640x520 dimension when you are converting other documents with html2mobi, etc.
myelinator is offline   Reply With Quote
Old 02-10-2008, 02:00 PM   #14
myelinator
Junior Member
myelinator began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Feb 2008
Device: kindle
I should also probably add two points about the way the script currently works:
  • I attempted to optimize the ImageMagick command for text - it will not work well for images because the dither is turned off to keep the text from bleeding (especially on a screened background)
  • You will note that the minipages overlap - I did this on purpose because I can't cut the images like Amazon does (at a place where there is a horizontal blank area) because most of my documents don't have such an area (lines between columns, oddly placed figures) -- this was acutally one of my motivations for this because the amazon conversion didn't work as well for me on these types of documents. If you want to reduce the overlap for your personal tastes the crop commands, e.g., changing:
    Code:
    -crop 520x640+450
    to
    Code:
    -crop 520x640+520
    would produce no overlap between the first and second images (adjust the third accordingly as needed)

I hope some of these comments will help some of the tinkerers...

Last edited by myelinator; 02-10-2008 at 02:01 PM. Reason: typos
myelinator is offline   Reply With Quote
Old 02-10-2008, 02:58 PM   #15
wallcraft
reader
wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.wallcraft ought to be getting tired of karma fortunes by now.
 
wallcraft's Avatar
 
Posts: 6,979
Karma: 5183568
Join Date: Mar 2006
Location: Mississippi, USA
Device: Kindle 3 and Fire
Quote:
Originally Posted by tompe View Post
Did you disable html2mobi's rescaling of the images in this procedure?
No, and so the last 5 images in the original MOBI file are all 480 pixels wide. Fortunately, this error was self correcting (because html2mobi and the Kindle both maintain image aspect ratios) and the 520x640 limit seems to be right. I attach the last 2 images (520x640 and 540x640) from a MOBI created using mobigen - which does maintain the original image sizes. These end up the same size on the Kindle as the original versions. They should in principle have higher quality (one less rescaling step), although this isn't all that apparent.
Attached Thumbnails
Click image for larger version

Name:	KSS_PA520x640MG.gif
Views:	412
Size:	41.2 KB
ID:	10176   Click image for larger version

Name:	KSS_PA540x640MG.gif
Views:	422
Size:	39.8 KB
ID:	10177  
wallcraft is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Formatting SD card for Boox on Mac OS X pietvo Onyx Boox 8 03-17-2010 11:30 AM
PDFs, metadata and conversion on a Mac iain_benson Sony Reader 2 01-19-2009 04:51 AM
Convert PDFs WITH formatting njustn Amazon Kindle 8 06-05-2008 02:10 PM
Mac OS X and PPDs/PDFs mrdini Workshop 2 03-27-2008 04:00 PM
Best formatting for PDF creation on a mac TND iRex 5 09-01-2006 03:11 AM


All times are GMT -4. The time now is 11:13 PM.


MobileRead.com is a privately owned, operated and funded community.