Tutorial: Formulas to PNG

[Tex2002ans]

Programs that are required for this tutorial:

• LibreOffice Math: https://www.libreoffice.org/download/
  • Side Note: These steps should all be applicable to OpenOffice Math as well.
• Inkscape: http://inkscape.org/download/?lang=en
• GIMP: http://www.gimp.org/downloads/
  • Side Note: Any image program should be applicable, but this tutorial will specifically be using GIMP.
• Side Note: I settled on these three programs because they are open source, and these instructions should work on nearly any OS.

The problem:

You have finished converting a PDF to EPUB. The entire book is crisp, clear, digital font. BEAUTIFUL. And then you stumble upon a formula taken right from the original scan!

Usually the images of formulas are:

• JPG
  • If you see this, RUN FOR THE HILLS!
• The Resolution is Miniscule
  • Even ants cannot read this
• Have horrible scanning artifacts.

Compare these two images:

Before:
After:

Side Note: The "Before" picture is NOWHERE NEAR the horror that you will typically see.

Ultimately, you would want something that scales beautifully and according to user preferences, like MathML. Sadly, you cannot use MathML because this EPUB must be read on a multitude of different readers.

Something like SVG would be great, but it is quite complex, and can only be used in EPUB (and KF8).

You want this to be read on early Kindles (and older devices) as well, so you must settle on an image. This is where the high resolution PNG comes to save the day!

At the end of the tutorial:

• You will have a high resolution, beautiful formula in PNG.
  • Since it is an image, this should work on all devices (EPUB, Kindle), and look quite clean big/small (PC/smartphone).
• You will have a digital version of the formula in PDF form.
  • If your company ever does create a completely new Edition of the book. This digital formula will easily be able to fit right in.
  • You can generate ANY size/format image you need from this.
  • When MathML becomes more ubiquitous, it should be much easier to go from Libre Math to MathML than from a low resolution image to MathML.
• No more hideous ant-sized formulas!!!

Step -1: The Source

This tutorial will be using pages 160, 174, and page 180 from "An Outline of International Price Theory" by Chi-Yuen Wu:



PDF: http://library.mises.org/books/Chi-Y...e%20Theory.pdf

I have attached an EPUB at the end of the tutorial which includes the final text + formulas for these three pages.

Step 0: Very basics of LibreOffice Math

I have marked three sections in the image, showing the Math UI:



Section 1: This is where you use buttons to create the formula. The dropdown menu allows you to change categories to many many different types.

Side Note: Press a button and see what it does! Try to see if you can go through the menus and make a "square root".

Section 2: This is where the formula is displayed.

Section 3: This is where the "code" goes to create a formula. Pressing a button in Section 1 will generate some "code" in Section 3. You will then have to replace the placeholder text with your actual numbers/letters. See:



Now replace the first <?> with 2, and the second <?> with 2. (Or you could just type "2 + 2" directly into Section 3).

2 + 2

Side Note: I tend to wrap each side in squiggly brackets {}. This is EXTREMELY helpful when dealing with more complex equations. Best to start with following good practice right in the beginning!

Equivalent to 2 + 2:

{2} + {2}

Sample that is one step up in complexity:

2 + {a over b}

OR:

{2} + {{a} over {b}}

Step 0.5 (Optional): Change Zoom level in LibreOffice Math:



In the Menu, select View - Zoom.

From here, select whatever preset Zoom level you want (for example 200%). I personally press View - Zoom - Zoom... And then set the level to something large (like 300%, 400%). This will make it easier to see the displayed equation in Section 2.

Note: Zoom level DOES NOT effect the output, only the way that it looks while you are editing in LibreOffice Math.

(Optional) Official LibreOffice Math Help Documentation

Help Documents for LibreOffice Math: https://help.libreoffice.org/Math/We..._the_Math_Help
Complex Formula Examples: https://help.libreoffice.org/Math/Math_Examples

Step 1: Recreate the Formula Using LibreOffice Math:

Every formula you run across will be different. The buttons in Section 1 (see Step 0) will help guide you in creating the actual "code" to place in Section 3 (see Step 0).

Each of the images in this Step show an actual formula from a PDF scan, and the LibreOffice Math code to recreate it:

Formula #1:

{m_a} over {m_b} < {n_a} over {n_b} < {o_a} over {o_b} < {p_a} over {p_b} < ... < {q_a} over {q_b}

Formula #2:

{Q_m over m_b} over {Q_n over n_a} " or " {Q_m over Q_n} times {n_a over m_b}

Formula #3:

p = {pD_{am}} over {D_{am}} = {MP} over {OM} = {ON} over {OM},

Formula #4:

{alignc "Price"} over {alignc "Price-Marginal Revenue"} = {{ON} over {OM}} over {{ON} over {OM} - {tN}over{OM}} = {ON} over {ON - tN} = {ON} over {Ot}

Step 2: LibreOffice Math: Export as PDF

After inserting the code for Formula #1. You want to press File - Export...:



Name your file and press Save.

Side Note: My preferable naming convention is "pg###-Formula#.pdf". Where pg### is the page number of the formula, and Formula# is used if there are multiple formulas per page. This allows for easy organization, and allows it to be easily found/fixed at a later date if needed.



After naming your PDF and pressing Save, you will reach this screen. Make sure "Lossless Compression" and "Tagged PDF" are chosen and then press Export.

[Tex2002ans]

Step 3: Inkscape

When you open Inkscape, you should be greeted with a blank screen. Press File - Open. Go to where you saved your LibreOffice Math PDF files, and open the first one:



You will be greeted with this screen. Just press Ok:



You can now see the PDF that LibreOffice created open in Inkscape.



Now you have to click on the formula, and arrows should appear around it:



Side Note: IF you are greeted with the entire thing surrounded by arrows. Press Object - Ungroup. This should split it into clickable sections. It should now be possible to press on the formula.



Make sure just the formula is selected and has arrows around it, and press File - Export Bitmap...



The Export Bitmap dialog should pop up. This is where you Inkscape what resolution the final PNG will be, and where to save the PNG.

IMPORTANT: Make sure along the top, that the "Selection" button is chosen. Selection means it will create a PNG of everything that is inside the "surrounding arrows".



Personally, I have settled on "320" as a good DPI for formulas. This determines how large the letters/numbers will be in the final PNG (and thus, the EPUB). Feel free to raise/lower this to suit your liking.

You can also manually tell Inkscape which X/Y resolution you want created by changing the "Width" and "Height".

Side Note: I recommend keeping the DPI the same throughout an entire book, so when formula PNGs are scaled to their max width, they will all have the same size font.

After setting the DPI (or Width/Height), press Export. This will now save the file to the location you selected.

Side Note: Inkscape auto-names the file g#####.png. This is quite annoying. Make sure you change to a better filename so you can keep the images organized.

Repeat Step 3 for all formulas until you have 320 DPI PNGs generated for all Formulas.

Step 4: GIMP

Open GIMP, and open the PNG formulas created from Inkscape.



Press Layer - Transparency - Remove Alpha Channel:



This should result in the transparent background turning white:



Since there is no color in this image, you want to change the mode from RGB to Grayscale.

Note: Going from RGB (nearly every color under the sun) to Grayscale will cut down the filesize DRAMATICALLY. "Grayscale" means it will be 256 shades of gray.

Press Image - Mode - Grayscale.



Now we have to save the PNG again. Press File - Export...



Choose where you save your file and what to name it.
IMPORTANT: Make sure that ".png" is at the end of the filename. This will tell GIMP to Export this as a PNG.



Uncheck all the boxes in the dialog box, and make sure Compression is set to max. Press Export.



Repeat Step 4 for all Formulas.

Step 5: Insert into the EPUB

That is all! Now you can insert the PNG into the EPUB just like you would any other image.

I have attached a sample EPUB full of the real world text, with two XHTML files: "Old" and "New". The "Old" has PNG snapshots taken from the PDF, while "New" has the new PNGs generated by this tutorial.

Feel free to compare them both for yourself.

Side Note: I have left an Extra Credit image (the formula at the top of Page 182: See Step -1). See if you can generate a new and improved version using the Tutorial above!

If you need a hint with the Libre Math "code" for the Extra Credit:

Spoiler:

"Barter terms of trade" = {W_a over n_a} div {{W_b times R} over m_b} = W_a over {W_b times R} times {m_b over n_a}"."

[Tex2002ans]

Beyond Tutorial (Optional): Optimize Images Further

**** This is a placeholder. Squeezing KB out of the final image. Reducing to 16 shades of gray. ****

I hope to see some great success stories posted, with some beautified formulas!

[GrannyGrump]

Ah. Great tutorial. this could also be adapted for fragments of any special text in-line.

I do want to warn about 16-shade pngs though. They display badly on some readers and some apps. See the thread I posted recently https://www.mobileread.com/forums/sho...51#post2633151

Jellby responded that it is a bug in the ADE engine, dealing poorly with 2-bit and 4-bit color.

I tried GMcG's suggestion for reducing to 16-shades and then back to 256 (8-bit). It does make a smaller size than an image with the full 256 shades, and in my testing came out about 10-25% bigger than the original 16-shade png. The resulting display seems to work fine. I was not working with web-safe palettes --- those would be a lot smaller, and should be fine for text.

[Toxaris]

Uhmmm, Why don't use the codecogs site? You can save it as gif/png/svg and others. That would only require one step.

[Tex2002ans]

The "Beyond" will be for the die-hards (like me) who like to squeeze out every last KB out of the EPUB (and where I can promote more ScriptPNG and/or other PNG optimization tools).

Or I might just split it into another Tutorial, and link to it from here, I haven't decided yet (need a few nights of rest, this was a lot of work).

I will definitely be doing a Tutorial on:

SVG -> PNG

(Was requested in this topic: https://www.mobileread.com/forums/sho...d.php?t=222825)

I might do:

Images of Tables -> LibreOffice Calc -> HTML

(Very wide tables, or pixelated JPG, or hideous ant-sized images with unreadable text).

and whatever else comes to my head.... I have stumbled upon so much stuff in my conversions, so I might as well teach others the methods that I use. Might save lots of people lots of time, and in the end, get higher quality books out there.

Originally Posted by grannyGrumpy

I do want to warn about 16-shade pngs though. They display badly on some readers and some apps. See the thread I posted recently https://www.mobileread.com/forums/sho...51#post2633151

Jellby responded that it is a bug in the ADE engine, dealing poorly with 2-bit and 4-bit color.

Indeed. I noticed the same thing months ago when reducing my formulas to 4 color grayscale, my Nook rendered the formula PNGs with large "streaks".

I was going to be posting some of my information/tests in your topic within the coming days.

Originally Posted by grannyGrumpy

I tried GMcG's suggestion for reducing to 16-shades and then back to 256 (8-bit). It does make a smaller size than an image with the full 256 shades, and in my testing came out about 10-25% bigger than the original 16-shade png. The resulting display seems to work fine. I was not working with web-safe palettes --- those would be a lot smaller, and should be fine for text.

Testing testing testing!!! I am glad to see you are still chipping away at those Tom Sawyer images.

Originally Posted by Toxaris

Uhmmm, Why don't use the codecogs site? You can save it as gif/png/svg and others. That would only require one step.

You have ruined me!

[GrannyGrump]

Tex2002ans said

Testing testing testing!!! I am glad to see you are still chipping away at those Tom Sawyer images.

Yup, still chipping. Got two more big Twain books in progress, with over 600 illustrations to clean and convert. (I'm either masochistic or insane!)

I was going to be posting some of my information/tests in your topic within the coming days.

I am looking forward to it. I won't finalize any more images till I learn about your results.

You have ruined me

ah, but it's only a tiny ruin. The rest of us can still use most of the tutorial for other kinds of text.

[Jellby]

Two cents:

1st ¢ You can skip Inkscape and open the PDF directly with GIMP. When asked, set the resolution to something high: 300 or 600 dpi, strong antialiasing everywhere and grayscale. You can then crop and resize the image in GIMP.

2nd ¢ If you know some LaTeX, it may be better to create the initial PDF with pdflatex.

[mrmikel]

Thanks for pointing out that GIMP can open up PDFs this way. A time saver for multiple images to place into epubs.

[Tex2002ans]

Originally Posted by Jellby

1st ¢ You can skip Inkscape and open the PDF directly with GIMP. When asked, set the resolution to something high: 300 or 600 dpi, strong antialiasing everywhere and grayscale. You can then crop and resize the image in GIMP.

Hmmmm... I will have to do more testing. I think there was a reason a while back I jumped over to Inkscape to handle this. The reasons will probably come flowing back to me.

But yes, Math PDF -> GIMP is most likely even faster for this specific case. Lots of different ways to get from A to B. Hopefully the tutorials spawn some more discussion too, on how everyone else handles the situation, giving lots of A to B options for anyone who comes across the tutorial!

I can quickly think of two reasons why I went the Inkscape route:

• Initially I used to just use the Width/Height for outputting the formulas instead of DPI. Inkscape allowed me to specify the exact size of the resultant PNG. GIMP lets you specify the X/Y of the entire "page" only.
  • Exporting only the "Selection" is helpful with vector graphs/tables that I want at a specified width.
• IF needed, you could do some actual editing in Inkscape (changing font, etc. etc.). Although this would best be handled back at the actual source document stage.
  • GIMP is an image editor, not a vector editor.
    • This is probably the problem I ran into, I wanted to change some things around in some (not so clean) vector files I had gotten, and it was impossible/a complete pain to do in GIMP.

Originally Posted by Jellby

2nd ¢ If you know some LaTeX, it may be better to create the initial PDF with pdflatex.

Of course of course... LaTeX would be ultimate (I have to teach myself that, it would be EXTREMELY useful... although I deal with very formula heavy books rarely). Perhaps a better title for this would have been "Beginner's Guide to Formula to PNG".

This tutorial was just something I thought of within the day, perhaps next time I should plan out everything for a month!

"Step 1A: LibreOffice Math"
"Step 1B: LaTeX"
"Step 1C: Codecogs"
"Step 1D: This is a placeholder for something Jellby/Toxaris will mention after posting"
"Step 2: If you followed Step 1A, do this. Otherwise, skip this step!".

I think LaTeX would be WAY out of the league of someone just doing some basic formulas every so often in books they come across.

Using LibreOffice Math (or http://www.codecogs.com/latex/eqneditor.php as Toxaris pointed out) simplifies a lot of that initial creation.

Perhaps you want to write a tutorial on the basics of LaTeX -> high resolution images for EPUBs? I know that I would benefit.

[Hitch]

Gang:

And of course, in Kindle, all PNG's are converted to GIF images for the mobi files. Also remember that with the newer Kindle HD devices, you're goign to want the largest/highest-rez images you can manage, as otherwise, a 72dpi image of, say, 100px wide will be about half an inch wide on that KindleFire HD 8.9" screen.

(Yes, I saw that you were outputting the pngs at 320, but just as a reminder for those who don't deal with all the Amazon devices.)

Hitch

[Tex2002ans]

I have attached a sample EPUB with four sets of images (times all four formulas), each inside of their own grey boxes:



#1. No Transparency (Removed and saved right from GIMP just like the tutorial) (8bpp Grayscale)
#2. Transparent formula (Inkscape -> GIMP -> Mode -> Grayscale -> Saved) (32bpp Grayscale)
#3. Transparent formula (Inkscape) (32bpp Grayscale)
#4. Transparent formula (fed through ScriptPNG to optimize) (Indexed 256)

Sadly, I don't have any physical Kindles to test on (I would not trust Kindle Previewer one iota when dealing with situations such as this).

Originally Posted by Hitch

And of course, in Kindle, all PNG's are converted to GIF images for the mobi files.

There used to be a KindlGen bug where transparent PNGs turned the transparency into a black background. I just looked up the changelog, and I see this was fixed in KindleGen 2.5:

https://kindlegen.s3.amazonaws.com/d...e%20Notes.html

- Improved transparent PNG conversion: Transparent PNG images (now converted to JPEG) will use white background. This originally used black background during conversion causing degraded reading experience.

According to the documentation, I always thought that KindleGen:

• All PNG -> GIF.
• Transparent PNG -> GIF with white background.

No clue why they would go from Transparent PNG to JPG!!!

I fed my Transparency Test EPUB into Kindle Previewer, and I found a few things to note:

• 32bpp Grayscale from GIMP turn into big black boxes. (This is the bug?)
• I unpacked the Kindle file with "KindleUnpack", and I see that it converts
  • #3 and #4 to JPG with white background (GAH).
  • #1 gets converted to GIF.
  • #2 is a GIF, but is just a big black box.

The bug:

I believe these images will speak for themselves:



On to the causes of the Bug:

This is all just educated guesses.

I used Imagemagick's "identify -verbose" on all 4 versions of Formula #1 (verbose output attached to this post). After comparing the output closely, I can see that the only major difference between Inkscape's version and GIMP's version is between color type:

https://en.wikipedia.org/wiki/Portab...cs#Color_depth

Inkscape's is set to 6 (RGB + Alpha) while GIMPs is set to the proper 4 (Grayscale + Alpha).

Guess: Perhaps KindleGen can't handle PNG Color Type 4 images?

Request:

Can someone with a lot more image experience create Grayscale + Alpha images and feed them into KindleGen/Previewer? Perhaps try to create them from a different program besides GIMP (maybe Photoshop, or the multitude of others).

Edit:



I just tested this on my Nook + ADE, and it seems like ADE likes to break on the ScriptPNG (Indexed + Transparency), but works fine on the others. The causes of this one? No clue, maybe an Indexing + transparency is EXTREMELY rare.

I would say avoid Transparency PNGs for now.

[PoP]

Originally Posted by Tex2002ans

...
Sadly, I don't have any physical Kindles to test on (I would not trust Kindle Previewer one iota when dealing with situations such as this).
...

If that can help I took screenshots on my K3W of

Your epub conversion to .azw3 by Calibre:

Spoiler:

Your epub conversion to .mobi by Kindle Previewer:

Spoiler:

[Hitch]

Hey, Tex:

The last time I unpacked a mobi we'd made with PNGs, what I got were gifs, across the board. I'd have to go back and research it, to see what the updates are doing (although...pretty sure I unpacked this after 2.5, mind you). We're a bit swamped right now, so it won't be immediate replies/response on this.

Hitch

[Tex2002ans]

Originally Posted by PoP

If that can help I took screenshots on my K3W of

Fantastic, thank you for the real world images. (Forgive the overflowing formulas out of the boxes, I just whipped that EPUB together).

I just converted to AZW3 using Calibre as well, and then took it apart using KindleUnpack, and I see that Calibre leaves the images as PNGs.

I always thought KindleGen did PNGs -> GIFs because the old Kindles couldn't support the format... perhaps it is some leftover code from the old MOBI Pocket days?

Originally Posted by Hitch

The last time I unpacked a mobi we'd made with PNGs, what I got were gifs, across the board.

So I thought as well. I could have sworn that PNG -> GIF is what I read in the Kindle Documentation a long time ago, it was common knowledge that I absorbed from MR, and I knew that there was a KindleGen black background bug with PNG transparency.

I went to go look again at the Kindle Documentation specifically on PNG + changelog to see if that black background bug was fixed.... and then I stumbled across that JPG statement!

I might also want to see now what it does to a PNG that is > 128 KB (or whatever the limit is on images in KF7).

Originally Posted by Hitch

I'd have to go back and research it, to see what the updates are doing (although...pretty sure I unpacked this after 2.5, mind you). We're a bit swamped right now, so it won't be immediate replies/response on this.

Thank you, more tests are always welcome. And since you are in touch with all those Amazon people all the time, perhaps you can slip them a question about it please.