Convert to AZW3 - comparison

hfpop · 02-12-2016, 05:08 PM

I did a few experiments, as follows:

(1) Start from DOCX file, public domain texts of a few hundreds pages. Convert to filtered HTML using MS Office.

(2) Use Calibre to convert the HTML to EPUB and MOBI and AZW3.

(3) Use KindleGen with option -c2 to convert EPUB to dual-MOBI and them KindleUnpack to extract the MOBI and AZW3 from this dual-MOBI.

Finding: the MOBI and AZW3 files generated via (3) are extremely small as compared to those generated via Calibre. Something along these lines:

- calibre MOBI = EPUB * 1.40
- calibre AZW3 = EPUB * 1.70
- kindlegen MOBI = EPUB * 1.10
- kindlegen AZW3 = EPUB * 1.15

What is your preferred working pathway, and why specifically? When do you use KindleGen for conversion and when Calibre? Why?

eschwartz · 02-14-2016, 02:39 AM

kindlegen uses more efficient, proprietary compression algorithms.

I cannot be bothered to use kindlegen and import all those extra formats one at a time.

hfpop · 02-15-2016, 01:40 PM

@eschwartz: Thanks for the info.

(1) I try to make heads and tails of kindlegen and calibre conversions between ebook filetypes.

Other than the compression algorithms that are used, there is another problem: the fonts size of the books are not the same, with the default settings used. All examples below start from DOCX -> HTML filtered. Then:

HTML -> (kindlegen) -> MOBI (0)
HTML -> (calibre) -> EPUB (1) and MOBI (1)
EPUB (1) -> (kindlegen) -> MOBI (2)
MOBI (1) -> (calibre) -> AZW3 (1)

Strangely, the fonts displayed in Kindle Voyage are in the following relation:
MOBI (0) = AZW (1) > MOBI (1) > MOBI (2)
i.e. even using calibre only, MOBI -> AZW3 increases the font to what appears to be the "standard".

I am certain that this picture is incomplete. Still, what can I do with my workflow to ensure that all files use the fonts at the same size? Is this a MOBI problem? And AZW3 is less prone to these font size jumps?

(2) From what Voyage is concerned, Calibre creates books, with default covers; Kindlegen creates documents (not books) with proper covers. All the books are manually sideloaded via USB (i.e. not via Calibre). I am fine with Calibre behavior, I would personally remove the covers altogether: it's about books, not cover art.

Thank you for your thoughts on this.

Doitsu · 02-15-2016, 03:01 PM

Quote:

Originally Posted by hfpop

All the books are manually sideloaded via USB (i.e. not via Calibre). I am fine with Calibre behavior, I would personally remove the covers altogether: it's about books, not cover art.

Shameless plug: if you don't manage your books with Calibre, you could use Sigil and my KindleGen plugin. It can optionally split KindleGen generated .mobi files into separate mobi7 and mobi8 books, add fake ASIN numbers, which turns "documents" into "books" and create cover thumbnails. (It uses the same libraries that KindleUnpack uses.)

hfpop · 02-15-2016, 03:20 PM

@Doitsu: thanks for the tip. I am playing with the plugin now. I have Sigil 0.9.3. But after installing the plugin I get the Sigil fatal python error: py_initialize unable to load the file system codec. Any idea?

Doitsu · 02-15-2016, 03:28 PM

Quote:

Originally Posted by hfpop

@Doitsu: thanks for the tip. I am playing with the plugin now. I have Sigil 0.9.3. But after installing the plugin I get the Sigil fatal python error: py_initialize unable to load the file system codec. Any idea?

It looks like that there's a problem with the Python path. If you have a Windows machine, you'll need to select the Use Bundled Python option in the Manage Plugins dialog box.
To exclude plugin specific problems, also install and run KevinH's testplugin.

hfpop · 02-15-2016, 03:38 PM

I have the PortableApps Sigil. With the Use Bundled Python option it crashes. If i manually set the path to a separate Python 3.4, portable as well, it runs ok. I will take a look indeed.

hfpop · 02-15-2016, 04:46 PM

Completion to #3.

Take a public domain EPUB, that was not touched by calibre.
(1) Convert to AZW3 through calibre. The result is with small fonts.
(2) Convert to AZW3 through kindlegen. The result is with normal fonts.
(3) Convert to MOBI and from MOBI to AZW3 through calibre. The same fonts as (1).
Here I used Calibre the environment.

(4) Convert to AZW3 through ebook-convert (command line tool). The result is with normal fonts.

Calibre the environment breaks the font sizes somewhere on the way.
How to fix this?
The more I play with these. the less I understand.

Doitsu · 02-15-2016, 05:03 PM

Quote:

Originally Posted by hfpop

Calibre breaks the font sizes somewhere on the way.
How to fix this?scapular

I don't think that Calibre "breaks" fonts. It's more likely that you somehow activated the built-in font subsetting feature, which will remove unused characters from fonts.

Check the following settings:

Preferences > Common Options > Subset all embedded fonts
Polish Books > Subset all embedded fonts

hfpop · 02-15-2016, 05:09 PM

Quote:

Originally Posted by Doitsu

I don't think that Calibre "breaks" fonts. It's more likely that you somehow activated the built-in font subsetting feature, which will remove unused characters from fonts.

Check the following settings:

Preferences > Common Options > Subset all embedded fonts
Polish Books > Subset all embedded fonts

No, I do not mean the size of the file in terms of occupied space on disk. I mean visual size of the text, when displayed on Kindle, at the same settings.

All the settings indicated by you are cleared.

JSWolf · 02-15-2016, 05:53 PM

Calibre does not break the way fonts look.

hfpop · 02-15-2016, 05:55 PM

@JSWolf: Then I have an error in my setup somewhere. Ok. Will go into it. Thank you.

eschwartz · 02-16-2016, 01:38 AM

Very possibly.

calibre has various settings, some of those affect the CSS for the font size.

P.S. ebook-convert is the same as converting via the GUI. It has the same options and uses the same code.

GrannyGrump · 02-16-2016, 03:13 AM

@hfpop --- I don't know if this has anything to do with your text-size issue, but when converting with Calibre, on the "Look & Feel" dialog, try putting a check-mark to DISABLE FONT SIZE RE-SCALING. This helped with a problem I had with Calibre re-sizing some of my text that was supposed to be set to larger sizes.

hfpop · 02-16-2016, 05:00 AM

Right. I had a wrong setup: Common Options / Look & Feel / Base font size = 1.0pt instead of the default 0.0pt.

I did one more test:
(1) Convert an original EPUB to AZW3 with this base font size set to 0.0 => text size = OK.
(2) Convert an original EPUB to AZW3 with this base font size set to 1.0 => text size = small.
(3) Convert the AZW3 from (2) to AZW3 with the settings following the wizard / Use default values (i.e. base font = 16.0pt and font size keys = 12.0, 12.0, 14.0, 16.0, 18.0, 20.0, 22.0, 24.0) => text size OK, same as with (1) directly.

This is consistent.
I thank all of you guys that helped me with this.

02-12-2016, 05:08 PM	#1
hfpop Addict Posts: 250 Karma: 136662 Join Date: Apr 2011 Device: N/A	Convert to AZW3 - comparison I did a few experiments, as follows: (1) Start from DOCX file, public domain texts of a few hundreds pages. Convert to filtered HTML using MS Office. (2) Use Calibre to convert the HTML to EPUB and MOBI and AZW3. (3) Use KindleGen with option -c2 to convert EPUB to dual-MOBI and them KindleUnpack to extract the MOBI and AZW3 from this dual-MOBI. Finding: the MOBI and AZW3 files generated via (3) are extremely small as compared to those generated via Calibre. Something along these lines: - calibre MOBI = EPUB * 1.40 - calibre AZW3 = EPUB * 1.70 - kindlegen MOBI = EPUB * 1.10 - kindlegen AZW3 = EPUB * 1.15 What is your preferred working pathway, and why specifically? When do you use KindleGen for conversion and when Calibre? Why?

02-15-2016, 03:38 PM	#7
hfpop Addict Posts: 250 Karma: 136662 Join Date: Apr 2011 Device: N/A	I have the PortableApps Sigil. With the Use Bundled Python option it crashes. If i manually set the path to a separate Python 3.4, portable as well, it runs ok. I will take a look indeed. Last edited by hfpop; 02-15-2016 at 04:25 PM.

02-15-2016, 04:46 PM	#8
hfpop Addict Posts: 250 Karma: 136662 Join Date: Apr 2011 Device: N/A	Completion to #3. Take a public domain EPUB, that was not touched by calibre. (1) Convert to AZW3 through calibre. The result is with small fonts. (2) Convert to AZW3 through kindlegen. The result is with normal fonts. (3) Convert to MOBI and from MOBI to AZW3 through calibre. The same fonts as (1). Here I used Calibre the environment. (4) Convert to AZW3 through ebook-convert (command line tool). The result is with normal fonts. Calibre the environment breaks the font sizes somewhere on the way. How to fix this? The more I play with these. the less I understand. Last edited by hfpop; 02-15-2016 at 05:00 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
AZW3 File Format - Can't Convert	rjbooks	Conversion	7	12-20-2014 08:53 PM
Why does Calibre convert AZW to AZW3?	epstewart	Conversion	7	09-05-2014 06:36 PM
How to convert Epub or AZW3 to Word?	mkhuebner	Conversion	5	01-13-2014 09:45 AM
Cannot open or convert certain AZW3 books	bshar489	Calibre	2	03-18-2013 11:47 AM
Unable to convert from EPUB to AZW3	jujubox7	Conversion	12	02-12-2013 01:43 PM

02-14-2016, 02:39 AM	#2
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85400180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	kindlegen uses more efficient, proprietary compression algorithms. I cannot be bothered to use kindlegen and import all those extra formats one at a time.

02-15-2016, 01:40 PM	#3
hfpop Addict Posts: 250 Karma: 136662 Join Date: Apr 2011 Device: N/A	@eschwartz: Thanks for the info. (1) I try to make heads and tails of kindlegen and calibre conversions between ebook filetypes. Other than the compression algorithms that are used, there is another problem: the fonts size of the books are not the same, with the default settings used. All examples below start from DOCX -> HTML filtered. Then: HTML -> (kindlegen) -> MOBI (0) HTML -> (calibre) -> EPUB (1) and MOBI (1) EPUB (1) -> (kindlegen) -> MOBI (2) MOBI (1) -> (calibre) -> AZW3 (1) Strangely, the fonts displayed in Kindle Voyage are in the following relation: MOBI (0) = AZW (1) > MOBI (1) > MOBI (2) i.e. even using calibre only, MOBI -> AZW3 increases the font to what appears to be the "standard". I am certain that this picture is incomplete. Still, what can I do with my workflow to ensure that all files use the fonts at the same size? Is this a MOBI problem? And AZW3 is less prone to these font size jumps? (2) From what Voyage is concerned, Calibre creates books, with default covers; Kindlegen creates documents (not books) with proper covers. All the books are manually sideloaded via USB (i.e. not via Calibre). I am fine with Calibre behavior, I would personally remove the covers altogether: it's about books, not cover art. Thank you for your thoughts on this. Last edited by hfpop; 02-15-2016 at 01:54 PM.

02-15-2016, 03:20 PM	#5
hfpop Addict Posts: 250 Karma: 136662 Join Date: Apr 2011 Device: N/A	@Doitsu: thanks for the tip. I am playing with the plugin now. I have Sigil 0.9.3. But after installing the plugin I get the Sigil fatal python error: py_initialize unable to load the file system codec. Any idea?

02-15-2016, 05:53 PM	#11
JSWolf Resident Curmudgeon Posts: 79,740 Karma: 145864619 Join Date: Nov 2006 Location: Roslindale, Massachusetts Device: Kobo Libra 2, Kobo Aura H2O, PRS-650, PRS-T1, nook STR, PW3	Calibre does not break the way fonts look.

02-15-2016, 05:55 PM	#12
hfpop Addict Posts: 250 Karma: 136662 Join Date: Apr 2011 Device: N/A	@JSWolf: Then I have an error in my setup somewhere. Ok. Will go into it. Thank you.

02-16-2016, 01:38 AM	#13
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85400180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	Very possibly. calibre has various settings, some of those affect the CSS for the font size. P.S. ebook-convert is the same as converting via the GUI. It has the same options and uses the same code.

02-16-2016, 03:13 AM	#14
GrannyGrump Obsessively Dedicated... Posts: 3,221 Karma: 35037583 Join Date: May 2011 Location: PA {back in the usa!} Device: Sony PRS-T2, ADE on PC	@hfpop --- I don't know if this has anything to do with your text-size issue, but when converting with Calibre, on the "Look & Feel" dialog, try putting a check-mark to DISABLE FONT SIZE RE-SCALING. This helped with a problem I had with Calibre re-sizing some of my text that was supposed to be set to larger sizes.

02-16-2016, 05:00 AM	#15
hfpop Addict Posts: 250 Karma: 136662 Join Date: Apr 2011 Device: N/A	Right. I had a wrong setup: Common Options / Look & Feel / Base font size = 1.0pt instead of the default 0.0pt. I did one more test: (1) Convert an original EPUB to AZW3 with this base font size set to 0.0 => text size = OK. (2) Convert an original EPUB to AZW3 with this base font size set to 1.0 => text size = small. (3) Convert the AZW3 from (2) to AZW3 with the settings following the wizard / Use default values (i.e. base font = 16.0pt and font size keys = 12.0, 12.0, 14.0, 16.0, 18.0, 20.0, 22.0, 24.0) => text size OK, same as with (1) directly. This is consistent. I thank all of you guys that helped me with this.

Advert

Advert