EPUB (14 MB) becomes AZW3 (140 MB) after conversion!?

[FelixKrull]

Hi everyone,

I’m running into a pretty extreme file size issue when converting an EPUB to AZW3 using Calibre.

Example:

Original EPUB: 14 MB (contains lots of pictures)
Converted AZW3: 140 MB

So the output file is about 10× larger, which seem way beyond normal differences.

From what I understand so far:

EPUB is essentially a ZIP container, so its size is heavily compressed
During conversion, images and content may be extracted and stored less efficiently
There might be duplication of resources (HTML, images, fonts?) in AZW3
PNG images in particular might be a problem if they are not recompressed

However, even considering all that, a jump from 14 MB to 140 MB feels excessive.

What I already checked/tested:

The option “Do not compress images” is NOT enabled
Tried different output profiles (Kindle Paperwhite etc.)
No obvious difference in settings affecting file size

Questions:

1. What are the most common reasons for such a drastic size increase?
2. Does Calibre re-encode images at all, or mostly pass them through unchanged?
3. Is there any built-in way to force proper image compression (e.g. reduce JPEG quality or convert PNG → JPEG)?
4. Could internal duplication in AZW3 (KF8 structure) explain such a large factor?

If needed, I can inspect the EPUB contents (images, formats, resolutions), but I’d like to understand first whether this behavior is expected or indicates a specific issue.

Any insigts or best practices would be appreciated!

Thanks in advance.

[pdurrant]

Use KindleUnpack on the AZW3 and see where the space is going. It's probably in images, so compare them to the images in the ePub.

[FelixKrull]

Yes, it does seem to be the images, but I’m still trying to understand how this is even possible.

In the EPUB, I see a lot of PNG files (in the built-in editor it says “1592@1.4 GB” under 'pictures'), but after conversion they appear to be massively inflated in size.

Why would PNG images grow that much during conversion? I would have expected them to stay roughly the same, or at least not increase by such a large factor. PNG is lossless.

Is Calibre reprocessing PNGs in a way that removes compression efficiency, or is this related to how AZW3/KF8 stores images internally?

Trying to understand the mechanism behind this.

[gbm]

In Convert-AZW3 output make sure that "Disable compression of the file contents" is not checked.

bernie

Originally Posted by FelixKrull

Yes, it does seem to be the images, but I’m still trying to understand how this is even possible.

In the EPUB, I see a lot of PNG files (in the built-in editor it says “1592@1.4 GB” under 'pictures'), but after conversion they appear to be massively inflated in size.

Why would PNG images grow that much during conversion? I would have expected them to stay roughly the same, or at least not increase by such a large factor. PNG is lossless.

Is Calibre reprocessing PNGs in a way that removes compression efficiency, or is this related to how AZW3/KF8 stores images internally?

Trying to understand the mechanism behind this.

[FelixKrull]

I have done that, see my posting:

What I already checked/tested:

The option “Do not compress images” is NOT enabled

[pdurrant]

Originally Posted by FelixKrull

In the EPUB, I see a lot of PNG files (in the built-in editor it says “1592@1.4 GB” under 'pictures'), but after conversion they appear to be massively inflated in size.

Rather than looking at pictures in general, I'd look at one picture in the unzipped epub, and the same picture in the unpacked AZW3, and then you'll be able to tell what kind of conversion is or isn't happening.

zip shouldn't be able to compress a PNG very much. But I suppose if the original PNGs were saved uncompressed, the EPUB might be able to compress them more than AZW3.

[Gormagon]

Originally Posted by FelixKrull

Yes, it does seem to be the images, but I’m still trying to understand how this is even possible.

It seems (like in "I don't really know but assume") or it is (like in "I really compared the files in depth")?

Did you choose "Edit Book" in calibre for the azw3 and epub file and/or open the epub file as archive in 7zip to look at the numbers and sizes of the html, style and picture files. Then you should be able to see what's causing the enlargement.

You could replace the picture files in the azw3 file in calibre with the picture files from the epub file (after extracting them). Since I only use calibre for very rare conversions into epub I only tested this with one picture I added to a azw3. Maybe there's a script to replace/import a large number of files into azw3. I don't know.

[FelixKrull]

Unpacking the epub with 7-zip shows that it is in fact exactly the pictures which cause the enlargement.

So if I would replace the pictures after unpacking nothing would change. I would have to replace the pictures without unpacking the epub, but how is this supposed to work?

This seems to be a flaw within calibre's compression when converting epub to azw3.

[Gormagon]

Someone has to unzip the epub file, either you yourself or some program/script.
If you extract them yourself, you could delete the enlarged pictures in the azw3 file with calibre after opening the azw3 file with "edit book", then add the original pictures of the unzipped epub file. If you do this one by one, I guess, it will take quite some time with a large number of pictures. Better look for a script or plugin that does this in calibre.

[FelixKrull]

Originally Posted by Gormagon

Someone has to unzip the epub file, either you yourself or some program/script.
If you extract them yourself, you could delete the enlarged pictures in the azw3 file with calibre after opening the azw3 file with "edit book", then add the original pictures of the unzipped epub file....

Maybe I didn’t explain that clearly:

I already unzipped the EPUB and looked at the images. The issue is that the images themselves are already large at that stage. So even before any AZW3 conversion, the extracted PNG files are much bigger than what the 14 MB EPUB size would suggest.

That’s why simply replacing images in the AZW3 with those from the EPUB doesn’t really help in my case, they’re effectively the same (already “inflated”) files once extracted.

So the core of my confusion is this:

How can the EPUB be only arond 14 MB, while the extracted image assets (PNG) already add up to a much larger size? Is this purely due to zip compression efficiency on PNG files, or is there something else going on (e.g. optimization tricks inside the EPUB container)?

In other words, the size jump seems to happen between:
EPUB (compressed container) > extracted contents

not just during:
EPUB to AZW3 conversion

[Gormagon]

You could use IrfanView to save the png files with a higher compression level (that is if they are not already using compression level 9) or convert them to jpg files and adjust the quality to reduce the file size. Irfanview can also make batch conversions.
Then you could add these optimized picture files to the azw3 file.

[pdurrant]

Originally Posted by FelixKrull

How can the EPUB be only arond 14 MB, while the extracted image assets (PNG) already add up to a much larger size? Is this purely due to zip compression efficiency on PNG files, or is there something else going on (e.g. optimization tricks inside the EPUB container)?

Assuming all the pictures are unique, it's almost certainly that the png images aren't compressed as well as they should be, and so the zip compression of the EPUB compresses them further.

You could try using the "losslessly compress images" option of the Polish Ebooks calibre extension on the epub should should makes the PNG images smaller. (A lot easier than doing it manually). Then the AZW3 would also be smaller.

To find out exactly what's going on would require examining one of the original PNGs to see what settings it's been saved with.

I'd say that uploading one PNG here would be considered fair use, if you don't have the expertise to examine it yourself. But that's really just for interest.

I expect that running the Polish Ebooks plugin on the ePub will make the AZW3 from a conversion of the polished ePub a lot smaller.