Bloated epub file sizes?

[Joques]

Huh. I just tried reconverting one of the worse offenders, The Heroes by Abercrombie at 3,6 MB. Ticking "subset all embedded fonts" yielded no difference in file size. But I checked the contents, and those map file are a whopping 300kB per file. For low-res B/W image files. No compression at all, it seems. That is crazy!

[DomesticExtremis]

Originally Posted by Joques

Huh. I just tried reconverting one of the worse offenders, The Heroes by Abercrombie at 3,6 MB. Ticking "subset all embedded fonts" yielded no difference in file size. But I checked the contents, and those map file are a whopping 300kB per file. For low-res B/W image files. No compression at all, it seems. That is crazy!

Well, those images will be viewed on everything from a smartphone to a hi-res tablet, as well as on PC based ePUB software, not just an e-ink e-reader, so it isn't so obvious that they should be low res, small dimension images.

[HarryT]

Originally Posted by DomesticExtremis

Well, those images will be viewed on everything from a smartphone to a hi-res tablet, as well as on PC based ePUB software, not just an e-ink e-reader, so it isn't so obvious that they should be low res, small dimension images.

Not to mention that many readers offer the ability to zoom in on an image, and a map in particular benefits from the ability to do that and have it remain sharp and clear.

[j.p.s]

Originally Posted by Joques

Huh. I just tried reconverting one of the worse offenders, The Heroes by Abercrombie at 3,6 MB. Ticking "subset all embedded fonts" yielded no difference in file size. But I checked the contents, and those map file are a whopping 300kB per file. For low-res B/W image files. No compression at all, it seems. That is crazy!

If those maps are line drawings and are JPEGs, they should have been created as PNG. PNG line drawings generally take less space and look nicer than JPEGs of line drawings. It might be possible to clean up the JPEGs and convert to PNG, but JPEG artifacts would probably remain, adding ugliness increasing file size.

If they really are an uncompressed type (does EPUB even support uncompressed images?), compressing will help a lot. Again, for line drawings, PNG will usually be much better than JPEG.

[Tex2002ans]

Originally Posted by j.p.s

If those maps are line drawings and are JPEGs, they should have been created as PNG. PNG line drawings generally take less space and look nicer than JPEGs of line drawings. It might be possible to clean up the JPEGs and convert to PNG, but JPEG artifacts would probably remain, adding ugliness increasing file size.

I second PNG for "artificial images":

https://www.mobileread.com/forums/sho...54&postcount=8
https://www.mobileread.com/forums/sho...3&postcount=30
https://www.mobileread.com/forums/sho...5&postcount=26

Although it is quite hard to go from JPG -> a clean PNG without the source. Once you go lossy, you can't really go back, the artifacts/damage was already done.

I really wish that with maps and diagrams, the dang publishers/authors would release the vector files. It would be SO MUCH nicer for people who like to go in and create some quality stuff.

In the print version of a book you won't get some tiny ant-sized thumbnail. You get a nice, large, crisp map.

Originally Posted by j.p.s

(does EPUB even support uncompressed images?)

JPG, PNG, and GIF are the valid image types in an EPUB (and SVG, but I would consider that a different cup of tea).

[Joques]

Originally Posted by DomesticExtremis

Well, those images will be viewed on everything from a smartphone to a hi-res tablet, as well as on PC based ePUB software, not just an e-ink e-reader, so it isn't so obvious that they should be low res, small dimension images.

But they _are_ low-res. I'm not saying they should be, I like my maps as crisp as can be. But they are simultaneously low-res and large filesize - the worst of both worlds

[JSWolf]

Originally Posted by Joques

Huh. I just tried reconverting one of the worse offenders, The Heroes by Abercrombie at 3,6 MB. Ticking "subset all embedded fonts" yielded no difference in file size. But I checked the contents, and those map file are a whopping 300kB per file. For low-res B/W image files. No compression at all, it seems. That is crazy!

Here you go.. hi-resolution maps for The Heroes that you can use to replace the low-res ones in the book.

http://www.joeabercrombie.com/maps-for-the-heroes/

[Joques]

Awesome, thanks! In truth I prefer keeping maps on a tablet anyway, since paging back-and-forth isn't exactly en ereader's strong suit.

[Joques]

I _REALLY_ wish more authors would do this.

[j.p.s]

Originally Posted by Joques

Awesome, thanks! In truth I prefer keeping maps on a tablet anyway, since paging back-and-forth isn't exactly en ereader's strong suit.

What are the dimensions of the map images in the e-book? I am asking because your post #16 says they are 300kB each and low resolution with no compression, and what I see from the images on the author's we site are:

Full size: 2048x1448 2.7MB JPEG
Thumbnails: 300x212 36KB JPEG

I shrunk the full sized images to 1024x724 and 512x362 JPEGs and they 308KB and 72KB respectively. I wouldn't call 1024x724 low resolution on an e-ink reader.

[j.p.s]

Originally Posted by JSWolf

Here you go.. hi-resolution maps for The Heroes that you can use to replace the low-res ones in the book.

http://www.joeabercrombie.com/maps-for-the-heroes/

Those are very densely detailed for line drawings, but PNG would still be a big win over JPEG, and in more than one way.

Simply converting them to PNG with 256 shades of gray made 1.7MB images, a 37% reduction. Under the bash shell.
for i in heroes-*jpg do; djpeg $i | pnmtopng > PNG0/`basename $i .jpg`.png

At the other extreme, converting to true B&W (1 bit) images resulted in 209KB 2048x1448 images, but those look significantly worse, even no counting the gray regions being shown
as white.
for i in heroes-*jpg; do djpeg $i | pnmdepth 1 | pnmtopng > PNG1b/`basename $i .jpg`.png; done

Converting to 2 bit (4 "color") PNG made 402KB full size images, less than 15% of the space required, or an over 85% reduction. The reduced number of gray shades probably slightly reduces the anti-aliasing benefits of more gray shades, but it also somewhat reduces JPEG artifacts. It even reduces scanning error. This happens all over the image, but it is most clear in areas with a single gray shade, such as the lower left and upper right corners. This makes the image cleaner and use fewer bytes.

An optimal way to produce the web images would have been to scan them at 600dp (4096x2896)i 2bit (4 color) and save them as 2048x1448 4bit PNG.

To really go all out, scan at 1 bit 1200 dpi, fill in the regions that should be gray with some tool, and save as 2048x1448 4bit PNG.

[Joques]

Originally Posted by j.p.s

What are the dimensions of the map images in the e-book? I am asking because your post #16 says they are 300kB each and low resolution with no compression, and what I see from the images on the author's we site are:

Full size: 2048x1448 2.7MB JPEG
Thumbnails: 300x212 36KB JPEG

I shrunk the full sized images to 1024x724 and 512x362 JPEGs and they 308KB and 72KB respectively. I wouldn't call 1024x724 low resolution on an e-ink reader.

The ones that are in the ebook are 502x714, almost exactly half of your already shrunk files, and bad enough that several of the location names are really hard to make out. Edit: And those are the ones that take up 300kB of space.

Like I said - having hi-res maps on a tablet beside the ereader is ideal for me, more important really than having hi-res maps in the ebook file as it is so much work going back and forth between map and text.

In fact - that would be my number 1 potential new feature in Kobo FW: a hotkey to go directly to a specific predetermined page (map page for instance, or dramatis personae - or cover art for that matter) without having to faff about with bookmarks and going through the unwieldy "annotations" menu - and then one to go back where you came from.

[theducks]

Originally Posted by Joques

Thanks everybody for the responses! I am sorry I posted in the wrong forum - but it was because though I personally use epubs, I assumed this was a universal problem that plagued all formats.

I _do_ try to find the highest-res cover art that I can for all my books, but when converted to B/W they don't take up all that much space. I'll experiment a little with removing embedded fonts with Calibre, but cleaning up bloated CSS styles for 900 books is not going to happen :-)

Calibre's Polish will clean unused styles from the stylesheet and can be used from a bulk.

While the change in the book SIZE will be small, performance can improve with a less bulky stylesheet

[Joques]

theducks, that is interesting. I have a few books that are especially sluggish. I will try that out.

[j.p.s]

Originally Posted by Joques

In fact - that would be my number 1 potential new feature in Kobo FW: a hotkey to go directly to a specific predetermined page (map page for instance, or dramatis personae - or cover art for that matter) without having to faff about with bookmarks and going through the unwieldy "annotations" menu - and then one to go back where you came from.

I'm not sure whether there is sufficient demand for a hot key, but I have found the back function on Kindles with buttons very useful and well implemented. (I have never used a touch screen Kindle.) The Pocketbook 360 is just as good for this. Everything else has been a miserable experience trying to go back, to the point that hesitate to leave the spot where I am, except to go back a page or two.