Change AZW3 compression algorithm?

[Sekkei]

Afaik AZW3 is based on mobi, wich uses probably deflate (?) to compress. Would it be possible to recompress/change the algorithm to PPMd?
The memory usage is one mb higher, but the compression is way better. (e.g 314KB only text in html > 179KB)

[j.p.s]

kindlegen (which writes mobi and azw3) has 3 compression options:
-c0: no compression
-c1: standard DOC compression
-c2: Kindle huffdic compression

What would be the point of using a compression that reading software does not handle?

[Sekkei]

Well, I didn't know there are two option to compress. Which one does calibre use to create azw3 files; Would it be possible to handle kindleGen a folder with azw3 structure to create an azw3, or can I only convert mobi and epub?

What would be the point of using a compression that reading software does not handle?

This depends on the reader and how it is coded, if it uses a library with more than one supported decompression method it MIGHT simple work out of box

I just want smaller files, I think 300kb for one book is quite alot.

[Doitsu]

Originally Posted by j.p.s

kindlegen (which writes mobi and azw3) has 3 compression options:
-c0: no compression
-c1: standard DOC compression
-c2: Kindle huffdic compression

By default, KindleGen will also append the source files, unless the -dont_append_source option is specified.

Originally Posted by Sekkei

I just want smaller files, I think 300kb for one book is quite alot.

IMHO, you won't be able to create much smaller ebooks with KindleGen or Calibre.

[stumped]

Originally Posted by Sekkei

I just want smaller files, I think 300kb for one book is quite alot.

ridiculous

if you cant live with 300kb, buy shorter books

you do realise that an e ink reader has enough storage for a few thousand 300Kb books ?

lets say you improve the compression by 10% and save an amazing 30kb per book
what could you possibly use that for ?

[j.p.s]

Originally Posted by Sekkei

Well, I didn't know there are two option to compress. Which one does calibre use to create azw3 files; Would it be possible to handle kindleGen a folder with azw3 structure to create an azw3, or can I only convert mobi and epub?

Your best bet is to ask calibre questions in a calibre forum.

kindlegen only converts to kindle formats, not from them.

[j.p.s]

Originally Posted by Doitsu

By default, KindleGen will also append the source files, unless the -dont_append_source option is specified.

My point was that amazon written readers are unlikely to support decompression of types not provided by amazon written producers.

[JSWolf]

One thing you should be aware of is that with higher compression comes more processor usage and more battery usage to decompress. It's not worth it for the pittance in space you'd save.

[j.p.s]

Originally Posted by JSWolf

One thing you should be aware of is that with higher compression comes more processor usage and more battery usage to decompress. It's not worth it for the pittance in space you'd save.

That is not true in all cases.

A compression algorithm specialized for particular input data might produce smaller output while using less CPU than a general compression algorithm.

It is very common for compression algorithms that squeeze out a bit more compression at the cost of a lot of CPU to have decompressors that use little or no additional CPU.

[JSWolf]

Originally Posted by j.p.s

That is not true in all cases.

A compression algorithm specialized for particular input data might produce smaller output while using less CPU than a general compression algorithm.

It is very common for compression algorithms that squeeze out a bit more compression at the cost of a lot of CPU to have decompressors that use little or no additional CPU.

I don't know about KF8, but I do know that when Mobi gets compressed to maximum, it does use more CPU to decompress.

[Sekkei]

Thank you all for responding

Originally Posted by Doitsu

By default, KindleGen will also append the source files, unless the -dont_append_source option is specified.

So this helps that unsupported files are not striped away? Would this also help with content inside a e.g HTML that would not be supported by kindleGen?

Originally Posted by Doitsu

IMHO, you won't be able to create much smaller ebooks with KindleGen or Calibre.

Sad but propably true, I really hoped they slaped a default zip lib on it and call it a day

Originally Posted by stumped

ridiculous, if you cant live with 300kb, buy shorter books
you do realise that an e ink reader has enough storage for a few thousand 300Kb books ? lets say you improve the compression by 10% and save an amazing 30kb per book
what could you possibly use that for ?

I would not even write ebooks in xml if I could, so much wasted space, literally 30kb! haha Sadly my Voyage has only 4GB and I like manga, I even recompress them manually to a per page ~180KB High-res low-color PNG, but as you can imagine, there is not space left if you want to carry your favorite manga saga on the go :/
And saving atleast 30kb on 10 books is one additional book
(If we could use PPMd this would cut the size in half, allowing the doubble amount of books you could have without it)

Originally Posted by j.p.s

kindlegen only converts to kindle formats, not from them.

So is there any way to create myself a clean azw3 file, that atleast doesn't attempt to "fix" everything I do, or adding unused meta data every two steps.
I would much prefer to reformate my files in vsCode and push it to phyton lib (or other valid opensource converter)

[pdurrant]

Originally Posted by Sekkei

Sadly my Voyage has only 4GB and I like manga, I even recompress them manually to a per page ~180KB High-res low-color PNG, but as you can imagine, there is not space left if you want to carry your favorite manga saga on the go :/
And saving atleast 30kb on 10 books is one additional book
(If we could use PPMd this would cut the size in half, allowing the doubble amount of books you could have without it)

If we're talking about images, your best bet is to look into compressing the images further before turning them into an ebook.

[Hitch]

Originally Posted by pdurrant

If we're talking about images, your best bet is to look into compressing the images further before turning them into an ebook.

Yes, this, ^ exactly.

At my firm we take a few extra steps, for every book with images, using proprietary tools that we've developed and some tools that are available for anyone (like TinyPNG, which is awesome) to eliminate empty pixels and other wasted space in images.

Amazon actually does a pretty incredible job of compression. But the better job you do beforehand, the easier their job is.

Manga and other comic formats (yeah, yeah, I know, but let me lump them together for now) andif you want to save space, man, that's the place to start. IME and FWIW.

Hitch

[Tex2002ans]

Originally Posted by stumped

lets say you improve the compression by 10% and save an amazing 30kb per book
what could you possibly use that for ?

10% more books.

Originally Posted by j.p.s

It is very common for compression algorithms that squeeze out a bit more compression at the cost of a lot of CPU to have decompressors that use little or no additional CPU.

Yep, or just being much more intelligent with the parameters it's choosing.

Like with PNG compression, there are a million valid ways to compress.

Zopfli is an alternative very CPU-heavy algorithm Google came up with, but resulted in a smaller PNG (~3-8%) by selecting better parameters.

Takes the same exact amount of resources to decompress it though.

I gave some examples way back in 2013:

https://www.mobileread.com/forums/sh...45#post2499645

(Since then, ScriptPNG has been superseded by pingo.)

Originally Posted by Sekkei

Sadly my Voyage has only 4GB and I like manga, I even recompress them manually to a per page ~180KB High-res low-color PNG, but as you can imagine, there is not space left if you want to carry your favorite manga saga on the go :/

Manga? PNGs?!!? Then definitely run it through pingo and you'll probably save a hell of a lot of space. At least 3-8%, but depending on the complexity and if the publisher compressed previously, you may even save more.

If they're B&W/grayscale/<256-colors, you could also convert it to Indexed images for you and save A TON of space. (In the example linked above, my chart went from 143.1 KB -> 44.5 KB.)