To CBZ, or not to CBZ: that is the question

[Quoth]

Originally Posted by Aleron Ives

Windows uses the extension to determine which program should open the file, but any compression program on Windows will be smart enough to look at the file header when deciding how to decompress it, rather than blindly relying on the extension.

In general many programs on Windows that are associated woth multiple file-endings may decide what to do based on file contents. So with some versions of MS Word a .dot file renamed as .doc would still be a .dot file. But the point is that based on the file-ending the Windows OS or Explorer purely decides whether to invoke MS Word or 7zip or Excel or Calibre etc based on the current file endings associations stored in the Registry. Linux and Mac are more complex.

[tomsem]

Going through the 80 or so comics/graphic novels in my Kindle library, it is a mixed bag. Most are just containers for page images and be converted to another format in more or less a straightforward way. At least one has images for 2 page spreads, and I had to figure out how to split them in half first.

So for these, it is a question about how much work I want to with adding navigation features (table of contents or hyperlinks), which is usually lacking in the original format.

Another handful are evidently FXL ePub that were converted to Kindle format, and text is selectable and so forth. The most straightforward target should be back to FXL ePub, but I'm not sure calibre handles it correctly (Kovid does not claim that it does).

I tried it, but Kindle Previewer 3 complains about unresolved hyperlinks, calibre edit finds another set of errors, but epubcheck only complains about encrypted fonts. It's probably 'okay' unless I want to add it as a personal document for Kindle (say after adding navigation features). Then I'd need to at least fix the hyperlinks. I should also see how Sigil likes these.

At least one is in Print Replica format, which is not currently amenable to DRM removal.

It's looking more like FXL ePub may be my preferred format, as opposed to PDF or CBZ. But I would like to find some alternative to InDesign for creating them from scratch.

[Quoth]

Originally Posted by tomsem

At least one is in Print Replica format, which is not currently amenable to DRM removal..

A Comic?
Easiest of all. Screen shots or camera or scanner. Trivial.

FXL ePub may be my preferred format, as opposed to PDF

FXL ePub purpose is to publish print replica via ebook retail. If it's for yourself a PDF with text etc is better. Works on everything.
Also Indesign looks expensive to me when you aren't selling 1000s of copies of what it produces. It's really a Paper DTP program fudged for epub, which is why it actually does fixed layout / print replica epub better than real reflowable epubs.

[tomsem]

Originally Posted by Quoth

A Comic?
Easiest of all. Screen shots or camera or scanner. Trivial.


FXL ePub purpose is to publish print replica via ebook retail. If it's for yourself a PDF with text etc is better. Works on everything.
Also Indesign looks expensive to me when you aren't selling 1000s of copies of what it produces. It's really a Paper DTP program fudged for epub, which is why it actually does fixed layout / print replica epub better than real reflowable epubs.

Yes, it is on my list of projects to automate screen capture and create ebooks from that.

I would need to see how well calibre can convert these FXL things to PDF. I agree, there are many tool choices for PDF, probably better in the long run, and (for example) Python libraries for working with them. The one I have been looking at lately, PyMuPDF, also works with ePub, something I have not explored at all yet.

Apparently FXL derives from Apple's proprietary iBooks ePub extension. There was a pretty nice tool for creating those.

It's just weird there are no open source tools for generating FXL (say, from a PDF). I see there are companies you can pay to convert your PDF (with their proprietary tools).

[DNSB]

Originally Posted by tomsem

I would need to see how well calibre can convert these FXL things to PDF. I agree, there are many tool choices for PDF, probably better in the long run, and (for example) Python libraries for working with them. The one I have been looking at lately, PyMuPDF, also works with ePub, something I have not explored at all yet.\

At this time, calibre does an extremely poor job of converting a FLO ePub to PDF. and there are no tools that I am aware of that do any better converting PDF to FLO ePub or vice versa.

Originally Posted by tomsem

Apparently FXL derives from Apple's proprietary iBooks ePub extension. There was a pretty nice tool for creating those.

Last time I looked, fixed layout epubs were part of the ePub 3.0 specification which was kicking around a couple of years before it's approval in 2011. iBooks implementation was based on that specification. Are you thinking of Apple's IBA format created with the Apple Author tool? The one that was loosely based on ePub but depended on custom code in the iBooks/Books app?

Originally Posted by tomsem

It's just weird there are no open source tools for generating FXL (say, from a PDF). I see there are companies you can pay to convert your PDF (with their proprietary tools).

Considering that PDF is considered one of the worst formats to convert from, it's not all that weird. Not to mention that a FLO ebook has issues with changing font sizes, adapting to screen sizes, no adjusting line spacing, no adjusting margins. About the only books I've seen that look better in FLO are children's books with text over images, graphic novels, pretty much any ebook where on almost every page, graphics and text are intermixed/overlayed and you are using a larger screen device.

A FLO ePub positions everything on the page pretty much down to the pixel within the viewport. Even trying to convert to a reflowable ePub is a painful task.

[tomsem]

Thanks for the observations.

Actually calibre did a pretty good job converting to PDF with one I did awhile back. A couple of the headings needed to be repositioned. There's not really any reason not to try it and see how it does.

[Quoth]

Originally Posted by tomsem

It's just weird there are no open source tools for generating FXL (say, from a PDF). I see there are companies you can pay to convert your PDF (with their proprietary tools).

People go to the effort of building tools for things they want to use.

[Freeshadow]

Originally Posted by DNSB

Zopfli made that list even using it's deflate algorithm.

Oh, that's really the 1st time I hear about something zopfli-recompressed NOT working as a standard ZIP file.

[DNSB]

Originally Posted by Freeshadow

Oh, that's really the 1st time I hear about something zopfli-recompressed NOT working as a standard ZIP file.

On a computer with 4+GB of RAM, there wouldn't be any issues. Ereaders tend to have smaller memory sizes and in my testing, some of the generated CBZ files would not open on the Kobo Glo I was using for testing. The sheer slowness of zopfli which took 80 times longer to compress the files for a ~5% size savings was also noticed but not a real issue since the compression is basically a one time occurrence.

[Freeshadow]

I had working epub on even so small and old reader as an bebook mini so memsize isn't your problem; either is decomp speed ;the problem is that zopfli based tools as I tested them, don't allow to add unshrinked files to the archive (meta.inf in epubs) thus you can't CREATE epubs with zopfli tools. you have to take a traditionally created zip or other file containing deflate stringsand reshrink it with: advzip -z -4 -i 1000 $name.epub.
this operates only with the deflated sections WITHIN a file leaving the rest of a file (ANY file) untouched, and then all works fine.

[DNSB]

Originally Posted by Freeshadow

I had working epub on even so small and old reader as an bebook mini so memsize isn't your problem; either is decomp speed ;the problem is that zopfli based tools as I tested them, don't allow to add unshrinked files to the archive (meta.inf in epubs) thus you can't CREATE epubs with zopfli tools. you have to take a traditionally created zip or other file containing deflate stringsand reshrink it with: advzip -z -4 -i 1000 $name.epub.
this operates only with the deflated sections WITHIN a file leaving the rest of a file (ANY file) untouched, and then all works fine.

An ePub has little to do with this discussion. I was working with CBZ files which were images only. Basically, I came out of the testing with the belief that zopfli is too slow to be of use to me and the 5% increase in compression ratio was not worth the time to achieve it.

As for your command line, what is it intended to do? Advzip last time I looked used 7-Zip's implementation of deflate while using zopfli by changing the executable name to zopfli and removing the space between the -i and the 1000 generates a .gz archive containing the epub by default while for my test epub which is 44,060,303 bytes, the epub.gz file came out as 44,005,894 for a massive 53KB size savings.

[Sarmat89]

Originally Posted by DNSB

Ereaders tend to have smaller memory sizes and in my testing, some of the generated CBZ files would not open on the Kobo Glo I was using for testing.

The Deflate algorithm uses almost no memory. It is impossible not to decode a valid deflate stream due to lack of memory.

[DNSB]

Originally Posted by Sarmat89

The Deflate algorithm uses almost no memory. It is impossible not to decode a valid deflate stream due to lack of memory.

I didn't dig that far into the memory use though I did notice that 7-Zip for one, does say that deflate would only need 2MB of memory. The error message I saw when using telnet was supposedly a lack of free memory error (telnet to the Glo and trigger a manual unzip operation) and since using the standard compression levels worked, I never dug further.

[Freeshadow]

-4, --shrink-insane
Set the compression level to "insane" using the zopfli compressor. You can define the compressor iterations with the -i, --iter option.
-i, --iter N(umber of) and NOT -N

the 7-Zip LZMA algorithm is available with the -N option

don't use the -N option then you get no lzma
and DON'T use it for creating with advzip -a OR --add because this will lead to said problems with meta.inf being shrinked too.
use ONLY advzip -z OR --recompress +the options I gave you with an already zipped! epub.

As for your command line, what is it intended to do?

it "tightens" the deflate oppression streams in an epub file that has to be already present.
Again: don't use advzip for creating epub for said need having meta.inf remain unshrinked, don't use the -N option. Rezip only.

Reading as in RTFM
https://www.advancemame.it/doc-advzip

[DNSB]

Originally Posted by Freeshadow

-4, --shrink-insane
Set the compression level to "insane" using the zopfli compressor. You can define the compressor iterations with the -i, --iter option.
-i, --iter N(umber of) and NOT -N

the 7-Zip LZMA algorithm is available with the -N option

don't use the -N option then you get no lzma
and DON'T use it for creating with advzip -a OR --add because this will lead to said problems with meta.inf being shrinked too.
use ONLY advzip -z OR --recompress +the options I gave you with an already zipped! epub.
it "tightens" the deflate oppression streams in an epub file that has to be already present.
Again: don't use advzip for creating epub for said need having meta.inf remain unshrinked, don't use the -N option. Rezip only.

Reading as in RTFM
https://www.advancemame.it/doc-advzip

I was under the impression that we were discussing zopfli since the advzip I have does not support the command line options you specified. Probably since it is a old GUI interface to zip & unzip. Anyhow, I ran some tests using advzip from the Advance projects site. Sadly, using zopfli compression is still very slow. And for png files, my two large PNG files take way too long to compress using either zopflipng or advpng making them the two of the slowest png compressors I've tested. Great at the job but for my purposes not worthwhile.