epublift — EPUB toolbox: modernise, shrink, kepub, validate, repair, archive

[Kayadelenium]

Three replies here came in while I was working on the release. Here are my answers.

Originally Posted by PeterT

Any plans on publishing epublift to cargo.io? My approach to all of you tools is doing a cargo build rather than a download of a precompiled binary

Yes, I will publish it on crates.io, as epubveri and epubsana already are. Until then it installs straight from the repository:

cargo install --locked --git https://github.com/ePubLift/epublift --tag cli-v3.3.0 epublift --features epub34,metadata,pdf-ocr,markdown,validate

Keep the features: they are what the release binaries are built with. Without validate there is no check command, and without metadata no ISBN lookup.

Originally Posted by JSWolf

What would really help to stop the complaints is to make the defaults to keep the existing images and ZIP format for the container

The container part is already the case. Optimize has only ever written an ordinary ZIP. The Zstandard container is an experiment that is not in the released program or on the web page at all. The one place Zstandard is used is Archive, which writes a separate .eparc file for storage, and only when you ask for it.

The images part: you are right, and I am changing it. Keep original will become the default, on the web page and on the command line, and WebP will be something you choose. It will come together with the conversion fixes.

Originally Posted by KevinH

So just because the Epub spec supports a format, do not expect the publishing community to start throwing away gif, jpeg and png any time soon.

I don't expect them to. epublift is not aimed at publishers; it works on books readers already have. That is also why EPUB 3.4 output, with AVIF or JPEG XL images, is only there if you ask for it (--target 3.4), and once the new default is in, WebP will be opt-in too. On JPEG XL in Chrome, that changed this week: Chrome 155 ships JPEG XL decoding, using jxl-rs, a decoder written in pure Rust. Readers built on Chromium will only get it as they update, though, and e-ink devices take longer still, so your point about how slowly publishing moves stands.

[JSWolf]

Oh and can you build in options to convert from Webp, AVIF, and JXL into JPG or PNG? If ePub start being in the wild with these new image formats, a lot of people won't be happy. But that will be happy if they can convert the images to a format that can be used.

[Kayadelenium]

Yes, I have an idea for a tool that converts between EPUB3 and EPUB2 and allows you to select image formats during the process. Since I haven't finished the core tasks yet, I can't attend to the other work.

- Fixing epubveri errors
- Writing plugins for epubsana
- Developing kepubverto and then writing its plugins
- ....

Phew Captain... there is so much work to do.

[JSWolf]

Originally Posted by Kayadelenium

Yes, I have an idea for a tool that converts between EPUB3 and EPUB2 and allows you to select image formats during the process. Since I haven't finished the core tasks yet, I can't attend to the other work.

- Fixing epubveri errors
- Writing plugins for epubsana
- Developing kepubverto and then writing its plugins
- ....

Phew Captain... there is so much work to do.

That's all wrong. The idea is to convert the images regardless of the ePub version. Most programs that handle ePub3, won't handle those formats. So allow conversion regardless of the ePub version. I don't want an ePub3 don't work because the image format(s) are not supported.

Just allow image conversion without converting the ePub in any other way.

And if you want the ePub to remain as ePub3 but be backwards compatible, these image formats, need to be converted in place and the OPF updated.

The thing is, these new (new to ePub) image formats should never have been sanctioned. The W3C are becoming really stupid (IMHO).

[Kayadelenium]

epublift 4.3.0 (command line) and 1.28.1 (web page) are out, and https://epublift.itpax.net is already running them. This is my first post here since 3.3.0, so it covers everything since then. As before, nothing touches the book you start from: Optimize, Repair and the metadata editor all write a new file, and Validate and Repair run in your browser without uploading the book.

Images are kept as they are by default

I said in my last reply that this would come with the conversion fixes, and it has. Optimize now upgrades the structure and leaves JPEG and PNG untouched, on the web page and on the command line. WebP is something you ask for: the Keep original choice is preselected on the web page, and on the command line it is --webp. If you have a script that relied on WebP output, add that flag.

JSWolf, on your last post: converting WebP, AVIF or JPEG XL images that a book already has into JPEG or PNG is not there yet. You are right that it should not depend on the EPUB version. It should convert the images, update the OPF in place, and leave everything else in the book as it is. It is on my list, but I can't give a date. The one thing I still have to decide is the output. PNG keeps the picture exactly but usually makes the file larger, and JPEG keeps it small but loses a little again, so the choice should probably be yours.

Optimize no longer makes valid books invalid

These are the conversion problems I mentioned last time. Until 4.0, most valid EPUB 2 books came out of Optimize with validation errors that the upgrade itself had added. They came from the upgrade carrying EPUB 2 markup that EPUB 3 does not allow. The upgrade now does this:

• It translates the EPUB 2 metadata (opf:role, opf:file-as, opf:scheme) to the EPUB 3 form, instead of leaving attributes EPUB 3 forbids.
• It writes out the named characters (the non-breaking space and the rest) that the old DOCTYPE declared and the new one does not.
• It builds the navigation document from toc.ncx with its labels escaped and its links encoded.
• It writes the layout attributes HTML dropped (an image's width="40%", a table's cellspacing, cellpadding and border, a cell's valign, a line break's clear) as the CSS the HTML Standard maps them to. That CSS goes in one small stylesheet, epublift-presentational.css. It also renames the obsolete font, big, center and strike elements to their HTML counterparts.
• It finishes with a pass of epubsana, the family's repairer, for what EPUB 3 requires and EPUB 2 did not (an empty title, an empty unique identifier, an id that is no longer valid).

Where I could not be sure, I left things alone. If the book's own CSS already sets the property an attribute would set, or one of its selectors reads the attribute, the attribute stays as written. The book then stays invalid over it, and the command line's report says how many were left and why. font color and size="7" are left too.

Repair first, then Optimize

Optimize fixes what the upgrade itself would break. Defects the book already had are Repair's job, and Repair asks before any change that needs a decision. The web page's tabs are now in that order: Validate, Repair, Metadata, Optimize. For the cleanest result, repair the book, then optimize the repaired copy.

Repair (epubsana 0.34.0) also learned new fixes since 3.3.0. Two of them come from books in my own library. One is a list placed directly inside another list instead of inside its item, which is common in tables of contents. The other is an image declared as one standard type but holding another, such as a PNG declared as JPEG.

Smaller changes

• --kepub now uses kepubverto, the converter from the Kobo forum thread. It adds its Kobo markup without rewriting the rest of the document, and it adds no validation errors of its own.
• The report file goes next to the output, not next to the input. Without -o nothing changes.
• Three meta options were renamed so that each name means the same in every tool of the family: enrich --lang is now --assume-language, enrich --cover is --include-cover, and show --json is show --format metadata. The old spellings still work in 4.x, with a warning.
• Validate runs epubveri 0.31.0 and Repair runs epubsana 0.34.0, the same versions as in their own threads.
• The HTTPS library used for metadata lookups was updated for two published security advisories.

PeterT, crates.io is still on my list. One dependency is only available from its git repository, and crates.io refuses that. Until it has a release, the install command is the same as before with the new tag:

cargo install --locked --git https://github.com/ePubLift/epublift --tag cli-v4.3.0 epublift --features epub34,metadata,pdf-ocr,markdown,validate

Full release notes: cli-v4.3.0, web-v1.28.0 and web-v1.28.1. If a book comes out of Optimize with more errors than it went in with, please tell me here. Validate, epubcheck or your own editor will show the difference, and your original is untouched.