Possible replacement for EPUBCHECK

[JSWolf]

I would rather have usage messages as an option. The @font-face message does look like an error with an incomplete error message. What other usage messages are there?

[Doitsu]

Originally Posted by Kayadelenium

Which would you rather have? Anyone with an opinion, now is a good moment — it's a default, and defaults are hard to change later.

You could mirror EPUBCheck and only output USAGE messages if epubveri is invoked with the -u/--usage parameter.

As you know, EPUBCheck has these options:

-f, --fatal              = include only fatal errors in the output
-e, --error              = include only error and fatal severity messages in ouput
-w, --warn               = include fatal, error, and warn severity messages in output
-u, --usage              = include ePub feature usage information in output
                           (default is OFF); if enabled, usage information will
                           always be included in the output file

Since you added ADV messages, maybe you could also add an -a/--advice parameter (default should be OFF).

It totally forgot that there already is an advisory parameter.

[JSWolf]

If ADV messages are to be off by default, then I think the plugins should allow toggling of these options.

[Doitsu]

Originally Posted by JSWolf

If ADV messages are to be off by default, then I think the plugins should allow toggling of these options.

If Kayadelenium makes them optional, I'll add a corresponding plugin option.

EDIT: They were already optional, but activated by default.

[JSWolf]

Originally Posted by Doitsu

If Kayadelenium makes them optional, I'll add a corresponding plugin option.

Thanks!

[JSWolf]

The output differs between epubveri and epubcheck. It may just be the warnings that are different.

epubveri has 5 warnings and epubcheck has 71 warnings. The number of errors matches.

Because the scrambled ePub is too large to attach, here is a link to download it. The link is good for 7 days.

https://www.transfernow.net/dl/20260821otihwTBI

[Kayadelenium]

Thanks — #216 is fixed, and it's in tonight's release.

You were right that only the warnings differed. A toc link whose fragment target doesn't exist was being dropped from the reading-order check altogether, so we reported 5 where epubcheck reports 71. Both now report 71, and the 67 errors were identical throughout. Your note that the error counts matched is what narrowed this to a single rule — that's the most useful form a report can take, so thank you.

Each finding now also names the offending link and points at it, instead of five identical lines on the <nav> element.

Tonight's release carries several other fixes as well, found by running epubveri against W3C's epub-tests suite. I'll post the details with it.

[Kayadelenium]

epubveri 0.9.29 is out — crates.io, npm, and binaries for 8 platforms.

This one is larger than usual, and most of it came from one thing: I ran epubveri against W3C's epub-tests — their 209 reading-system conformance publications — and diffed it against epubcheck. That found seven defects that neither my epubcheck-corpus runs nor my 385-book shelf could see, for a reason worth stating: no book on that shelf carries a media overlay, a rendition:layout spine override, or a viewport meta in more than one file. Exact message-ID agreement with epubcheck across those 209 publications went from 172 to 202.

JSWolf's report (#216) is in this release. A toc nav link whose fragment target doesn't exist was dropped from the reading-order check entirely, so we reported 5 warnings where epubcheck reports 71. Both now report 71, and each finding names the offending link instead of pointing five identical lines at the <nav> element. Your note that the error counts matched is what narrowed it to a single rule — thank you.

One change you may notice in your own scripts. Seventeen message IDs are now spelled with an underscore: HTM_054–HTM_059, HTM_061, and MED_007, MED_010–MED_018. That is how epubcheck prints that particular block, and matching its output is the whole reason epubveri uses its ID scheme at all. If you grep for MED-013, it is MED_013 now.

New findings you may see on books that were quiet before: a missing fallback on an <object> naming a foreign resource (RSC-032), a <source> inside <audio>/<video> whose type disagrees with the manifest (OPF-013), file: URLs in more places (RSC-006, RSC-030), and the fixed-layout viewport checks (HTM-046, HTM_060b). One went the other way and removes a spurious warning: OPF-013 now ignores media-type parameters, so type="audio/mpeg; codecs=mp3" against a manifest audio/mpeg is a match.

On usage messages (#211–#215): decided, and it follows Doitsu's proposal — a -u/--usage flag, off by default, as in epubcheck. It is written and tested but did not make this release; it ships in the next one. One clarification for the plugin side: --advisory is already opt-in and already off by default, so there is no need for a second flag alongside it.

Also fixed for the next release: a confusing RSC-005 message when a content document's root element has no namespace, reported on GitHub by BeckyDTP.

[JSWolf]

With the previous version, the warnings were together and the errors were together. Now the warnings and errors are intermixed. Can the warnings and errors be separated again with the warning first? Thanks.

[JSWolf]

I've found a small error this time. This was an ePub3 eBook but I changes it to ePub2 and left this in. I'm editing some short stories from Amazon to put together into the complete eBook and they would be better as ePub2 when using EpubMerge.

<spine page-progression-direction="ltr" toc="ncx">

The error is correct, the line is correct, the column is wrong. It should be 8 instead of 1

[JSWolf]

epubveri reports no errors.

epubcheck reports

Spoiler:

Validating using EPUB version 2.0.1 rules.
ERROR(RSC-005): Void (The Far Reaches collectio - Veronica Roth.epub/META-INF/encryption.xml(3,14): Error while parsing file: element "encryption" incomplete; expected element "enc:EncryptedData" or "enc:EncryptedKey"

Check finished with errors
Messages: 0 fatals / 1 error / 0 warnings / 0 infos

EPUBCheck completed

This is the file encryption.xml.

Spoiler:

<?xml version='1.0' encoding='utf-8'?>
<encryption xmlns="urnasis:names:tcpendocument:xmlns:conta iner" xmlns:enc="http://www.w3.org/2001/04/xmlenc#" xmlns:deenc="http://ns.adobe.com/digitaleditions/enc">
</encryption>

encryption.xml should have been deleted when the obfuscated embedded font was deleted.

And I have one suggestion for a new warning. I'd like a warning for any images not being used.

[Kayadelenium]

Thanks for the three reports last night — all measured, two fixed, and the third is a question I would rather put to the thread than answer on my own.

#220, the wrong column. Fixed. You were right and the cause was narrow: for an attribute fault we reported the position of the *element*, not of the attribute we had just named in the message. So `<spine page-progression-direction="ltr" toc="ncx">` said column 1 when the thing to look at starts at column 8. Worth noting that epubcheck does not point at column 8 either — its SAX locator reports the fault at the character *after* the start tag's `>`, so on a 52-character line it says column 53. There was no third answer to converge on, just a more useful one and a less useful one. Across 385 books this moves 11,332 positions, always forward, always on the same line, and it changes no finding, no ID and no verdict.

#221, encryption.xml. Fixed. We parsed the file and checked several things about it, but never asked whether it had any content at all. epubcheck's grammar is one line — `<encryption>` requires one or more `EncryptedData` or `EncryptedKey` children and admits nothing else — and we now match it exactly on all three shapes (empty, self-closed, and a foreign child, which draws two errors, not one). Only 2 of my 385 test books carry an `encryption.xml`, and both are fine, so this is the kind of thing that only shows up when someone edits a book, as you did. Thank you for that one.

The unused-images request I have not done, and I want to be careful rather than quick about it. epubcheck has no such check, so it would be an opt-in advisory, and my bar for those is that the finding has to be *true* every time it fires. An image can be referenced from CSS, from inside an SVG, from a media overlay or from script, so "this image is unused" is easy to get wrong in a way that is worse than not saying it. If someone has a book where an unused image really is a problem, I would like to see it.

#219, the ordering — this is the question. Your observation is real and I can measure it. epubveri sorts strictly into document order (file, then line, then column). epubcheck does not sort at all: it emits in the order its checks run, so severities end up clustered as a side effect. On 23 of my books that have both warnings and errors, the severity sequence differs between the two tools on 10 of them — where epubcheck prints one run of warnings then one run of errors, we alternate; on one book our output breaks into 34 alternating blocks.

But note that neither tool actually does what you asked for. "Warnings first, then errors" is not epubcheck's order — of those 23 books it happens to come out that way on 15, comes out errors-first on 3, and is interleaved on 5. So this is a real design choice rather than a bug I can just fix.

The two candidates:

- Document order (today). Best if you are working through a file top to bottom, and best for a plugin that moves the cursor to each finding.
- Grouped by severity. Best if you are triaging — deciding what kind of trouble a book is in before fixing anything.

I lean towards keeping document order as the default and adding severity grouping as an option, but I hold that view lightly, and this is the moment to say so: a default is cheap to change today and expensive once the Sigil and calibre plugins have people relying on it.

One more thing, because I suspect it is the real problem behind the request. On a book with 3,000 errors and 4 warnings, grouping only helps if the warnings come *first* — otherwise they are still off the top of the screen. That suggests what is actually wanted is a short summary at the end of the report, counts by severity and by message ID, which works no matter how long the list is. Would that help, alongside or instead of the ordering change?

Either way the machine formats (`--format json`, `--format ids`) stay in document order and stay unfiltered, so the plugins can arrange things however they like.

[JSWolf]

Can we have a warning about encryption.xml existing without encrypted fonts?

[Doitsu]

Originally Posted by Kayadelenium

The unused-images request I have not done, and I want to be careful rather than quick about it. [...] If someone has a book where an unused image really is a problem, I would like to see it..

This feature already exists both in EPUBCheck and epubveri for epub3 books.

epubveri:

USAGE OPF-097: 'XXX' is declared in the manifest, but no content document references it

EPUBCheck:

USAGE(OPF-097): Resource "XXX" is listed in the manifest, but no reference to it was found in content documents.

Originally Posted by Kayadelenium

#219, the ordering — this is the question. [...] I lean towards keeping document order.

I'm also in favor of keeping document order as the default.

Originally Posted by Kayadelenium

[...] so the plugins can arrange things however they like.

epubveri plugin users can easily group messages by clicking the Message headings in Sigil and Calibre. In Sigil, all messages shown in the Validation window can be exported as a .csv file and in Calibre the messages can be copied to the system clipboard as tab-delimited text.

Both methods allow users to sort the messages with their favorite spreadsheet program.

Originally Posted by JSWolf

Can we have a warning about encryption.xml existing without encrypted fonts?

I fail to see the point of it. An unused encryption.xml file will cause no problems whatsoever. You keep requesting features that nobody but you is interested in!

[Kayadelenium]

Three answers, all measured, and one of them settles a disagreement rather than taking a side.

Ordering. I have gone with grouped by severity, most serious first as the default, and added `--sort document` for the old behaviour. Doitsu, you argued for document order and JSWolf asked for warnings first, so let me give the reasoning rather than just the outcome: the set that makes a book invalid is the set you act on, and it is what the verdict line counts. Inside each group the file order is untouched, so it is one pass down the errors, fix, re-run, then one pass down the rest — each still reading top to bottom.

Neither arrangement is epubcheck's, which is worth saying because it looks like it might be. epubcheck does not sort at all: it emits in the order its checks run, and severities cluster as a side effect. On 23 of my test books carrying both severities its sequence differs from mine on 10, and of those 23 it happens to come out warnings-first on 15, errors-first on 3, and interleaved on 5.

Doitsu — please pin whichever you prefer rather than relying on my default. `epubveri --sort document -i book.epub` (or `--sort=document`) works alongside `-u` and `--advisory`. I would rather your plugin be immune to my default than have you depend on my agreeing with you today, because that agreement would not protect you from a future change and an explicit flag will. One caveat: the flag does not exist before this release, so if you support an older binary you would need a fallback — an unknown option is an exit-2 usage error, not a warning.

The same goes for anyone building on the output more generally: the human report is written for a person and is free to change — its wording, its order, and as of this release which severities it shows. `--format json` is the stable, documented one, and it is never filtered or reordered. There is now a page about exactly this: `docs/INTEGRATING.md`.

Unused resources — you are both right, about different books. Doitsu is correct that OPF-097 already covers this; JSWolf is correct that he never sees it. I built the same book twice, one unreferenced stylesheet, and ran both tools:

- EPUB 3 — epubcheck and epubveri both report `USAGE OPF-097` ("declared in the manifest, but no content document references it"). Exact parity.
- EPUB 2 — neither tool reports anything.

JSWolf, you work in EPUB 2, which is why the check looks missing to you. So the open question is not "should someone build this" but "should that question be asked of an EPUB 2 book at all?" — which is a deliberate step beyond epubcheck, so it would live behind `--advisory` if it happens. Note it is a `usage` finding either way, so from this release you need `-u` to see it.

encryption.xml. JSWolf, both shapes your case can take are already errors in epubcheck, and epubveri was reporting neither — that is now fixed:

- the entry was deleted with the font, leaving `<encryption>` empty — epubcheck says `element "encryption" incomplete`;
- the entry stayed and points at a font that is gone — epubcheck says `RSC-007: Referenced resource … could not be found in the EPUB`.

I had it as an INFO note saying the file was encrypted, and called such a book VALID. Both now match epubcheck exactly, along with three other `encryption.xml` shapes I checked while I was there. So this was not a new feature — it was a real miss on my side, and it took your report to find it. Thank you.

Doitsu, on "an unused encryption.xml causes no problems": that is true of the third shape, a valid one pointing at files that are actually there, and nothing new is needed for that. It is not true of the two above, where epubcheck has been erroring all along.

All of this, plus a column that now points at the offending attribute instead of its element (JSWolf, #220), is in tonight's release.