Possible replacement for EPUBCHECK

[Kayadelenium]

A question for the thread: what should a validator do with the findings that are not errors?

Two of you have now reported the same thing from three different directions, and I would rather ask you than keep guessing. This is not a bug report I am answering — it is a design choice I have not made yet, and it is easier to change before people rely on it than after.

What happened

epubcheck sorts its findings into fatal, error, warning, info and usage, and hides the usage ones unless you pass -u. A usage finding is not a defect: it names something your book does, so that you can decide. Ours are the same messages with the same ids.

• JSWolf, #206 / #209 — CSS-028 on correct @font-face: "It looks like an error message. It really should not be showing a place in the eBook that's correct." We were showing usage by default; epubcheck was not.
• JSWolf, #271 — after I matched epubcheck and hid them, the unused-image note went with them, and it read as a check that does not work.
• DNSB, #283 — the reverse again: our plugin showed OPF-088 on a nav, epubcheck showed nothing, and it looked like the two tools disagreed. They do not — epubcheck reports the identical message with -u, at the same line.

The same person asked for less and then for more. That is not anybody being inconsistent; it is a sign I have been adjusting the wrong dial.

How big it actually is

I measured it rather than guessed, on the 474 books I keep for regression testing:

• 219 of them have no error, warning or fatal at all — the books where epubcheck prints "No errors or warnings detected".
• 80 of those 219 (37%) still fill my plugin's panel with usage or advisory lines. That is 17% of the shelf.
• Median 4 lines on such a book; the worst has 527.
• Almost all of it is three ids: OPF-088 (3768 lines), OPF-086b (422), CSS-028 (265).

So DNSB did not find an unusual book. Roughly one book in six does this.

The part that is my fault, plainly

I keep asking this thread for "a book where epubveri reports something epubcheck does not". My own plugins show usage by default and epubcheck hides it, so anyone who does what I asked is guaranteed to find differences that are not differences. I built that and then invited people to walk into it.

And I got a fact wrong in my own code. The comment justifying "show everything" in my Sigil plugin says Sigil's results panel has a Type column, so every line says what it is. It does not. Sigil's panel has four columns — File, Line, Offset, Message — and the severity is carried only by the row's background tint: pale cyan for info, cream for warning, pale pink for error. When the panel holds a single row, as DNSB's did, that tint has nothing to be compared against and carries no information at all. My calibre plugin does have a real Type column, which is why this surfaced on Sigil first.

The options

1. Match epubcheck: hide usage and advisory by default, and print one line saying what was hidden — e.g. "3 usage notes and 1 advisory hidden — see settings".
Diffing the two tools then gives the same report from both, which is the whole reason people compare them. The summary line is what stops it becoming JSWolf's #271 again: you can still see that something is there.

2. Keep showing everything, but stop it looking like a finding — a separator, an indent, a different wording, the notes always last.
Nothing is hidden, so nobody has to discover a setting. But on Sigil I have only four columns and a background tint to work with, so how much I can really change is limited, and a cross-tool diff still shows differences that are not differences.

3. Two lists — defects in the panel, notes somewhere else.
Cleanest to read and the most work; on Sigil I am not sure it is possible at all, since a plugin gets one results table and nothing else.

4. Leave it and document it better.
I am listing this because it should be listed, but I do not believe in it: Doitsu told JSWolf in plain words that it was a usage message and not an error, and the next post still said it looks like an error message. Explaining harder has been tried.

Where I lean, and why

Option 1. It is the only one where somebody following my own request produces zero false reports, and it is the least work by a distance — both plugins already have the filter and the settings; it is a changed default plus one summary line. It also puts the two plugins and the command line on one behaviour, which is one fewer thing to explain.

What I dislike about it: a setting on Sigil lives in a JSON file with no interface, so "see settings" is a real errand rather than a checkbox. That is exactly why the summary line is not optional — without it, a hidden note is indistinguishable from no note, and silence is the one failure a user cannot notice.

Two things will not change whichever way this goes:

• It is a display filter, never a change to what the validator finds. epubsana — the repair tool — calls the library directly and is never filtered, and three of its repairs act on findings below error severity. Hiding a line from a panel must never make a repair quietly stop working.
• Advisory findings never move the verdict. A book that passes epubcheck passes epubveri. That is not up for a vote.

Please answer for the shape, not for this case

One thing worth saying before you answer, because it changes what a good answer looks like.

This is not really a question about OPF-088 in a Sigil panel. Whatever comes out of it becomes the rule for the command line, for both of my plugins, for epubsana (the repair tool that reads epubveri's findings), and for whatever integrations come after — an editor, a conversion pipeline, somebody else's tool. I would rather take an answer that still makes sense in a place none of us is looking at yet than the perfect answer for the one panel this came up in.

And being straight about the timeline, because it is part of the question. epubveri is what I work on every day at the moment, and that is exactly why it is moving fast enough to keep bothering you. It will not stay that way: once it is mature the daily work moves to another Rust project, and epubveri settles into being maintained rather than rebuilt. That is the normal shape of it, and it is the shape anyone integrating it wants — I am not going anywhere, but the pace of change should drop, and it should drop on purpose rather than because I got distracted.

The practical consequence is the same for you as for me. Every time I have to come back and change a default, it lands on people who had already learned the old one — that cost is yours, not mine. My side of it is only that the project I am on at that moment stops while I do it. Both get smaller the fewer times I have to return, which is why I would rather spend a week asking now than change this twice over the next year.

So if your answer is "this, and here is why it would still be right in an editor I have never used", that is worth far more to me than a vote.

What would help

You are the ones reading these panels on real books:

• Which option would you actually want, and does it change if you are in Sigil rather than calibre?
• When you open a book, are you asking "is this valid" or "tell me everything about this book"? I suspect different people want different ones and that is the real split.
• If usage notes were hidden by default, would a one-line summary be enough for you to go and find them?
• And the one I would most like answered: can you think of a workflow where the answer you just gave would be the wrong one? A conversion pipeline, a batch check over a whole library, somebody reading the output as a machine rather than a person. If a choice breaks down somewhere, I would rather hear it now than ship it and find out.

No hurry from my side. Nothing is broken while this is open — the fix for the report that started it is already out as 0.13.6, and this is only about what the panel shows. I would rather decide it once, slowly, and then leave it alone.

[Kayadelenium]

epubveri 0.13.7 — https://github.com/veripublica/epubv...es/tag/v0.13.7

One fix. An element whose entire content is a NO-BREAK SPACE was treated as empty here, and epubcheck does not treat it that way: Rust's trim counts * as whitespace, while XML's whitespace is four characters and does not include it.

Four checks read it the wrong way — an empty <dc:title> (OPF-055), any empty <dc:*> (OPF-072), an NCX <navLabel> (NCX-006), and a content document's <title> (RSC-005). The last one is an error, so a book epubcheck passes was reported invalid here. That is why this went out on its own rather than waiting for the end of the day.

Settled against epubcheck 5.3.0, one book per case, with an ordinary space as the control — both tools still report that one.

Nothing here could have found it: no book on my 474-book shelf holds such an element, and the 981-scenario corpus is unchanged either way. It came from epubsana, the repair tool, whose session read this source and asked a question it could not test and I could.

[patrik]

In the beginning of this thread, I made a quick&dirty script that recursively went through all my epubs, run checkdsk and yours, and when number of errors/warnings differed, added to a log. Running over night, it found none! My script was obviously faulty... :-) I have been meaning to redo it, but never got around to it.

Considering your skills, maybe you can create such a script for us? You know exactly what info is valueable to you and can create logs etc. (without any privacy info)

Just a thought... :-)

[Kayadelenium]

epubveri 0.14.0 — https://github.com/veripublica/epubv...es/tag/v0.14.0

No finding, ID, severity or position changes in this release. The corpus is unchanged — 603/603 exact-ID matches, no false positives on the 355 should-be-clean cases — and all 474 books on my reference shelf report byte-identically. If you only use the command line and read the human output, there is nothing here for you and you can skip the rest.

What changed is the JSON. Two things, both of which existed to stop the output making a claim it could not back.

Every count is now present, including zero. fatal, info and usage used to be dropped from summary when they were zero. That turned "the key is missing" into "this book has no usage findings", which is a false statement rather than an ambiguous one — you could not tell an absent count from an absent feature. A clean book now reports all five.

And the output says when it has been filtered. Without -u the usage findings are held back, exactly as epubcheck does — but until now the JSON gave no sign of it, so a "usage": 0 could mean either "none found" or "not shown". There is now a "suppressed": ["usage"] beside the counts saying which. It records that the filter was on, not that something was actually hidden, because the question it answers is "can I trust this number?".

Both come from the shared output convention the veripublica tools follow, which moved to 0.5.0; the convention field in the envelope says 0.5 now. Nothing was removed, so a reader that used summary.usage still works.

One thing under the surface, mentioned only because it touches the zip reader: 18 dependencies moved up, and one of them changed how the archive is decompressed. That is the part of the code every single book goes through, so rather than trust it I re-ran the whole reference shelf — 474 books, byte-identical output — and the corpus with it.

If you build from source: the minimum Rust is unchanged at 1.88, and it is now actually tested rather than only claimed.

[Kayadelenium]

epubveri 0.14.2 is out.

Someone ran their entire Apple Books library — 2,798 books — through both epubveri and epubcheck, and sent a full comparison that traced every disagreement back to the markup that caused it. Their numbers: the two tools reached the same verdict on 98.5% of the books.

0.14.2 closes 11 of the 12 false positives that run found, and 15 of the 18 checks we were missing. Three rows are not fixed: one I cannot reproduce, one is a deliberate decision, and one turned out not to be a gap at all.

Each was settled against the specification before any code moved, because epubcheck can be wrong too — and twice the spec turned out to be more precise than epubcheck is.

Release notes, with the reasoning for every change: github.com/veri ublica/epubveri/releases/tag/v0.14.2

[Doitsu]

You wrote the following in the release notes:

Originally Posted by Kayadelenium

<a> is transparent, so at flow level it takes flow content
(RSC-005). <a href="..."><div>...</div></a> was rejected, and <p> and
<ul> inside an <a> came from the same line. 3 books. epubcheck models
HTML's transparency with two grammar variants, a.elem.phrasing and
a.elem.flow (mod/html5/phrase.rnc:9-18); our a was the one transparent
element still pinned to phrasing content, where ins, del, canvas,
noscript, object and map already took flow.
The cost is stated rather than hidden: inside a <p>, epubcheck still
rejects <a><div> and we no longer will. That is this grammar's own
documented trade — "STRICT on vocabulary, PERMISSIVE on nesting" — and an
unknown element inside an <a> is still an error.

(emphasis mine)

I don't understand your decision to deliberately go against EPUBCheck.

For example, epubveri won't flag this code fragment in an epub3 book:

<body>
  <p>
     <a id="id">lorem
       <div>ipsum</div>
     </a>
  </p>
</body>

but it will flag the same code fragment if I replace the <a> tag with a <span> tag.

<body>
  <p>
     <span id="id">lorem
       <div>ipsum</div>
     </span>
  </p>
</body>

EPUBCheck will report the following error message for both cases:

 ERROR(RSC-005): Error while parsing file: element "div" not allowed here; expected the element end-tag, text, element "a", "abbr", "area", "audio", "b", "bdi", "bdo", "br", "button", "canvas", "cite", "code", "data", "datalist", "del", "dfn", "em", "embed", "epub:switch", "i", "iframe", "img", "input", "ins", "kbd", "label", "link", "map", "mark", "meta", "meter", "ns1:math", "ns2:svg", "object", "output", "picture", "progress", "q", "ruby", "s", "samp", "script", "select", "slot", "small", "span", "strong", "sub", "sup", "template", "textarea", "time", "u", "var", "video" or "wbr" (with xmlns:ns1="http://www.w3.org/1998/Math/MathML" xmlns:ns2="http://www.w3.org/2000/svg") or an element from another namespace

IMHO, you're actually trading away correctness on a case where the EPUBCheck reference implementation gets it right. You treat two elements with the same actual content-model obligation in that position differently only because one of them is textually flagged "transparent" and the flag is being applied without the context-sensitivity the spec attaches to it.

[JSWolf]

I found an error in epubsansa that also triggered an error in epubveri. I'm not going to post it here as I posted it in the epubsansa thread.

https://www.mobileread.com/forums/sh...22#post4608922

[Kayadelenium]

You are right, and it is fixed in 0.14.3, out now.

Transparent means the element takes its parent's content model, and I gave it flow content outright — which agrees at flow level and is simply a different rule inside a <p>. Your two fragments put that side by side, which is exactly why it was the first thing anyone noticed. It is modelled with two variants now, the way epubcheck does it, so <a> and <span> answer the same in the same position.

Thank you for catching it the day it shipped.

[Kayadelenium]

Originally Posted by patrik

Considering your skills, maybe you can create such a script for us?

Sorry this sat for a few days, patrik — it deserved a proper answer rather than a quick one.

A comparison like that is the most useful thing anyone can run against epubveri. The 0.14.2 release came almost entirely from one: someone ran 2,798 books through both tools and traced every disagreement back to the markup behind it. So yes, I want more of them. But before a script, the reason yours found nothing, because I have made exactly the same mistake with my own tooling and the traps are not obvious.

1. epubcheck hides usage messages unless you pass -u. epubveri shows them by default. Compare without -u on the epubcheck side and every usage note looks like something epubveri invented. Either pass -u to epubcheck, or leave usage out of both.

2. Compare the set of message IDs, not the number of errors. The counts differ for reasons that are not disagreements. When one mistake breaks a whole element, epubcheck can report it several times where epubveri reports it once, and a few messages are counted per rule on one side and per declaration on the other. A script that compares counts will either drown in those or, if it only logs when a total changes, miss a real difference that happens to leave the total the same. The question worth asking of each book is: is there an ID one tool reports and the other does not?

3. After a FATAL, epubcheck stops reading that file. Anything epubveri reports further down the same file is not a disagreement — epubcheck never looked. Those books are best set aside and checked by hand.

4. Distrust a result of zero differences. Over a large library, "no differences at all" is far more likely to mean one side produced no output — a wrong path, a crash swallowed by the script, a timeout — than that two independent tools agreed on every book. Before believing it, check that both outputs are non-empty for a few books, and deliberately feed it one book you know they disagree on to see the script catch it. I once had a checker of my own report "no book rejected" for a week over a shelf with thousands of errors on it, for exactly that reason.

I do have a tool that does this — it runs both, compares ID sets per book, and lists the count differences separately — but today it lives in the repository's test harness and needs Rust, Java and the epubcheck jar to build and run, which is not something I would ask you to set up. Whether it can be turned into something you just download and point at a folder is a fair question, and I will look at it.

In the meantime, if you redo your script with the four points above, what helps most is small: for each book where the ID sets differ, the two outputs (epubveri with --format json, epubcheck with -u and --json). No titles or file names from your library are needed — the IDs and the paths inside the book are enough to find the cause.

[Kayadelenium]

Originally Posted by JSWolf

I found an error in epubsansa that also triggered an error in epubveri.

JSWolf, I read your post in the epubsana thread — the EB Garamond, Saira, Tomorrow and Noto fonts, all named .ttf and declared as application/vnd.ms-opentype. I went to the specifications for this rather than to epubcheck, and the answer is that neither OPF is wrong, and font/otf is a correct type for those files.

What epubveri says about it. OPF-090 is a usage note, not an error: application/vnd.ms-opentype is an accepted spelling, just not the preferred one, and it does not change the verdict. The note names the preferred spelling for that row of the table.

Why that row gives font/otf. In the EPUB 3.3 core media types table you linked, application/vnd.ms-opentype appears on one row only, the OpenType row, and that row's preferred type is font/otf. The .ttf in a file name is not something the table looks at.

Why font/otf is not a mislabel for a TrueType file. A font with TrueType outlines is an OpenType font — the OpenType specification says such a font should start with the version number 0x00010000. RFC 8081, which registers both font types, lists 0x00010000 as a valid signature for font/otf as well as for font/ttf. So a TrueType-outline file fits both. font/ttf is also correct for those fonts; it is simply not the type the declaration they carry points to.

What would be wrong is the opposite case — a font with CFF outlines (its first four bytes are OTTO) declared as font/ttf — and epubveri does check that now: since 0.14.1 it looks at the font's own first bytes and will not suggest a type they rule out.

I checked this against real books too before answering: across 474 books, 47 fonts like yours (TrueType outlines, declared vnd.ms-opentype), and every one also carries the tables the OpenType format requires. A rule reporting them as mislabelled would have been wrong 47 times out of 47.

If you would rather have font/ttf in your own books, that is a perfectly valid choice and epubveri will not complain about it.

[Kayadelenium]

EPUBCheck 5.4.0 is out

EPUBCheck 5.4.0 was released yesterday, with EPUB 3.4 support and a long list of changes. Many thanks to rdeltour and the DAISY Consortium — and for fixing three of the issues we reported upstream while building epubveri.

I have started adapting epubveri to 5.4.0. Until that work lands, epubveri follows 5.3.0, so if you run both tools and they disagree on a book, 5.4.0 is the one to believe. The changes will be listed in the release notes as they arrive.

[Kayadelenium]

A correction to my post above. There is one known case where 5.4.0 is the one not to believe: it rejects aria-label and aria-labelledby on the nav elements of the navigation document (RSC-005, "attribute not allowed here"). Every Project Gutenberg EPUB 3 we tried is affected. It is a regression in 5.4.0, already confirmed by rdeltour and marked critical (w3c/epubcheck#1726), with a patch release being discussed. epubveri accepts that markup, which is correct.

[Kayadelenium]

epubveri 0.15.0 — caught up with EPUBCheck 5.4.0

Two days ago I posted that 5.4.0 had landed and that epubveri still followed 5.3.0. That gap is closed: 0.15.0 implements the 5.4.0 rules, and on EPUBCheck's own test suite the two tools now agree on 99.7% of the expected message IDs, with no false positive on any of its 368 clean books.

Errors that no longer fire
These are books 5.4.0 calls valid and epubveri called invalid until today.

• Obsolete but conforming HTML in EPUB 3 — the name attribute on a, border="0" on img, charset on script, type on style. These are usage notes now (RSC-036), not errors. If you have older EPUB 3 files, this is the one most likely to change your verdict.
• role="doc-pageheader" and role="doc-pagefooter", which are real DPUB-ARIA roles as of 5.4.0.
• RSC-013, RSC-014 and RSC-015 — the advanced URL fragment checks — are usage rather than errors, upstream and here.
• renditionage-spread-left beside the unprefixed page-spread-left on one spine item: one property written twice, not a conflict.
• A script element with type="module" no longer demands the scripted property.
• A trailing space on a URL with no path, such as href="http://example.com " — this one was ours alone and is described below.

New messages
Nothing here turns a valid book invalid, but you will see more output, especially with usage messages shown.

• OBS-001, the features EPUB 3.4 marks as outdated: an NCX, an OPF 2 guide, the old OPF 2 meta element, the collection element, rendition:flow / orientation / spread, font obfuscation, manifest content fallbacks, and the -epub- prefixed CSS properties. Most EPUB 3 books have at least one — 91 of the 474 on my shelf do.
• HTM-062 and HTM-063 on SVG xlink:href, which SVG 2 deprecated. The plain href spelling is now followed as a reference too, so a missing target hiding behind it is found.
• RSC-034 and RSC-035 on a script resource whose declared media type is not JavaScript, or is one of the legacy spellings.
• OPF-086c on the deprecated xsd, msv and prism prefixes.

New errors, and what they cost on real books
5.4.0 also tightened a few things: a roll publication's spine items must be fixed-layout documents, an EPUB 3 object element requires data, param is gone from EPUB 3, the package metadata element takes no attributes, and the audio core media types now read their codecs parameter. Across the 474 books I test against, exactly one gained an error — and EPUBCheck reports the same error on it.

If you use the Sigil or calibre plugins
All three pass -u, so they show usage messages. You will see new OBS-001 and HTM-062 lines on books that have not changed and are not newly broken. No plugin update is needed for this.

A 2,798-book comparison, and the two things it found
Someone ran epubveri over a 2,798-book library, book by book against EPUBCheck, and published the whole diff. That run found one genuine false positive of ours — the trailing-space URL above, which turned one book invalid on its own — and one place where we were silent: an unresolvable unique-identifier in the package document switched off every NCX check, including the play-order rules. On a book whose 34 navPoints were all playOrder="0", EPUBCheck reported those and we reported nothing. Both are fixed in 0.15.0, and my thanks to whoever did that work — a library that size finds things no test suite here can.

One difference against 5.4.0 that is not ours
As I mentioned in my earlier post: 5.4.0 rejects aria-label and aria-labelledby on the nav elements of a navigation document. It is a confirmed regression (w3c/epubcheck#1726), marked critical, and every Project Gutenberg EPUB 3 I have trips it. epubveri accepts that markup, which is correct.

Downloads and the full list of changes are on the release page: https://github.com/veripublica/epubv...es/tag/v0.15.0

[Kayadelenium]

epubveri 0.15.2 is out

The reader who compared epubveri and EPUBCheck across a 2,798-book library ran the comparison a third time, 0.15.1 against 5.4.0. For the first time, none of the differences was an error of ours. Setting aside EPUBCheck's own nav regression, the two tools reached the same verdict on 2,796 of the 2,798 books. 0.15.2 closes the differences that report still showed, plus several more I found while checking them. (0.15.1, released yesterday without a post of its own, fixed three media overlay checks.)

Errors that no longer fire
In each of these, EPUBCheck calls the book valid and epubveri called it invalid until today.

• A scheme-relative link such as href="//example.com/page" drew two errors, because it was treated as a missing file at the root of the container. It is a link to another site, and that is how it is handled now. No one reported this one; I found it while testing the items below.
• A dc:date that EPUBCheck accepts but that is not strict W3C date format: a time with no timezone (2010-01-01T00:00:00), a one-digit month or day (2010-1-1). In an EPUB 2 book this was an error. Nine books in that library had one.
• A class selector whose name is not a CSS identifier, such as span.- (one book in that library). Browsers ignore that whole rule, but EPUBCheck accepts it, so it is no longer an error here. With --advisory it is reported as ADV-012.
• A manifest item written out twice (same id, same file) now gets only the duplicate-id error, without an extra OPF-074 on top.

Errors EPUBCheck reports that epubveri now reports too
These can turn a book that passed epubveri into a failing one, but only where EPUBCheck already failed it.

• Reserved prefixes are checked per document type, as EPUBCheck does it. dcterms:, xsd:, a11y: and marc: are package prefixes, so using one in epub:type in a content document is an undeclared prefix (OPF-028); msv: and prism: in the package document are the same error the other way round.
• The scheme attribute on a meta element goes through the same check: scheme="xsd:string" gets the deprecated-prefix warning OPF-086c (five books in that library), and an undeclared prefix there gets OPF-028.
• In EPUB 2, two ids that differ only by surrounding spaces are the same id, and the duplicate is now reported. In EPUB 3, xml:id on an XHTML element is not allowed.
• A dc:date that does not exist, such as 2010-02-30, is now reported.

None of the 474 books on my own test shelf changed. EPUBCheck's own test suite results are unchanged too: the same message ID on 685 of 687 expected errors, and no false positive on its 368 valid books.

If you use the Sigil or calibre plugins
They download the new version by themselves, so no plugin update is needed. Across a 60-book sample, all three plugins show exactly what they showed with 0.15.1.

Two new advisory checks
With --advisory: ADV-011 reports a dc:date that EPUBCheck accepts but that is not in the W3C date format the specifications point to, and ADV-012 reports the class selector case above. Neither ever affects the verdict.

One for EPUBCheck
While porting its date parser I found that a dc:date ending in a bare dot, such as 2010-01-01T10:00:00., crashes EPUBCheck. It prints a stack trace and stops checking the rest of the book, and its JSON report comes out with no messages and zero errors. I reported it as w3c/epubcheck#1731. epubveri reports that date as invalid. The nav regression (w3c/epubcheck#1726) is still in 5.4.0.

Downloads and the full list of changes: https://github.com/veripublica/epubv...es/tag/v0.15.2