epubveri – standalone EPUB validator

[Kayadelenium]

epubveri 0.26.0 - two kinds of valid book that epubveri wrongly failed now pass, and a batch of checks now count and word their findings the way epubcheck does.

https://github.com/veripublica/epubv...es/tag/v0.26.0

If you use one of the plugins, there is nothing to do: all three fetch the latest release on their own.

Books that wrongly failed. None of these were reported by anyone; I found them while comparing epubveri with epubcheck. In an EPUB 3 book, an img with a srcset and no src was an error here; epubcheck accepts it. And an epub:switch inside a paragraph was "not allowed here"; epubcheck allows it there. One more wrong finding is gone, though it does not change a verdict: a link with only a query string, like href="?page=2", was also reported as a missing file. epubcheck reports only the query string (RSC-033), and now epubveri does the same.

What can now fail. This is a minor release because some books that passed here can now fail. In each case epubcheck already reports the same error, so a book that epubcheck passes still passes here.
- A link or image address that epubcheck's grammar rejects as a URL: a square bracket in a file name (image[1].jpg), a second # in the same address, or something before a colon that cannot be a scheme.
- An ARIA value with a stray space in it, such as aria-hidden=" true". epubcheck compares these values exactly.
- epub:type on the head or template element.
- In an EPUB 2 book, an SVG element inside an SVG title, desc or metadata, such as a tspan inside a title. SVG 1.1 allows only text there.
- In an EPUB 3 book, XHTML inside an SVG title that does not fit there, such as text followed by a body element.
- A Media Overlay clip with clipEnd="0" and no clipBegin, which makes the two equal.
None of the 544 books on my shelf changed.

Fewer repeated findings. A file that exists but is missing from the manifest (RSC-008) is now reported once, not once for every page that uses it. An element that comes before something required, such as a body with no head before it, is now one finding, "not allowed yet; missing required element", instead of two. Some foreign markup inside an SVG title, and an image address that appears in both src and srcset, were also counted twice; they are now counted once, as in epubcheck.

Only with usage messages turned on. In EPUB 3 SVG, ARIA attributes are now checked against the element's role: aria-checked needs a role that takes it, and role="checkbox" needs aria-checked. epubcheck reports these at the usage level, as informative notes, and so does epubveri. They never change a verdict.

How I know these match. For each change I built small test books, one shape per book or one shape per line, and ran them through epubcheck 5.4.0 and epubveri. That was more than 700 shapes in all, the two largest being 343 ARIA values and 283 ARIA role shapes. They now agree, finding for finding. On epubcheck's own 1086 test books, the number of books where the two tools report the same IDs but in different numbers went from 44 to 31. Most of the 31 that remain are the same finding counted differently: 17 are the font-face usage note, which epubcheck gives once per declaration and epubveri once per rule, a choice I made on purpose. Three I have not looked into yet.

If one of the new errors shows up on a book where you did not expect it, the element and attribute it names should make it quick to show me.

[Kayadelenium]

epubveri 0.27.0 - text encoding is now checked in every XML file of a book, not only in the OPF, and several wrong findings about encodings are gone.

https://github.com/veripublica/epubv...es/tag/v0.27.0

If you use one of the plugins, there is nothing to do: all three fetch the latest release on their own.

Wrong findings that are gone. Nobody reported these; I found them while comparing epubveri with epubcheck. An OPF that declares Shift_JIS, windows-1254, Big5, KOI8-R or most other encodings that epubcheck's XML parser can read got a fatal "unrecognized encoding" here, and that fatal stopped every other check of the package. epubcheck reports only that the encoding is not UTF-8 (RSC-028), and now epubveri does the same. A UTF-16 SVG, a UTF-16 NCX, or a UTF-16 chapter without a byte order mark was read as if it were UTF-8 and came out as a fatal "not well-formed" that hid everything after it; a UTF-16 container.xml drew an RSC-005 as well. Each of these now gets only the encoding finding, as in epubcheck. And one book wrongly failed: an OPF that wrote encoding = "ISO-8859-1" with spaces around the = sign got RSC-028, but epubcheck does not see a declaration there and reports nothing. Now neither does epubveri.

What can now fail. This is a minor release because some books that passed here can now fail. In each case epubcheck already reports the same error, so a book that epubcheck passes still passes here.
- A chapter, SVG, NCX, Media Overlay or META-INF file that declares an encoding other than UTF-8, for example a Latin-1 chapter (RSC-028). Before, only the OPF was checked.
- A UTF-16 chapter is HTM_058 in EPUB 2 books too, and also without a byte order mark.
- A declared name that epubcheck's XML parser does not accept at all, such as SJIS, cp1252, utf8 or UTF-32, is a fatal RSC-016, as in epubcheck. Note that utf8 without the hyphen is one of them.
None of the 562 books on my shelf changed, because none of them uses an encoding other than UTF-8.

How I know these match. epubcheck's XML parser knows 341 encoding names, and no rule predicts which ones it rejects, so I built a test book for each name, once with the name in the OPF and once in a chapter, and ran all of them through epubcheck 5.4.0. With 88 more books for the other cases, that makes 770, and epubveri reports the same message IDs as epubcheck on all of them, with two exceptions. epubcheck reads the OPF and container.xml twice, so it reports their encoding twice, and epubveri reports it once. And five EBCDIC names make epubcheck say "Unable to read file", because the Java runtime has no such character set; that is Java's answer, not the parser's, so I did not copy it.

The shelf and the benchmark. I added 18 books to my test shelf, so it now has 562, and I measured both tools again on all of them for the README and the benchmark page. epubveri is about 62 times faster over the whole shelf and faster on every book. It reaches the same verdict as epubcheck on every book except 49 Project Gutenberg books, which epubcheck 5.4.0 rejects because of a confirmed bug of its own (w3c/epubcheck#1726).

If an encoding finding shows up on a book where you did not expect it, the file it names should make it quick to show me.

crates.io: https://crates.io/crates/epubveri
npm: https://www.npmjs.com/package/@verip.../epubveri-wasm

[Kayadelenium]

epubveri 0.28.0 - MathML attributes are now checked, so a book with MathML that epubcheck rejects is now rejected here too.

https://github.com/veripublica/epubv...es/tag/v0.28.0

If you use one of the plugins, there is nothing to do: all three fetch the latest release on their own.

What can now fail. This is a minor release because some books that passed here can now fail. In each case epubcheck already reports the same error, so a book that epubcheck passes still passes here.
- MathML attributes. Until now every MathML element accepted any attribute at all. Now each element takes only the attributes epubcheck allows on it, so <mi title="...">, <math xml:lang="en"> or a definitionurl on <semantics> written in lowercase is an error (RSC-005), and so is an maction without its actiontype. The lowercase case comes from tools that treat XHTML as HTML: an HTML parser lowercases names, and MathML and SVG need some of them in camelCase. Attribute values are still not checked.
- An xml:base that points to a web address. epubveri only noticed one on the root element or in a <base> tag. epubcheck notices it on any element, and from that element to the end of the file every relative link points to the web address instead. An image after it is then a remote resource (RSC-006, and OPF-014 for the missing remote-resources property).
- In EPUB 2 books, XHTML inside the title of an SVG drawn in a chapter. The misplaced element itself was already reported; what is inside it is now checked as XHTML too.

Smaller changes. MathML whose only alternative text is a plain annotation, often TeX, now gets the accessibility note ACC-009, as in epubcheck: only alttext or annotation-xml counts. An SVG element name written in lowercase, such as lineargradient, now comes with the element and its parent named in the finding, so a repair tool can fix it; textpath and fegaussianblur, which were reported twice, are reported once.

None of the 562 books on my shelf changed, because none of them uses these shapes.

How I know these match. I built 151 small test books for these cases and ran each one through epubcheck 5.4.0 and epubveri. On 139 of them both tools report the same message IDs the same number of times. On 7 more the verdict is the same and only the count differs: in EPUB 2, epubcheck reports a misplaced SVG element a second time at its first child, and I did not copy that. The other 5 are two things I did not build, because no book on my shelf uses either: Content MathML inside annotation-xml is still not checked, and an xml:base that is a relative folder, such as sub/, is still ignored. epubcheck reports both, so on those 5 books it finds an error that epubveri misses.

If a MathML book gets a finding you did not expect, the attribute it names should make it quick to show me.

crates.io: https://crates.io/crates/epubveri
npm: https://www.npmjs.com/package/@verip.../epubveri-wasm

[JSWolf]

I think this is a mistake in epubcheck. But if not, epuveeri should be doing the same. epubcheck is displaying a message for every line in the @font-face declaration except the @font-face line. epubvari is only doing so for the @font-face line. The @font-face code posted starts at line 1 in the CSS. Here is the output for that @font-face from epubcheck.

OEBPS/css/9780593983768_style.css	2	3	USAGE CSS-028: Use of Font-face declaration.
OEBPS/css/9780593983768_style.css	3	3	USAGE CSS-028: Use of Font-face declaration.
OEBPS/css/9780593983768_style.css	4	3	USAGE CSS-028: Use of Font-face declaration.
OEBPS/css/9780593983768_style.css	5	3	USAGE CSS-028: Use of Font-face declaration.

@font-face {
  font-family: Courier;
  font-style: normal;
  font-weight: normal;
  src: url(../fonts/Courier-Regular.ttf);
}

Does epubvari check <title> to see if what's in it is possibly correct? If I have an eBook that has something like <title>This is the title</title>, that would be incorrect.

[Kayadelenium]

Originally Posted by JSWolf

I think this is a mistake in epubcheck. But if not, epuveeri should be doing the same.

Both counts are true. The difference is in what each tool counts. epubcheck reports CSS-028 once for every declaration inside an @font-face rule, at that declaration's line. That is why your four descriptors give four notes on lines 2 to 5 and none on line 1. epubveri reports it once per @font-face rule, at the @font-face line. It is a usage note in both tools. It says a font is declared, not that anything is wrong, and it never changes the verdict.

Neither the EPUB specification nor the CSS one says how a validator should report this. CSS-028 is epubcheck's own informational note, so this is a choice, not a rule. I chose once per rule because you said back in #206 that this note already looks like an error in the plugin panel, and repeating it four times for one font would make that worse.

While checking epubcheck's code for this, I found something that looks like a real bug there. epubcheck attaches the font file to each note, and on a second @font-face rule the declarations that come before its own src carry the first rule's font file. That suggests the original intent was one note per font, but only epubcheck's maintainers can say. So I opened an issue at W3C asking which counting they intend, with the code lines and a small test book:

https://github.com/w3c/epubcheck/issues/1741

When they answer, epubveri will follow their intent, whichever way it goes.

If you want this to change, that issue is the right place for it. epubcheck's behaviour is decided there, by W3C, not here, and epubveri follows it. A comment from someone who reads these reports on real books every day will carry more weight there than mine. Any GitHub account can comment, and it is free. You don't need to agree with me; whether you prefer one note per rule or one per declaration, saying so there is what gets it settled for everyone.

Originally Posted by JSWolf

Does epubvari check <title> to see if what's in it is possibly correct?

No, and epubcheck doesn't either. No specification says what a title has to say, so there is nothing to check it against. A content document's title is often a chapter name, not the book's title, and a book can really be called "This is the title". Any guess would fire on valid books, and I don't add a note unless it is true every time it fires.

What both tools do check: an empty title in an EPUB 3 content document is an error (RSC-005), and an empty dc:title in the package document is a warning (OPF-055).