Possible replacement for EPUBCHECK

[Kayadelenium]

Originally Posted by patrik

Here is the complete book (scrambled).

Thank you — the scrambled book was exactly what was needed, and more than I asked for. It took about ten minutes to reproduce with it.

The short version: the difference isn't in your book, and it isn't a check I'm missing.

Your book contains 374 actual epub:type attributes. I report 335 of them. EPUBCheck reports 3339.

I'd rather show that than assert it. Take text/part0020.html and cut it to its first 14 lines: both tools report 3, and the file has 3. Cut it to 20 lines instead — the six extra lines are plain <p class="paranoindent"> paragraphs with no epub:type on them at all — and EPUBCheck reports 9. Line 15 in that file has zero epub:type, and the position EPUBCheck gives for its error there (15,325) is the closing </p>.

It starts once an HTML5 element that EPUB 2 doesn't allow has appeared — in your book a <section> and an empty <header> near the top of each file. After that, one further message per following element, all labelled epub:type.

I'm not going to tell you whose defect that is. EPUBCheck's verdict is right — the book genuinely is invalid, and at each of those points the content model genuinely doesn't match. Whether the message should name an attribute that isn't there, and whether that belongs to EPUBCheck or the validator underneath it, I haven't established and don't want to guess at in public.

Everything else in the two outputs matches exactly: 75 element errors to 75, 15 id datatype errors to 15, 2 playOrder to 2.

The 39 between my 335 and the real 374 are one <section epub:type="chapter"> per file. I report that <section> as a disallowed element and don't then also report its attribute — and neither does EPUBCheck, which gives 0 for that case in isolation.

One real gap on my side, from your book: page_styles.css has src: url(fonts/00001.ttf) and no such file exists. EPUBCheck flags it, I say nothing. I wrote the fix, it broke three cases in EPUBCheck's own test suite, and I backed it out rather than trade one gap for three. It's on the list properly now.

[DNSB]

For what it may be worth, most of the issues in the scrambled ePub are from using calibre's convert-ebook function to convert an azw3 file created by Amazon from a submitted ePub3 original into an ePub2. The current calibre version does a bit better job (calibre 6.3.0 is a bit long in the tooth) but not that much of an improvement. KindleUnpack comes a lot closer to the original ePub.

[Kayadelenium]

That fits the file exactly: the package declares version="2.0" while the content is still EPUB 3 — <section>, <header>, and epub:type values from the EPUB 3 structural semantics vocabulary. The conversion moved the package version down and left the markup where it was, which is why both tools have so much to say about it.

Thanks — useful to know it came through azw3.

epubveri 0.9.6 Out now, and three of it came from this thread:

- A @font-face src pointing at a font the book doesn't contain is now reported (RSC-007). From patrik's book above — epubcheck flagged it, I said nothing.
- --advisory now flags a CSS type selector that names no real element (JSWolf, post #92): h4a { } is valid CSS that matches nothing. Advisory only, never affects the verdict — across 84 real books it produces one finding.
- <menu> is no longer rejected in EPUB 3 — a false positive of mine; HTML5 has it and epubcheck accepts it. It stays an error in EPUB 2, which genuinely doesn't include it.

Also a proper message for a <rootfile> whose full-path is missing or empty, instead of only "no usable rootfile found".

Coverage is now 191 of 194 live epubcheck checks (~98%). The three that remain are deliberate scope decisions, all listed in docs/COVERAGE.md.

Binaries for 8 platforms: https://github.com/veripublica/epubveri/releases/latest
Browser demo, nothing to install, the file never leaves your machine: https://veripublica.github.io/epubveri/

If epubveri reports something on a book epubcheck accepts, please post it — ideally with both outputs as plain text for the same file.

[JSWolf]

Originally Posted by patrik

Here is the complete book (scrambled).

What OS are you using? calibre 6.3.0 is rather old and you shroud upgrade if possible.

[DiapDealer]

Originally Posted by JSWolf

What OS are you using? calibre 6.3.0 is rather old and you shroud upgrade if possible.

Irrelevant to the task at hand in this thread.

[JSWolf]

Originally Posted by DiapDealer

Irrelevant to the task at hand in this thread.

Better code, less errors with a newer version of calibre.

[Kayadelenium]

epubveri 0.9.7 is out

One fix, and it changes no verdicts: stray text in an EPUB 2 body is now reported at the text, not at the element around it. Before, sixteen loose runs in one file all pointed at the same line and column — no way to tell them apart or jump to one. Same findings, same counts; only the positions are right.

Downloads (8 platforms): https://github.com/veripublica/epubv...ses/tag/v0.9.7
Browser demo: https://veripublica.github.io/epubveri/

As always: if our output disagrees with epubcheck on a book, posting both outputs as plain text is the most useful thing you can do.

[Jellby]

Originally Posted by JSWolf

Better code, less errors with a newer version of calibre.

Perhaps, but I believe the "task at hand" is not "how to get a good epub conversion" but "how to flag all the issues in a defective conversion".

[DiapDealer]

Originally Posted by JSWolf

Better code, less errors with a newer version of calibre.

Less errors won't help fine tune new error-checking software.

[Kayadelenium]

epubveri 0.9.8

This one came out of running epubveri and EPUBCheck over the same 104 books and diffing them by message ID. Most of it is us reporting too much.

• A misplaced element no longer drags its contents down with it. If an element was rejected as "not allowed here", everything inside it was then checked against the parent's rules — so nested <span>s inside one misplaced <span> each got their own error. One book: 1259 errors from us, 661 from EPUBCheck. It is now 631. The same fix stopped us missing things, too — a <div> inside that <span> is legal where the span was rejected and illegal inside a span, so nobody reported it.
• An empty <dc:identifier/> drew three errors on top of the real one. EPUBCheck reports one. So do we now.
• Message IDs now match EPUBCheck's exactly. It has lettered IDs — HTM_060b, OPF-004a through f, OPF-007a/b/c, RSC-006b, RSC-007w — and we were emitting the bare code, or in one case an ID that does not exist at all. If you filter output by ID, this matters.
• Two of those turned out to be real gaps rather than renames: a prefix that is not a valid name was never checked, and a remote resource nothing references was not reported at all.

One correction worth making in public: our published coverage figure was measured against a denominator that was missing 17 of EPUBCheck's message IDs, so it was too flattering. Fixed, the checks behind it implemented, and it now reads 207 of 210 — but the old number was wrong, not the new one.

104 real books, 100 of them now agree with EPUBCheck on the exact set of message IDs.

Downloads (8 platforms): https://github.com/veripublica/epubveri/releases
Try it in the browser: https://veripublica.github.io/epubveri/

If you hit a disagreement, the most useful thing you can send is both tools' output as plain text for the same book — that is what turned the last few of these into fixes.

[Doitsu]

@Kayadelenium

I noticed several differences when checking this very old epub3 sample file.

See attached logs for details.

[Kayadelenium]

epubveri 0.9.9 is out

Doitsu — thank you, those two logs were worth more than any description. All three differences were mine, and all three are fixed.

• epub:trigger is a real EPUB 3 element and I did not have it at all. Each of the six occurrences drew five errors from me — the element, its action and ref, and two more for the ev: namespace — where EPUBCheck draws one deprecation warning. It is now transcribed from EPUBCheck's own epub-trigger.rnc, and the deprecation warning (which was always right) is unchanged.
• <video> and <audio> are transparent. The ordinary fallback shape — sources, then a <div> saying "your reading system does not support this" — was rejected, because I had only modelled the phrasing variant of their content model, never the flow one.
• Which prefixes count as "reserved" depends on the document. EPUBCheck reserves one set in the package document, a different one in a content document, and none at all in a Media Overlay. I used the union everywhere, so the sample's epubrefix="media: ..." on a content document drew a warning it should not.

On that file: 38 errors before, 2 after — the same 2 EPUBCheck reports.

Also in 0.9.9, and the larger change: EPUB 2 documents no longer accept 195 global attributes XHTML 1.1 does not have — the event handlers, RDFa, microdata, ITS, ARIA and role. OPS 2.0.1 was finalised in 2010 and WAI-ARIA only became a W3C Recommendation in 2014, so the EPUB 2 schema could never have carried them. Each one was checked against EPUBCheck individually, and two (tabindex, xml:space) turned out to belong to specific elements rather than to everything, so they moved rather than disappeared.

Since the sample is CC-licensed, it is now part of the test library here — the only book in it with epub:trigger, a video fallback and Media Overlays, so this particular class of mistake now has a permanent guard.

Downloads (8 platforms): https://github.com/veripublica/epubveri/releases
Try it in the browser: https://veripublica.github.io/epubveri/

Both tools' output as plain text for the same book remains the single most useful thing anyone can send — it is what turned this one into three fixes rather than a guess.

[Doitsu]

@Kayadelenium:

One more issue where epubveri and EPUBCheck differ:

OPF:

<dc:title id="title">Main title</dc:title>
<meta refines="title" property="title-type">main</meta>

epubveri:

WARNING RSC-017: @refines should use a fragment identifier pointing to its manifest item

EPUBCheck:

ERROR(RSC-005): Error while parsing file: Property "title-type" must refine a "title" property.

[Kayadelenium]

epubveri 0.9.10

Doitsu — that one was worse than it looked, and it turned out to be two mistakes rather than one. Thank you.

• The missing # is an error, not a warning. EPUBCheck's Schematron compares @refines against concat('#', @id), so a bare id fails its "must refine a title property" assertion. I had no such check at all, which meant the verdict differed and not just the wording — your file was INVALID to EPUBCheck and VALID to me. Seven of those assertions are now in: title-type, authority, term, identifier-type, role, collection-type, source-of.
• And the warning I did give was too broad. It fired on any @refines without a #. EPUBCheck only reports that when the value resolves to a real manifest item's href — the case where someone names a file instead of an id. So a plain missing # drew a warning EPUBCheck never gives. It is now narrowed to that case, and the message names the item id it should have pointed at, as EPUBCheck's does.

Four shapes checked against EPUBCheck one at a time — bare id with a rule, the correct #id form, a bare id on a property with no such rule, and a real manifest href — all four now identical.

Worth admitting how the second one was found: my first probe used the wrong shape, so I briefly concluded EPUBCheck had no such check and was about to delete my warning outright. What corrected me was EPUBCheck's own test fixture, which is named metadata-refines-not-a-fragment-warning.opf and shows exactly which shape it is for.

Downloads (8 platforms): https://github.com/veripublica/epubveri/releases
Try it in the browser: https://veripublica.github.io/epubveri/

Two reports, two releases, five real bugs between them. Please keep them coming — the pattern that keeps working is a short description plus both tools' output for the same file.

[JSWolf]

I've more errors even if it's just a mistake in the column number.