epubveri – standalone EPUB validator

[DNSB]

Originally Posted by JSWolf

When checking the eBooks in calibre (not the editor), it would be good to know if any of the eBooks have HTML files that are too large. calibre's check book does not work outside the editor. The thing is, do you really want to make us have to run Quality Check before or after running epubvari? That's adding more work to check eBooks. It's best to have everything in one app to run once.

Jon, I was under the impression that epubveri was intended as a replacement for epubcheck without the need for the rather hefty Java overhead and was never intended to be a one step check everything in an ePub tool. Scope creep is a pestiferation.

I do not mind using multiple tools to check my ePubs—perhaps my age is showing there since I like small tools that do one job and do it well. Quality Check is just one of the tools I use on my Intake library and I neither expect nor want epubveri to replace all of them.

[JSWolf]

I found an issue with both epubckeck and epubvari. This happens with both ePub2 and ePub3.

The content in the meta name="cover" is supposed to be the id of the cover image. But it it isn't and both epubckeck and epubvari don't report this error.

<meta name="cover" content="fig_backad"/>
<item href="images/cover.jpg" id="cover_image" media-type="image/jpeg"/>

[DNSB]

Not even the obsolete meta element for ePub3?

[Jellby]

I believe the ePub2 spec says nothing about covers. The meta name="cover" tag is just an arbitrary tag that can contain whatever, it's up to reading systems to decide what to do with it. The de facto (calibre) standard may be having the file id, but nothing in the spec mandates it.

[Kayadelenium]

@JSWolf, @Jellby, @DNSB: Jellby is right. Neither EPUB 2 nor EPUB 3 defines <meta name="cover">; it's a convention that calibre and Kindle use. epubcheck has no rule for it, which I checked in its source, so it isn't an error, and neither tool should report it as one. DNSB, in EPUB 3 epubveri reports the element itself as outdated (OBS-001, as a usage), but that says nothing about what it points to.

Still, a cover meta that names nothing is a real defect for the readers that rely on it. So it's a candidate for an advisory, which is off by default and never changes the verdict. On my shelf, 436 of 474 books carry this meta, and every one points at an existing image. So it would only fire on books that are actually broken, like the one you found. I'll look at it before promising anything, mainly to see whether some tools write a file path there instead of an id.

[JSWolf]

The file path is not part of the meta name="cover". content points to the ID of the cover.

If there is a file path, then it's an error.

Here is a link from The Daisy Consortium showing this is how you define a cover for ePub2.

https://kb.daisy.org/publishing/docs/epub/cover.html

[DNSB]

Originally Posted by JSWolf

The file path is not part of the meta name="cover". content points to the ID of the cover.

If there is a file path, then it's an error.

Here is a link from The Daisy Consortium showing this is how you define a cover for ePub2.

https://kb.daisy.org/publishing/docs/epub/cover.html

And that last part of that description which reads:

In this case, the content attribute references the ID of the manifest item for the image. (Note that this practice is not officially part of the EPUB 2 specification, but is recognized by EPUB 2 reading systems.)

[JSWolf]

But without <meta name="cover"... some programs won't know the cover. So it is rather important to have this be correct.

[Kayadelenium]

Thanks for the Daisy link, that's the useful part: it shows where the convention comes from.

I went to the specs to see what they say, EPUB 2 included, since that's where this convention lives.

• OPF 2.0.1 defines meta only as a free name/content pair ("arbitrary metadata beyond the data described by the Dublin Core specification"). Its schema puts no restriction on the name values and no link between content and a manifest id. The OPS and OCF documents don't mention it either. The only cover mechanism EPUB 2 actually defines is a guide reference with type="cover".
• EPUB 3.4 lists the OPF 2 meta element among its "outdated features" (E.1): reading systems ignore it, and the spec advises checkers not to alert on its presence. The EPUB 3 way to mark a cover is the cover-image property on the manifest item (D.6.1).

So you're right that a content value naming no manifest item is a broken reference by the convention calibre, Kindle and Adobe use, and DNSB and Jellby are right that no spec, EPUB 2 or 3, defines that convention, so it can't be an error or a warning. What that leaves is an optional note. I'd word it as "the cover meta names no manifest item", and I wouldn't claim what a given reader will do with it.

Before I decide anything, I'd like some real cases from you. Is it always an id that doesn't exist (like your fig_backad), or have you seen a file name or a path in content? And roughly how many books in your library does it hit? On my own shelf of 524 books, 490 carry the meta and all 490 name an existing manifest item, so a check would fire on none of them, which tells me nothing about libraries like yours. I'm not promising it yet, but a few real examples would settle it.

[DNSB]

I did a search on 17758 ePubs for '<meta name="cover"' in the .opf file. The search returned 17192 results.


The results were:

14,270 had '<meta name="cover" content="cover"/>'
741 had '<meta name="cover" content="cover" />'
539 had '<meta name="cover" content="cover-image"/>'
59 had '<meta name="cover" content="my-cover-image"/>'
52 had '<meta name="cover" content="cover_img" />'
48 had '<meta name="cover" content="my-cover-image" />'
43 had '<meta name="cover" content="cover_img"/>'
35 had ''<meta name="cover" content="x_cover-image" />'

For a total of 15,787 though some of those variants were from placing a space before the closing /.

The rest were a mix of filenames (cover.jpg was popular) and what looked like random IDs/ISBNs/book titles/etc. There was only 11 examples where the contents= did not match the ID in the file. As far as I could tell, those were from I had replaced a cover image and the ID was not updated.

I randomly checked the remaining block of ePubs where the '<meta name="cover"' search string was not found and they were a mix of old ePub2 files and ePub3 files.

The joys of sucking a text file into a spreadsheet and then sorting wildly.

[Kayadelenium]

epubveri 0.19.0 is out: GitHub release, crates.io and npm. The Sigil and calibre plugins pick it up on their own.

This release is about references: links, table-of-contents entries, media overlay text links and the like, and how each of them is counted. A user on Reddit ran epubcheck and epubveri side by side on 100 of their own books and shared the results. Of the 99 books epubcheck could finish, the two tools agreed on 98. The one disagreement was an extra error from epubveri, and tracing it led me through every place epubveri resolves a reference. I compared each one with epubcheck on a small test book of its own.

What this means for your books:

• Most books get exactly the same report as before. On my own shelf of 544 books, two reports changed, and both now match epubcheck.
• A link to a file that the manifest declares but the EPUB does not contain now gets only RSC-001, as in epubcheck. epubveri used to add an RSC-007 for the same missing file. If the link has a #fragment, it now also gets RSC-012, which epubcheck reports as well.
• RSC-010 and RSC-011 are now reported for every link, not once per target. A book with 35 links to images got 29 findings from epubveri and 35 from epubcheck; now both give 35. An image map's area element now counts as a link too.
• The reading order checks now work the way epubcheck's do. That covers MED-015 for media overlays and NAV-011 for the table of contents, including fragments such as epubcfi(...) and empty ones.
• If two manifest items share the same id, the first one no longer counts as declared, and its file now gets OPF-003, as in epubcheck.

That means a book may now get errors it did not get before. They are errors epubcheck already reports, which is why this is 0.19 and not 0.18.1.

Two wrong findings are gone. The RSC-007 described above, which the Reddit comparison found. And a media overlay that reached a chapter only through epub:textref made epubveri report MED-013 on that chapter, which epubcheck does not do.

It is also faster. Following the profile raeq posted on GitHub (#137), I profiled the two slowest books on my shelf. Validation of those 544 books now takes 9% less CPU time than without these changes. The speed changes did not change a single report, on any of those books or on W3C's 209 conformance publications. How much you gain depends on your books.

One more thing came out of that check. A few findings about remote resources and standalone SVG files were printed in a different order on each run, so two runs of the same book could look different when nothing had changed. They are now always in the same order.

epubveri and epubcheck still give the same message IDs on all 687 expected messages in epubcheck's own test suite.

DNSB, thank you for the cover numbers (#370). That answers the question I put to JSWolf in #369 better than my own shelf could: in 17,192 books carrying the cover meta, 11 had a content value naming no manifest item, and those came from covers replaced without updating the id. So a note for that case would be true every time it fired and would fire rarely, which is what an optional note needs. I haven't decided on it yet.