Veripublica epub-test-books: small EPUBs for testing EPUB tools

[Kayadelenium]

I've started a library of small EPUB files for testing EPUB tools: validators, converters, fixers and reading apps. Each book shows one thing, either a rule from the EPUB or CSS specifications or a defect I have met in real books, and nothing else. When a tool gets a book wrong, you can see exactly what it got wrong.

I made it because I was testing my own tools against shelves of commercial books. They can't be shared, they take a long time to scan, and the one thing you are looking for hides among a thousand others. W3C's EPUB tests already cover reading systems well. These books are mostly for the tools that check or change books, and many of them are broken on purpose.

The rule I care about most is where the answer comes from. For every book, I write the expected result from the specification, quoting the sentence, before any tool runs. Then I run epubcheck and epubveri, my own validator, on it and keep their output next to the book. When a tool disagrees with the specification, the book says so, and that includes epubveri.

Every book starts as a public issue on GitHub with the question, the clause and the expected result. The discussion happens there, and the issue closes when the book is in. If you think a result is wrong, say so under its issue, or here, and I will carry it over. When I change a result, the release notes say what changed and why, and the old outputs stay as they were, so anyone can see where I was wrong and how it was fixed.

Three releases are out. The latest, v0.2.0, has the two templates, one for EPUB 3 and one for EPUB 2, and 22 sets on the EPUB container: damaged ZIP files, the mimetype file, file names, container.xml and its rootfile. Each release is a tag you can pin. The books are ready-made .epub files, listed on one index page with what each should give and what each validator says, and you can download them one by one or all together in a single zip. They are CC BY 4.0, so you can copy them into your own test suite with one line of credit.

The container section is not finished, and that is partly why I'm posting now. I have left out every case where the specification and the validators disagree, and listed them in one issue: open questions. Most of them come down to one question: should a book say invalid when the specification says so and no validator reports it? I would rather settle that with people who read the specification closely than on my own.

If you have a book that broke your tool, a rule you think nobody checks, or a reading of the specification you disagree with, post it here or open an issue. You don't need to build anything. I'll make the book.

https://github.com/veripublica/epub-test-books