MobileRead Forums - View Single Post

Kayadelenium · Yesterday, 01:05 PM

epubveri 0.9.2 — six ways a book could crash the validator, and large books no longer crawl

Since 0.8.6 there have been three releases. Two of them matter to you; the third is internal only, and I'd rather say so than pad the list.

0.9.0 — a validator that could be killed by the file it was validating

This one is a confession rather than a feature. Six ways an ordinary .epub could kill the epubveri process, or make it lie:

Deeply nested XML aborted it — from a 1.1 KB file. The XML tokenizer recurses per level, so depth costs stack. On a 2 MiB worker thread (which is what an embedded/browser build actually gets) about 4,000 levels was enough.
A compressed entry could be inflated without bound — a 400 KB .epub drove 1.3 GB of memory, and then reported the book VALID. That last part is the worse half: no crash, no warning, just a wrong answer.
Deeply nested CSS aborted it in four different shapes, each from a ~1.2 KB stylesheet. (That is the six: one XML shape, one zip shape, four CSS s

In Rust a stack overflow is a hard abort, not something a calling program can catch — so if you had embedded epubveri, nothing you wrote could have defended against it. It had to be fixed here.

Will this reject your books? It shouldn't. Both limits were sized from real books rather than picked by taste: the depth limit is 256 against a deepest-real-book of 24 (median 8) across 65 real books, and the entry limit is 64 MiB against the largest single entry in those same books, 2.1 MB. If a resource ever is refused you get told — a new LIM-001 message says a resource went unchecked. It is never a silent skip, because a book with real errors reporting "clean" is the one failure you cannot notice.

Worth being straight about how these were found: not by the test corpus, and not by my shelf of real books. Both stayed completely green through all six. Both of them only ever answer "what verdict do we give a well-formed book", and none of these inputs is one. I had to sit down and write hostile files by hand.

0.9.1 — validation is now linear in the size of the manifest

The worst real book on my shelf (1,951 manifest items): [B]9.4 s -> 2.1 s[ (4.5x)
A synthetic 4,000-item package document: 42.6 s -> 0.52 s (82x — bu that is a stress test, not a book you own; the real-world figure is the one above)
Scaling is now linear — 1,000 / 2,000 / 4,000 / 8,000 items cost 0.13 / 0.26 / 0.52 / 1.02 s, i.e. double the book, double the time.

Three quadratic loops, of which one (an SVG-reference lookup that rescanned the whole manifest for every image reference, and re-normalized every path each time) was essentially all of it. If you have a large fixed-layout book or a comics/manga title with thousands of images, this is the release you want.

Zero behaviour change in that work — every finding is identical to 0.9. which is exactly what made it safe to do: the corpus output stayed byte-for-byte identical and every book on the shelf reported the same thing.

0.9.2 — internal only

Nothing user-facing. Tooling moved off Python and onto Rust, and the release pipeline lost its last scripting dependency. Listed only for completeness.

What has NOT changed

No new checks in this range, so nothing about what your books report has changed. Coverage is still 189 of 196 live epubcheck checks (~96%), and th per-message-ID matrix with every gap listed is in the repo (docs/COVERAGE.md).

Getting it

Pre-built binaries for 8 platforms (Linux x86_64/aarch64 in gnu and static musl, macOS Intel and Apple Silicon, Windows x86_64/aarch64): https://github.com/veripublica/epubveri/releases/latest

Or try it without installing anything — drag an .epub onto the browser demo, the file never leaves your machine:
https://veripublica.github.io/epubveri/

Install notes, first-run security prompts and a walkthrough of the output: https://github.com/veripublica/epubv.../docs/USAGE.md

Thanks, and a request

The bugs above I found myself, so I can't credit the thread for them. But that is not where most fixes come from: the false-positive reports in this thread remain by far the best source of real defects — the corpus and my own tests we green through every single one of them, because neither can see epubveri complaining about markup that is actually fine.

So if epubveri reports something on a book that epubcheck accepts, please do p it. And the single most useful thing you can attach is both outputs as plain text — epubcheck's and epubveri's for the same file. One such pair recentl resolved into three separate causes plus a false positive I would have had no other way of finding.

Yesterday, 01:05 PM	#145
Kayadelenium Enthusiast Posts: 42 Karma: 100000 Join Date: Jul 2026 Location: Planet Earth Device: Kobo Forma	epubveri 0.9.2 — six ways a book could crash the validator, and large books no longer crawl Since 0.8.6 there have been three releases. Two of them matter to you; the third is internal only, and I'd rather say so than pad the list. 0.9.0 — a validator that could be killed by the file it was validating This one is a confession rather than a feature. Six ways an ordinary .epub could kill the epubveri process, or make it lie: Deeply nested XML aborted it — from a 1.1 KB file. The XML tokenizer recurses per level, so depth costs stack. On a 2 MiB worker thread (which is what an embedded/browser build actually gets) about 4,000 levels was enough. A compressed entry could be inflated without bound — a 400 KB .epub drove 1.3 GB of memory, and then reported the book VALID. That last part is the worse half: no crash, no warning, just a wrong answer. Deeply nested CSS aborted it in four different shapes, each from a ~1.2 KB stylesheet. (That is the six: one XML shape, one zip shape, four CSS s In Rust a stack overflow is a hard abort, not something a calling program can catch — so if you had embedded epubveri, nothing you wrote could have defended against it. It had to be fixed here. Will this reject your books? It shouldn't. Both limits were sized from real books rather than picked by taste: the depth limit is 256 against a deepest-real-book of 24 (median 8) across 65 real books, and the entry limit is 64 MiB against the largest single entry in those same books, 2.1 MB. If a resource ever is refused you get told — a new LIM-001 message says a resource went unchecked. It is never a silent skip, because a book with real errors reporting "clean" is the one failure you cannot notice. Worth being straight about how these were found: not by the test corpus, and not by my shelf of real books. Both stayed completely green through all six. Both of them only ever answer "what verdict do we give a well-formed book", and none of these inputs is one. I had to sit down and write hostile files by hand. 0.9.1 — validation is now linear in the size of the manifest The worst real book on my shelf (1,951 manifest items): [B]9.4 s -> 2.1 s[ (4.5x) A synthetic 4,000-item package document: 42.6 s -> 0.52 s (82x — bu that is a stress test, not a book you own; the real-world figure is the one above) Scaling is now linear — 1,000 / 2,000 / 4,000 / 8,000 items cost 0.13 / 0.26 / 0.52 / 1.02 s, i.e. double the book, double the time. Three quadratic loops, of which one (an SVG-reference lookup that rescanned the whole manifest for every image reference, and re-normalized every path each time) was essentially all of it. If you have a large fixed-layout book or a comics/manga title with thousands of images, this is the release you want. Zero behaviour change in that work — every finding is identical to 0.9. which is exactly what made it safe to do: the corpus output stayed byte-for-byte identical and every book on the shelf reported the same thing. 0.9.2 — internal only Nothing user-facing. Tooling moved off Python and onto Rust, and the release pipeline lost its last scripting dependency. Listed only for completeness. What has NOT changed No new checks in this range, so nothing about what your books report has changed. Coverage is still 189 of 196 live epubcheck checks (~96%), and th per-message-ID matrix with every gap listed is in the repo (docs/COVERAGE.md). Getting it Pre-built binaries for 8 platforms (Linux x86_64/aarch64 in gnu and static musl, macOS Intel and Apple Silicon, Windows x86_64/aarch64): https://github.com/veripublica/epubveri/releases/latest Or try it without installing anything — drag an .epub onto the browser demo, the file never leaves your machine: https://veripublica.github.io/epubveri/ Install notes, first-run security prompts and a walkthrough of the output: https://github.com/veripublica/epubv.../docs/USAGE.md Thanks, and a request The bugs above I found myself, so I can't credit the thread for them. But that is not where most fixes come from: the false-positive reports in this thread remain by far the best source of real defects — the corpus and my own tests we green through every single one of them, because neither can see epubveri complaining about markup that is actually fine. So if epubveri reports something on a book that epubcheck accepts, please do p it. And the single most useful thing you can attach is both outputs as plain text — epubcheck's and epubveri's for the same file. One such pair recentl resolved into three separate causes plus a false positive I would have had no other way of finding.