navpoint ids and epubcheck

[AlanHK]

Epubcheck complains if the navpoint ids in the toc.ncx begin with a digit.

e.g:

<navPoint class="chapter" id="23b2a4fa-6968-47cd-a55b-68ec2650ccde" playOrder="3">
      <navLabel>
        <text>A Reminiscence of Dr Samuel Johnson</text>
      </navLabel>
      <content src="Text/story05.html"/>
</navPoint>

I've looked and can't find any other reference to this id in the ncx, opf or any xhtml file, so I assume they are arbitrary and only need to be unique.

So I just prepended an X to all the ids and it passed epubcheck and seemed OK.

 id="X23b2a4fa-6968-47cd-a55b-68ec2650ccde"

What are these ids used for anyway?

[DiapDealer]

The id attribute is required for the navPoint element. And yes, it must he unique. But it also has to conform to xml standards which require that ids do not start with a digit. The one you're dealing with was auto-generated (a urn guaranteed to be unique) by a program when building the toc. Unfortunately, those generated, unique urns sometimes start with number. I honestly don't know that they are "used" per se.

Sigil used to be susceptible to generating invalid ids like that, but that bug was fixed a while back. At least I think it was. At the time, the bugfix was about opf manifest ids rather than toc ids, but I assume the fix was applied to a general id-generating utility function.

[KevinH]

I think the routine we touched only dealt with manifest ids. We should at least look how the toc ids and nav ids are generated to verify they have the same fix.

[KevinH]

Okay I checked src/Exporters/NCXWriter.cpp

and we use:

m_Writer->writeAttribute("id", QString("navPoint-%1").arg(play_order));

So guaranteed to not begin with a digit.

We do similar things in python3lib/ncxgenerator.py so our auto conversion routines from nav to ncx should be fine there.

I also checked the nav for epub3: src/ResourceObjects/NavProcessor.cpp
and it does not use "id" in a similar manner so we are okay there as well.

I did see lots of calls to Gumbo's get_local_text_of_node() and there are no calls to simplified so we should probably add some simplify calls to the NavProcessor.cpp every time we use get_local_text_of_node just to be safe. That said a newline in an html document will just become a space and multiple spaces are condensed in html, so for the nav this may not matter too much.

[DiapDealer]

I think the .simplified() additions I just pushed the other day (in Headings.cpp) are pretty central to all toc/nav production, but more wouldn't really hurt anything either.

Thanks for looking at the id generation. I didn't think uids were being introduced in the ncx that way, but I couldn't remember for sure.

[Doitsu]

Since you're looking at the NCX code, according to the epub 2.0.1. specs:

The NCX file must be a valid XML document conforming to ncx-2005-1.dtd, with the exception of the playOrder attribute, which is optional in EPUB NCX

and

Any NCX that contains a DOCTYPE that references the canonical NCX DTD must honor that DTD, thus including the playOrder attribute in all locations where it is required. NCX documents that do not contain a DOCTYPE may omit the playOrder attribute.

[Emphasis mine]

I.e., if you didn't include the ncx-2005-1.dtd DOCTYPE declaration, you could also remove the playOrder attributes, which should theoretically simplify NCX updates.

[KevinH]

@Doitsu
We actually use the playorder to generate the ids from. To change that now would be more work for no increase in validity.

[AlanHK]

I know that Sigil doesn't make such ids in an epub it creates, but this was one I cannibalised; no idea what made it to begin with.

Anyway, wanted to confirm that nothing actually refers to this id, so I won't break anything if I change them as I did to shut epubcheck up.

[DiapDealer]

Originally Posted by AlanHK

Anyway, wanted to confirm that nothing actually refers to this id, so I won't break anything if I change them as I did to shut epubcheck up.

No. As long as they're unique (and valid), no harm done. They can be anything. I've never even run into an issue with one that began with a digit on any reader (in any app), to tell the honest truth. But it's best to please Epubcheck when you can. I guess.

[AlanHK]

Originally Posted by DiapDealer

But it's best to please Epubcheck when you can. I guess.

When it's giving me dozens of errors like this, I might miss the "real" errors that do have impact. (I wish there was a way to tell it to ignore specific errors.)

And while this one is just a personal project, if you are doing it for a client, passing Epubcheck may be a requirement.

[slowsmile]

I've run into this issue before with the unique book id starting with a digit in the opf manifest that Kevin or DiapDealer fixed some while ago. I was the one who reported this problem way back when. What disappoints me is that when I just tested this current problem, Epupcheck still gives this weird error message for wrong ids in the toc.ncx:

Col: 72: ERROR(RSC-005): Error while parsing file 'value of attribute "id" is invalid; must be an XML name without colons'.

This error message would be highly confusing for any indie guy trying to get his book to pass Epubcheck. That error message tells you nothing whatsover about the problem and is highly misleading.

Just a suggestion, but would it be possible to implement a simple change in Sigil where, on opening any epub in Sigil, an automatic check of all the unique ids in the toc.ncx(with a helpful error message for the user if an id incorrectly starts with a digit)? Or another alternative would be to just update the Mend XHTML Source... feature to actually repair this problem automatically if it is found. I also realize that this problem has nothing to do with Sigil -- I'm only suggesting a solution that would perhaps help other Sigil users with this problem in the future.

[DiapDealer]

No, we're not going to repair invalid ids automatically, and Sigil is no longer in the epub validation game itself. Validation is done by plugins. If you consider Epubcheck's error message to be unhelpful, you should consider reporting this to the Epubcheck developers.

Sigil's validation plugin framework offers plugin developers a way to create their own error-checking routines that go above and beyond what EpubCheck looks for (or how it describes a problem). Perhaps you can convince one to compile a list of extra checks/warnings into plugin that might be more helpful to neophytes.

[Doitsu]

Originally Posted by slowsmile

Just a suggestion, but would it be possible to implement a simple change in Sigil where, on opening any epub in Sigil, an automatic check of all the unique ids in the toc.ncx(with a helpful error message for the user if an id incorrectly starts with a digit)?

BTW, Calibre Editor has a built-in repair feature that'll automatically fix, among other things, ncx ids that start with a numeral.
Since you can use both Calibre Editor and Sigil, there's no point in adding this feature to Sigil.

[slowsmile]

I've just reported the Epubcheck uuid error message problem on the IDPF Epubcheck issues list on Git Hub(Issue #795).

@Doitsu...Thanks for your info, which was quite useful to know.

[AlanHK]

Originally Posted by slowsmile

This error message would be highly confusing for any indie guy trying to get his book to pass Epubcheck. That error message tells you nothing whatsover about the problem and is highly misleading.

The only way I worked out what the real problem was was by Googling the error message and finding comments that told you.

Also found this discussion on GitHub

https://github.com/IDPF/epubcheck/is...ment-151267599

rdeltour commented on 27 Oct 2015

Right, "XML name without colons" comes from the value of the id attribute being required to match the NCName production as required in the Namespaces in XML 1.0 spec.

This error message comes from Jing (the 3d party RelaxNG + Schematron validation engine used by EpubCheck) and as far as I remember we cannot easily override it to make it more user friendly.

See also #193, #224, #307, #533.

I'm closing as wontfix, since we do not have control over the message.

So we will continue to get this confusing and misleading error message indefinitely. Despite it not actually causing any problems except with epubcheck.

Originally Posted by slowsmile

I've just reported the Epubcheck uuid error message problem on the IDPF Epubcheck issues list on Git Hub(Issue #795).

As above, this has been reported there at least 5 times already, but it's a "wontfix".