toc.ncx error with playOrder causes validation failure

[KevinH]

Hi,
You keep calling it a "workaround" but I stll think my "For Example" is in fact the correct way to do what you are trying to do. The only rule on how playOrders should be allocated for ePub 2 is for Talking Books and not normal ebooks. The rule on talking books is that if a link points to the same SMIL time reference, then it needs to have the same playOrder.
But in the general, non-talking book case, this rule does not apply. And adding "id" attributes to tags is not clunky. It is simply correct practice to refer to a specific point in the file. This is how real page numbers are handled, etc.

As for where bugs are reported, you can do that here or of the github site.

Again, you are trying to do something different without adding the proper id tags and then claiming it is a "bug", when I can find no explicit spec that agrees with what you claim for epub 2.

That is why we are having this discussion.

Kevin

[Peter Ahlstrom]

Then the question remains, why does the validator check for that specific error, but not the easier, more general case that I used to fix the error?

[Toxaris]

Because programmers usually think like programmers and not like users... If you want to test a program for bugs or missing features, give it to users to test. They will manage to break and find things because they do things programmers would not think about. Usually because they make no sense (at least from their perspective...).
The validator is not the final word of course. It is just checks their interpretation of the specifications. It is actually quite easy to create a books that fails on (almost) all readers/programs but validates fine.

[Peter Ahlstrom]

In this case, it also works fine in the readers that I've tested. So if it's not explicitly disallowed in the spec, I'm not sure why (even if it's not explicitly mentioned as allowed) it should be considered invalid. It still seems to me like of course it's something that should be allowed.

The programmers making this test in the validator would have to 1. Check to see if any navpoints point to the same target. 2. Check if the navpoints have the same playOrder. That is the only way this specific validation test could have been programmed, that I can see. There is no way they could program that in without anticipating the possibility of a test where only #1 is the case. So, reading the spec, the programmers must have assumed (as I have assumed) that #1 by itself is not an error at all, but works as intended.

So I just looked at the source code for the IDPF validation and it has this:

    <sch:pattern name="ncx_playOrderMatch2" id="ncx_playOrderMatch2">
      <!-- navPoints, navTargets, and pageTargets that have identical playOrder values must point to same target -->
      <sch:rule context="//*[@playOrder]">
        <sch:assert test="count(//*[@playOrder][number(@playOrder)=number(current()/@playOrder)]/ncx:content[@src != current()/ncx:content/@src])=0">
          identical playOrder values for navPoint/navTarget/pageTarget that do not refer to same target
        </sch:assert>        
      </sch:rule>
    </sch:pattern>

    <sch:pattern name="ncx_playOrderMatch" id="ncx_playOrderMatch">
      <!-- navPoints, navTargets, and pageTargets that point to same target must have identical playOrder values -->
      <sch:rule context="//*[@playOrder][ncx:content]">
        <sch:assert test="count(//*[@playOrder][ncx:content][ncx:content/@src=current()/ncx:content/@src][number(@playOrder)!=number(current()/@playOrder)])=0">
          different playOrder values for navPoint/navTarget/pageTarget that refer to same target
        </sch:assert>        
      </sch:rule>
    </sch:pattern>

It has both the positive test and the negative test. It checks to see if different navpoints pointing to the same target have the same playOrder, AND it checks to make sure that items with the same playOrder point to the same target. There is absolutely no way they would implement the second type of test if they didn't think it was valid for "same playOrder" cases to exist.

Now, that's their interpretation of the spec, but if anything, this proves I'm no more crazy in my interpretation than the validatior programmers.

[KevinH]

Hi Peter,

If you can point me to any spec that says that your way is the right way to do things (outside of the Talking Book spec which uses SMIL time references) I would be happy to take a stab and trying to address this. But without a clear spec one way or the other, I really think you should just properly use "id" attributes to do what you want and still make it pass whatever check there is.

As for rules in epubcheck*, they are many many times very arbitrary. For example in early versions of epubcheck 3, they decided that in epub3 there can be a DOCTYPE on top of the toc.ncx (if present for fallback) - which according to the spec, the toc.ncx should have been totally ignored - so that made sense. But in more current versions they have decided to mark that as an "error" for some reason claiming that nothing can have a DOCTYPE that references external namespaces and dtds parsing information. Then someone nicely pointed out to them that some valid toc.ncx files are unparse-able without the proper DOCTYPE with dtds, so how could the toc.ncx be used as a fallback for epub2 without a DOCTYPE? So now in epubcheck4 pre-releases that "error" is no longer an "error".

The spec for epub3 also clearly states that the old form of the metadata in the opf is allowed to be there (and must be there to specify the cover page properly in most readers) but must be ignored, but then they actually report back errors on what was supposed to be "ignored". If it was valid metadata in epub2 format it should be properly ignored in epub3 if it doesn't fit, and they ignore that as well.

You'll find this same pattern again and again. Simply read the epubcheck bug database to see how often this happens and how arbitrary some of these rules really are. So claiming that the epubcheck validator claims it is a bug, can in fact be totally meaningless.

That is why I like to read the spec itself. So if you can find a spec online that clearly states how we do things is wrong, I would be happy to look at it. But if not, I think you should simply add the proper id attributes and be done with it.

KevinH

[Peter Ahlstrom]

Since the spec relies on the Talking Book spec, I don't see how we can ignore it.
Section 2.4.2 of the epub 2.0.1 OPF spec spells out all the exceptions to the Talking Book spec definition of the NCX. And there's nothing in that list of exceptions that says two navPoints cannot point to the same target in an epub's NCX.

2.4.2 says that the NCX spec's references to links to SMIL locations should be read as referring to links to XML elements instead.

[KevinH]

Hi,

I just read that section again, and I disagree with your interpretation.

The spec says under exceptions:

"content Description: Pointer into SMIL file to beginning of the item referenced by the navPoint or navTarget. Note: In the OPF application of the NCX the pointer is to an XML element and not a SMIL location."

This says nothing about SMIL time references. Just that the links in Talking Books are links to a SMIL doc where as links in ebooks are to separate xml elements in xhtml files.

More clearly the spec states:

"playOrder attributes are not required. Although playOrder attributes are not used to sequence SMIL content, if present, they should contain valid values that reflect the linear document reading order."

As far as I can tell, your use case would in fact directly violate the "linear document reading order" You are trying to force a later link to effectively go backwards and show something that an earlier link has already referenced. So any link that says "Section 1" should *not* take you to something earlier in the document already referenced (for example your pre-amble) as that would violate the "linear document reading order".

Again I feel you are trying to simply "trick" the ncx to always show something that has already been shown (ie. the preamble part) and that doing so without the proper id tags in the right order violates the "linear document reading order" rule completely. And according to the spec, the playOrder's are actually meaningless in the ebook context as long as they follow the "linear document reading order".

So by my reading, the only way to legally do this is to have separate id points in proper linear order:

1. top of file
2. body tag with id
3. id in first paragraph tag of the preamble

You can call them anything you like but they should appear in that order to keep the linear reading order since the top of file comes first, the body comes second, and the first paragraph of the preamble would come third. To the reader, all 3 locations will show the same text on the screen but are clearly separate locations in the file in the proper linear reading order

KevinH

Originally Posted by Peter Ahlstrom

Since the spec relies on the Talking Book spec, I don't see how we can ignore it.
Section 2.4.2 of the epub 2.0.1 OPF spec spells out all the exceptions to the Talking Book spec definition of the NCX. And there's nothing in that list of exceptions that says two navPoints cannot point to the same target in an epub's NCX.

2.4.2 says that the NCX spec's references to links to SMIL locations should be read as referring to links to XML elements instead.

[Peter Ahlstrom]

But I'm never presenting anything out of order. Having two or three references in a row to the exact same location is not violating linear document order. I'm not listing A B C and then saying "for D and E, skip back over C and look at B again"—that would be a violation of linear document reading order.

Regardless, I'm giving up this fight. Since the playOrder is explicitly optional, and since I'm not happy with the other suggestions here, I'll just delete all the playOrder attributes next time. That's easier to do with a regex search than renumbering things manually.

I officially withdraw my bug report. If this issue ever comes up for me again in the future, I'll just deal with it myself.

[KevinH]

Hi Peter,

When you call it "Section 1" and Section 1 clearly and logically starts *after* the preamble, and the preamble has its own link and entry in the ncx, how can linking it backwards to show the preamble (possibly twice or even 3 times if people are electronically navigating by ncx), ever be a "linear reading order". As a reader I would be shown the preamble multiple times instead of the selected "Section 1" which might not even be on the first screen depending on the length of the preamble!! I personally would consider that a bug in how the ebook was created.

I am happy you found a solution that works for you but I am astonished by your unwillingness to even consider perfectly normal id attributes on tags.

If you ever find a clear spec handling this case in epub 2 (or even epub 3 as Sigil is moving toward epub 3), I would be happy to look at this again. Otherwise there are multiple equally valid solutions for you to choose from.

Good luck!

KevinH

Originally Posted by Peter Ahlstrom

But I'm never presenting anything out of order. Having two or three references in a row to the exact same location is not violating linear document order. I'm not listing A B C and then saying "for D and E, skip back over C and look at B again"—that would be a violation of linear document reading order.

Regardless, I'm giving up this fight. Since the playOrder is explicitly optional, and since I'm not happy with the other suggestions here, I'll just delete all the playOrder attributes next time. That's easier to do with a regex search than renumbering things manually.

I officially withdraw my bug report. If this issue ever comes up for me again in the future, I'll just deal with it myself.