toc.ncx error with playOrder causes validation failure

[Peter Ahlstrom]

Hey there, here's an error I haven't seen before.

I used Sigil to make the Table of Contents for a book, and the resulting file passed Sigil's validator but failed when uploading to both Apple and Kobo. The IDPF validator throws errors like this:

ERROR OEBPS/toc.ncx 152 49 assertion failed: different playOrder values for navPoint/navTarget/pageTarget that refer to same target

In the toc.ncx, the offending section looks like this:

      <navPoint id="navPoint-24" playOrder="24">
        <navLabel>
          <text>Sections from the first draft of Pandemonium</text>
        </navLabel>
        <content src="Text/AP_lauren_oliver_story.xhtml"/>
        <navPoint id="navPoint-25" playOrder="25">
          <navLabel>
            <text>1.</text>
          </navLabel>
          <content src="Text/AP_lauren_oliver_story.xhtml"/>
        </navPoint>

Manually changing the second playOrder="25" to playOrder="24" and manually renumbering all the playOrders after that causes this validation error to go away.

So when Sigil is writing the toc.ncx, if the content src for a navpoint is the same as the previous navpoint, it should give them the same playOrder (even though the navpoints will then have different numbers from their playOrders).

(You might say, "Well, don't have multiple TOC entries that point to the same location!" but there occasionally are good reasons for doing that, and it's structurally valid.)

[DiapDealer]

Originally Posted by Peter Ahlstrom

You might say, "Well, don't have multiple TOC entries that point to the same location!" but there occasionally are good reasons for doing that, and it's structurally valid.)

I can't think of a single good reason to have two toc entries pointing to the exact same location.

[KevinH]

Hi,
You could easily let them point to the same page but to different ids/anchor points as a workaround. For the second add the id of the section element as an href #fragment, for example. Wouldn't that do the trick and be more logical?

Kevin

[Peter Ahlstrom]

I'll give an example.

So, I have a book where each chapter has some introductory text at the top of the page, before the chapter title. The chapter title is what is listed in the Table of Contents, but if a user clicks on that entry, I want them to see the introductory text before the chapter title. So the link goes to the top of the page, not exactly to the chapter title header text.

Now, that chapter has 3 subheadings in it. If they click on the 2nd or 3rd subheading, they will go directly to that exact spot in the chapter. Great.

But the 1st subheading is directly under the chapter title. The reasons I have for wanting them to see the page's introductory text are just as valid for the first subheading as they are for the chapter title. So I want clicking on the first subheading in the TOC to also take them to the top of the page, just as clicking on the chapter title does.

So does that help?

I could work around this Sigil error by having multiple a id= tags up at the top of the page. But that would be a workaround, not a final solution.

The basic point is that Sigil's Table of Contents GUI editor in this case is taking valid input data and creating an invalid toc.ncx. So it's a bug in Sigil.

[KevinH]

Hi,
Then what is the point of the playorder? If you have an id on the body tag, and list the chapter preamble separately, then the user can choose to see or not see that material.

So that the Table of Contents would show the following

Chapter 1 Preamble
   Chapter 1
      Section1
      Section 2
      ...

And then the playorder would make sense as would the link locations.

Otherwise, your link locations or playorder would be incorrect no matter how it was built. You can't have both correct at the same time without this structure, can you?

KevinH

Originally Posted by Peter Ahlstrom

I'll give an example.

So, I have a book where each chapter has some introductory text at the top of the page, before the chapter title. The chapter title is what is listed in the Table of Contents, but if a user clicks on that entry, I want them to see the introductory text before the chapter title. So the link goes to the top of the page, not exactly to the chapter title header text.

Now, that chapter has 3 subheadings in it. If they click on the 2nd or 3rd subheading, they will go directly to that exact spot in the chapter. Great.

But the 1st subheading is directly under the chapter title. The reasons I have for wanting them to see the page's introductory text are just as valid for the first subheading as they are for the chapter title. So I want clicking on the first subheading in the TOC to also take them to the top of the page, just as clicking on the chapter title does.

So does that help?

I could work around this Sigil error by having multiple a id= tags up at the top of the page. But that would be a workaround, not a final solution.

The basic point is that Sigil's Table of Contents GUI editor in this case is taking valid input data and creating an invalid toc.ncx. So it's a bug in Sigil.

[Hitch]

Originally Posted by Peter Ahlstrom

I'll give an example.

So, I have a book where each chapter has some introductory text at the top of the page, before the chapter title. The chapter title is what is listed in the Table of Contents, but if a user clicks on that entry, I want them to see the introductory text before the chapter title. So the link goes to the top of the page, not exactly to the chapter title header text.

Now, that chapter has 3 subheadings in it. If they click on the 2nd or 3rd subheading, they will go directly to that exact spot in the chapter. Great.

But the 1st subheading is directly under the chapter title. The reasons I have for wanting them to see the page's introductory text are just as valid for the first subheading as they are for the chapter title. So I want clicking on the first subheading in the TOC to also take them to the top of the page, just as clicking on the chapter title does.

So does that help?

I could work around this Sigil error by having multiple a id= tags up at the top of the page. But that would be a workaround, not a final solution.

The basic point is that Sigil's Table of Contents GUI editor in this case is taking valid input data and creating an invalid toc.ncx. So it's a bug in Sigil.

No. It's not a bug in Sigil; it's invalid data. The "can't point two playorders to the same location" issue has been around for ages, IIRC. How is Sigil supposed to overcome that? You're pointing, apparently, to the same precise file, with nary a distinction--no id's, no anchors, nada.

We use id's for this type of use, OR, more customarily, we create more detailed TOC's, like Kevin's suggestion. In a print book, yes, the TOC, if you actually did it this way, would all point to the same "page," but now that's functioning like an INDEX, where the expectation is that the reader will figure out where on the page s/he is supposed to be, even though, by default, she's taken "to the top of the [print] page," when she thumbs to the relevant page, from the INDEX. OR, for that matter, from the TOC.

You're expecting the NCX to really function like a print book, by forcing the reader to start at the top of the page, no matter what they select. One option is simply to remove the extraneous elements from the NCX, and let the reader just go TO THE CHAPTER, which is, apparently, the behavior you want to force on them in the first place--the rest is window dressing, isn't it? I'm really not being argumentative, but if you don't want those readers to go elsewhere, why put the "elsewhere" in the NCX in the first place? (And, yes, I know that the answer will be, because later, that same reader might want to find X.")

But this isn't print. An NCX isn't some decorative, harmless page, as is a TOC in a print book. It's structural. You gotta give Sigil a little help here, and at the very least, give it SOMETHING to point to, like an id.

Just my $.02.

Hitch

[Peter Ahlstrom]

Originally Posted by KevinH

Hi,
You can't have both correct at the same time without this structure, can you

I'm not sure what you're saying. I can have it correct; I already made the ncx valid by manually editing it so the playOrders of those entries were the same as each other. So everything is great now. Having multiple items with the same playOrder is apparently perfectly valid, because it validates. Or is this something listed in the spec but the validator isn't checking for it?

I just think that Sigil should accommodate this valid data in the first place. Is it that much of a pain for Sigil to be coded to check whether any of the TOC targets are the same, and if so, make them have the same playOrder?

I don't want to remove extraneous elements from the ncx because then it will say
Chapter Title
2
3
and people will wonder "where's #1?"

[DiapDealer]

Originally Posted by Peter Ahlstrom

I don't want to remove extraneous elements from the ncx because then it will say
Chapter Title
2
3
and people will wonder "where's #1?"

I don't follow. The ncx will "say" whatever you tell it to say. There's no need for anyone to wonder anything. The ToC generator is a wonderful place to start, but there's no guarantee the structure you've chosen for your headers and subheaders is going to produce the ToC you envisioned. In such a case, you tweak the the ncx, or you tweak the semantics of your book's code and regenerate..

[Peter Ahlstrom]

I'm not talking about the auto-generated TOC. I'm talking about when you edit the TOC by picking Tools > Table of Contents... > Edit Table of Contents. This is a GUI frontend for the NCX, so it should produce a valid NCX if you give it valid data. Currently, in this case, I'm giving it valid data but it's producing an invalid NCX.

[DiapDealer]

Originally Posted by Peter Ahlstrom

This is a GUI frontend for the NCX, so it should produce a valid NCX if you give it valid data. Currently, in this case, I'm giving it valid data but it's producing an invalid NCX.

Whatever.
I remain unconvinced that you're giving it valid data.
I remain unconvinced that two navPoints (with or without duplicate PlayOrders) pointing to the exact same url is valid.
I remain unconvinced that duplicate PlayOrders are valid (regardless of what various validation engines might not bark about).

I believe all you're doing is fooling the IDPF validator into giving your ncx a clean bill of health by duplicating the playorders.

I don't think Sigil needs to take responsibility for helping to slip an unorthodox ncx past Apple's and Kobo's validator.

[Hitch]

Originally Posted by DiapDealer

Whatever.
I remain unconvinced that you're giving it valid data.
I remain unconvinced that two navPoints (with or without duplicate PlayOrders) pointing to the exact same url is valid.
I remain unconvinced that duplicate PlayOrders are valid (regardless of what various validation engines might not bark about).

I believe all you're doing is fooling the IDPF validator into giving your ncx a clean bill of health by duplicating the playorders.

I don't think Sigil needs to take responsibility for helping to slip an unorthodox ncx past Apple's and Kobo's validator.

Not to get in the midst of this, Peter, but: Diap's right, it's not "valid data." You're creating duplicate play-order entries, as different NCX values, but differing nav-points. The correct way to do this would be as previously discussed--with actual, different navpoints, identified via ID or # or whatever. You are "tricking" it, Peter, because you're manually manipulating it into accepting a playlist with duplicate entries, or, rather, essentially repetitive entries being recycled.

Personally--for what it's worth--I'd rather have Sigil error-checking for dupe playlist entries. That, as an error, is more likely, than the need to have multiple items on the NCX all pointing at the same place, in order to shoehorn the reader into starting at the top of the chapter. Most of us--making books--would just create the correct items. What you're doing is taking a shortcut, by not creating the id's, but then creating a boatload more work for yourself by manually renumbering the Play orders. {shrug}.

Just my $.02. The NCX spec for ePUB2 clearly doesn't anticipate duplicate playlist items/orders--that's why a duped entry creates an "error." If the ePUB3 spec for the new TOC/NCX/whatever-it's-called-I-can't-remember-now allows dupe entries, then this discussion is moot, as Sigil will be updated to reflect that, presumably.

Hitch

[Peter Ahlstrom]

The error that the validator gave me was "different playOrder values for navPoint/navTarget/pageTarget that refer to same target." The error wasn't "multiple navPoints/navTargets/pageTargets refer to same target." I corrected the error that the validator called out, and now it passes.

If it was supposed to be illegal for multiple navPoints/navTargets/pageTargets to refer to the same target, wouldn't the validator call that out as an error, instead of specifically checking for something more esoteric, making sure that multiple navPoints/navTargets/pageTargets referring to the same targets don't have different playOrders? To me, this indicates that the way I edited the toc.ncx is exactly what is intended by the IDPF.

[Toxaris]

Yeah right, 'cause the validator is always right...

[KevinH]

Hi,

The ncx is actually built on the fly recursively from the information either collected in the auto toc creation or via the editor.

So in order to make this work, we would have to modify the NCXWriter to remember all src hrefs and their associated playorder for each entry and if the exact same src href is ever used, instead of giving that entry a new playorder, I would have to give it the same playorder as was used for that src href previously. Otherwise incrementing the playorder. Since the playorder was also used to develop unique "id" attributes for each navpoint, that would have to change as well. Being a recursive routine this may end up being a bit tricky but should be doable.

But all that said ... I am still not fully convinced that it is a bug. It could all just be "user-error".

For example, to do what you want, if you simply add "id" attributes and use fragments even for the same location.

For example:

1. use file.xhtml for "preamble"
2. use file.xhtml#id1 for "chapter" where the body tag has "id1"
3. use file.xhtml#id2 for "section 1" where first p tag of the preamble has "id2"

Or almost anything along those lines as long as they are unique fragment ids.

Or alternatively, give the body tag two "id" attributes and use them for preamble, and section 1, and keep the file.xhtml without fragment for the chapter.

If you try this approach, everything should just work and the "issue" simply goes away, shouldn't it. Have you tried this?

KevinH

[Peter Ahlstrom]

Giving multiple ids to one tag is an interesting idea, but a little searching around seems to indicate that's not valid (though maybe you can call one of them "id" and a second one "xml:id" but I don't know if that would actually make them work for the TOC purposes. I have not tried that.).

And I'm sure your other solution (the "For example:" in your post above) would work, as was suggested earlier in this thread. That's not what I want to do, since I want the links to go to the same place. I could instead put some empty 'a id="foo"' tags right after each other. But again, that would be a workaround and would make more clunky code.

I'm not looking for a workaround. I already have a workaround that satisfies me, although it takes more work (manually editing the toc.ncx as I explained in my first post). I was simply reporting what I saw as a bug in Sigil. Maybe this isn't a good place to report bugs, but I find bug tracker interfaces to be unfriendly. And any programming I do is by the skin of my teeth and extremely kludgy, with complete ignorance of best practices, so I doubt I'd be doing any favors by trying to fix this myself.

Now, there's debate in this thread as to whether it's a bug or an error in the validator. Logic and Ockham's razor tell me that the validator would have been written a different way if my workaround was intended to be invalid, but people apply logic differently and what seems the most simple solution to one person is not to another. That's fine.

If this behavior of the program gets changed to allow for what I suggested above, that would be convenient for people who want to do this in the future. If it doesn't get changed, the workaround should at least still exist. I'd be frustrated if the developers decided to "have Sigil error-checking for dupe playlist entries" as was suggested above, since making my change instead would also allow it to pass validation.