Apple objects to wrong page numbers

[philja]

I've used Sigil 1.9.20 to update a couple of links in the back-matter of an epub (originally prepared in an earlier version of Sigil). I also moved a page from front-matter into the back-matter and changed the order of two files at the front. I amended the html contents and edited the ncx table of contents using Sigil Tools>Table of Contents>Edit. All checked out ok with epubcheck (v. 5).

This book had been published wide for several years. When I uploaded the new version to D2D, they got a rejection from Apple (only) with hints that there was something wrong with the table of contents.

After the second iteration of checks, rechecks and passing epubcheck with no fault found each time, the third Apple rejection contained some additional info ...

Incomplete/Inaccurate app generated Table of Contents.
The Table of Contents navigates to incorrect sections. Please make sure it navigates to the exact section indicated in the title.
Table of Contents is not correctly implemented. For example, Chapter 13 has page 159 listed in the Table of Contents but it is on page 160 in the book. Introduction is on page 13, not page 12 as listed in the Table of Contents, etc.. Please check and fix each Table of Contents entry that requires a change.

From this, it appears that Apple's app used for checking the book's epub file has generated its table of contents and does not like the result.

My file is a reflowable EPUB2 and has never had any page numbers assigned by me or Sigil. In fact the toc.ncx has the only two references to the word 'page' in the book and I would interpret that as an indication that there is no page numbering :

<meta name="dtb:totalPageCount" content="0"/>
<meta name="dtb:maxPageNumber" content="0"/>

How did Apple's app manage to produce page numbers? Any suggestions on what I can do to correct this and eliminate their objection?

[DiapDealer]

Did you try getting rid of those pointless ncx entries?

[Doitsu]

Originally Posted by philja

How did Apple's app manage to produce page numbers? Any suggestions on what I can do to correct this and eliminate their objection?

Maybe D2D has updated their intake app to add synthetic page numbers if no print page numbers are present?
You might want to ask D2D about it.

@Hitch have you encountered this problem with one of your clients?

If D2D accepts epub3 books, you could convert your epub2 book to an epub3 book with KevinH's ePub3-itizer plugin.
This might fix the issue or you might at least get more meaningful error messages.
If you're going the epub3 route, you might want to check out the Apple Books Asset Guide.

[KevinH]

Maybe they are referring to ncx playorder numbers not being sequential anymore because you edited the ncx?

[philja]

Did you try getting rid of those pointless ncx entries?

Maybe they are referring to ncx playorder numbers not being sequential anymore because you edited the ncx?

During the iterations seeking to understand the problem, D2D Support confirmed they too had checked the contents and all links in the contents went to the correct destinations. They did suggest that maybe my use of links to sub-sections (it's a non-fiction book) meant that in some cases two destinations might appear on the same page at same time. Of course that depends on the reading device and the font size selected by the reader. But, just in case, I removed all the sub-section links in the ncx Table of Contents (while retaining them in the html contents for the readers' convenience).

After editing the ncx contents with Sigil's tools [Tools>Table of Contents>Edit] and swapping around the order of a couple of smaller files, I checked out the toc.ncx file and Sigil had correctly reordered everything and the playorder was correct.

Maybe D2D has updated their intake app to add synthetic page numbers if no print page numbers are present? You might want to ask D2D about it.

I always download a proof copy from D2D and I have never seen any alterations made by them. As far as I can tell, the downloaded proof is identical to the copy I uploaded.

From the supplementary info supplied by Support and presented as emanating from Apple after the third attempt, it appears that Apple's on-boarding app derives some sort of expectation from the EPUB file about the page number it expects from the file's table of contents and then finds that this differs from what their app reports when it gets to that page.

For example, Chapter 13 has page 159 listed in the Table of Contents but it is on page 160 in the book.

I've tried reading this epub file in several readers including Calibre's reader, Kindle's Previewer3 and Adobe Digital Editions (which is D2D's recommended app). I never saw any problem - or page numbers come to that. I'll try and get confirmation from D2D that they have made no modification to the file before they sent it to Apple.

Could I maybe generate a new ncx table of contents from Tools > Table of Contents > Generate? Is there a preferred procedure for so doing - deleting the toc.ncx for example?

[KevinH]

You can run Generate Table of Contents at any time and it will always replace th current ncx. But before doing that you might try to running the TOCSaver plugin before trying to generate a new table of contents as it may help prevent loss of some post edited heading titles.

Does the current NCX have a pagelist section if you look directly at the NCX? I can't see how any pages can be assigned without it? Could your epub be using an adobe pagemap.xml?

[philja]

Originally Posted by KevinH

Does the current NCX have a pagelist section if you look directly at the NCX? I can't see how any pages can be assigned without it? Could your epub be using an adobe pagemap.xml?

There is no pagelist section that I can see. Apart from the nav map with the playorder list, there are just four line of Meta as per below:

    <meta name="dtb:uid" content="106cbcb7-6634-4e91-8b80-398da8476f13"/>
    <meta name="dtb:depth" content="1"/>
    <meta name="dtb:totalPageCount" content="0"/>
    <meta name="dtb:maxPageNumber" content="0"/>

I don't have any adobe pagemap.xml referenced. The only affair I have with Adobe is if I transfer the epub file to my W10 box to check how it displays in Digital Reader.

I've regenerated the table of contents and the ncx appears perfectly formed. The file performs as expected in all the apps I have to hand.

I've recontacted D2D who've said they will consult with Apple and get back to me. It has to be down to what Apple's reception is doing to the file.

[philja]

The situation doesn't improve even after D2D's Support supposedly intervening with Apple's eBook reception. The only thing that has changed is the list of 'errors' associated with the table of contents. Here is the latest listing:

The Table of Contents navigates to incorrect sections. Please make sure it navigates to the exact section indicated in the title.
Please save your changes. A replacement file has not been uploaded.
EPUB internals validation passed
Table of Contents is not correctly implemented. For example, Chapter 13 has page 159 listed in the Table of Contents but it is on page 160 in the book. Introduction is on page 13, not page 12 as listed in the Table of Contents, etc.. Please check and fix each Table of Contents entry that requires a change.
1- Introduction is on page 13, not page 12. 2 - is on page 27, not 26. Etc... Please fix the issues with the Table of Contents not linking to the correct pages.

From this, it appears that Apple considers each chapter appears in the book at one page later than is shown in the table of contents.

My problem is that the page numbers must be artifacts of Apple's reception software app. There are no page numbers in my epub2 book and never have been.

There seems to some sort of logical problem here. Apple's software is creating page numbers but somehow creates two different page numbers for each chapter: one from the table of contents and another from the book body material. And they are not happy that these two numbers differ.

In practice, in every app I have tried, a click on the link in the table of contents takes me to the right place in the book.

I wish I could understand why this is happening. Is there some underlying coding inside Sigil at a lower level that the html and CSS ?

All the more curious that for several years, Apple has been happily selling this book in its previous version. Either they didn't create page numbers until recently or something has been introduced by my using more up-to-date versions of Sigil this year.

[DiapDealer]

I can almost guarantee it's not Sigil. Sigil doesn't automatically (and certainly not silently) do anything related to page numbers. Never has.

[KevinH]

Are you sure there is not an html toc that refers to page numbers? Could you have set metadata that refers to a specific issn printed version of the book?

[philja]

Originally Posted by KevinH

Are you sure there is not an html toc that refers to page numbers? Could you have set metadata that refers to a specific issn printed version of the book?

The html table of contents has absolutely no references to page numbers.

The only time the word 'page' even appears in the xhtml files is one single entry of <div class="pagebreak-avoid"> and in the CSS file it only appears as a 'page-break-before' associated with h1 headings and a pagebreak-avoid definition.

There is a print version of this book but it only had 102 pages and Apple's complaint includes references to page numbers as high as 159 and 160. All references inside the print version were scrupulously removed before conversion to epub and that was back in 2015. So for the past 7 years, KDP and D2D (including Apple) have distributed this ebook without any problems.

I have examined all the metadata and there is no reference to any print version.

I'm going to have one more go at remaking the toc and then maybe I'll just try sending the original docx file to D2D and let them convert to epub and see if there's any difference.

[Sarmat89]

Have you checked that TOC entries refer only the HTML files and any header elements are the first children of their respective bodies?

[DiapDealer]

Has it been confirmed that the removal of the following meta items from the toc.ncx was tried? I can't find it in the thread if it was.

<meta name="dtb:totalPageCount" content="0"/>
<meta name="dtb:maxPageNumber" content="0"/>

Neither are needed by books with no page numbers. And although their presence (with zero values) shouldn't cause any problems under normal circumstance, I can't help but speculate whether a change to Apple's upstream process may not look at the dtb:totalPageCount and/or dtb:maxPageNumber attributes (regardless of their values) as evidence that page numbering is, in fact, in play and needs to be processed as such.

Just a thought.

[philja]

Originally Posted by Sarmat89

Have you checked that TOC entries refer only the HTML files and any header elements are the first children of their respective bodies?

Yes, indeed. Originally the toc had links to various subsection of chapters with h2 and h3 headings just like the html contents. That is because the book is a non-fiction somewhat technical subject where readers will most likely not read in a linear fashion but will use the contents listing to go directly to a subject of interest. But when these problems began, I reduced the toc.ncx table of contents to chapter h1 headings only and they are the first item in any chapter.

Originally Posted by DiapDealer

Has it been confirmed that the removal of the following meta items from the toc.ncx was tried? I can't find it in the thread if it was.

Code:

<meta name="dtb:totalPageCount" content="0"/>
<meta name="dtb:maxPageNumber" content="0"/>

Neither are needed by books with no page numbers. And although their presence (with zero values) shouldn't cause any problems under normal circumstance, I can't help but speculate whether a change to Apple's upstream process may not look at the dtb:totalPageCount and/or dtb:maxPageNumber attributes (regardless of their values) as evidence that page numbering is, in fact, in play and needs to be processed as such.

Just a thought.

No. I haven't removed those lines. I thought they must be required in order to indicate no pages. But if they are not needed, I'll certainly remove them from my next offering. Good idea, thanks.

[DiapDealer]

Originally Posted by philja

I thought they must be required in order to indicate no pages.

It's possible that they might be needed as some sort of submission prerequisite (D2D or Apple), but from an epub standards point of view, I don't believe there's a need to indicate "no pages". I could be wrong. Sigil adds them to a new ncx by default for some reason. But I can't find any indication they absolutely need to be there for reading systems to properly handle page-less epubs.