Lazy user wants to know... TOC generation

GrannyGrump · 03-30-2017, 01:28 AM

Is there any way to force the Sigil TOC auto-generation to use *all* the existing ID tags?

I always add my own ID codes to headings. I note that when the TOC is auto-generated, if the ID is at the top of the (x)html page, Sigil will omit that ID from the link, and only use the file name.

Thus, if I have coded a page so:

Quote:

[line 5] <h4 id="a01-23">Hoodoo McFiggin’s Christmas</h4>

[line 80]<h4 id="a01-24">The Life of John Smith</h4>

The html TOC will look like this (toc.ncx information is similar):

Quote:

<div class="sgc-toc-level-2">
<a href="../Text/A01b-LitLapses.html">Hoodoo McFiggin’s Christmas</a>
</div>

<div class="sgc-toc-level-2">
<a href="../Text/A01b-LitLapses.html#a01-24">The Life of John Smith</a>
</div>

I use the TOC information to create index or other pages needing links. Because I use AutoHotKey scripts, I rely on character-count to get the required link info.

Is there any way to get ALL the IDs to appear, or is it only possible to manually edit to replace them?
(I think I know what the answer is, but hope springs eternal.)

Doitsu · 03-30-2017, 02:29 AM

Quote:

Originally Posted by GrannyGrump

Is there any way to force the Sigil TOC auto-generation to use *all* the existing ID tags?

It might be possible to write a Python script for that, however, IIRC, you're using an older Sigil version.

AFAIK, the only way to outsmart Sigil is to insert a dummy h1 heading with a sigil_not_in_toc class after the body tag. I.e., you'd have to do the following:

1. Replace all <body> tags with <body><h1 id="dummy" class="sigil_not_in_toc">dummy</h1>.

2. Generate the TOC.

3. Delete all <h1 id="dummy" class="sigil_not_in_toc">dummy</h1> headings.

GrannyGrump · 03-30-2017, 02:45 AM

Yup, still using v 0.8.6

I am going to upgrade Any Day Now (TM).
(as I have been saying for months....)

The dummy heading is a fast and easy workaround, thank you so much for that! That will see me through.

JSWolf · 03-30-2017, 07:03 AM

Just a quick note, on some Readers, the more ID's you have in the NCX ToC, the slower it will be to pull up the NCX ToC. So what Sigil is correctly doing is not using the ID when all you are doing is having the ToC entry go to the top of the file.

GrannyGrump · 03-30-2017, 07:14 AM

Oh, understood. Sigil is doing exactly what it is designed to do, to provide the most "economical" ncx.

But in this case, I am essentially using "copy & paste" to use those links elsewhere than in the TOC. So I really do need the IDs for those pasted links in my alphabetical "Index of Titles", which is really just a page full of links. Otherwise, Pagina EpubCheck will warn me about unidentified URL fragments (missing IDs), and the links will fail.

theducks · 03-30-2017, 10:03 AM

GG
You might generate an Inline (TOC) Index with your id markers and keep the traditional NCX (for better performance), just add you 'Index" to the regular TOC (NCX)

JSWolf · 03-30-2017, 01:36 PM

Quote:

Originally Posted by GrannyGrump

Oh, understood. Sigil is doing exactly what it is designed to do, to provide the most "economical" ncx.

But in this case, I am essentially using "copy & paste" to use those links elsewhere than in the TOC. So I really do need the IDs for those pasted links in my alphabetical "Index of Titles", which is really just a page full of links. Otherwise, Pagina EpubCheck will warn me about unidentified URL fragments (missing IDs), and the links will fail.

Where do you need these ID's? The links without the ID's won't cause an unidentified URL fragments.

GrannyGrump · 03-31-2017, 12:35 AM

@theDucks --- that is the way I will go. Thanks!

@JSWolf --- sorry, tail end of a long day, I used the wrong terminology. Error message I meant is: Fragment identifier is not defined.
When I have multiple targets all on the same page/html file, each target must have a unique id. If each clickable link does not have a unique target id to go to, things go south real soon.

The page I am constructing has approx. 600 items, so I want to avoid manual editing if I possibly can.

davidfor · 03-31-2017, 07:12 AM

Quote:

Originally Posted by JSWolf

Just a quick note, on some Readers, the more ID's you have in the NCX ToC, the slower it will be to pull up the NCX ToC. So what Sigil is correctly doing is not using the ID when all you are doing is having the ToC entry go to the top of the file.

For my curiosity, which ereaders and at how many ids in the ToC? And are you sure it is the number of ids and not simply the number of ToC entries?

KevinH · 03-31-2017, 09:26 AM

Hi All,
Sorry to be late to this discussion but Doitsu just pointed it out to me ... ;-)

I am not sure I understand exactly what is being asked for here. Would you please provide one xhtml code snippet from the area around the body tag that shows exactly why or how you need to NOT use an id that defaults to the top of the file? That will help me better to understand what is going on that the current TOC algorithm is not doing well with.

Right now, the code in Sigil sets any *first* heading in a file after after the body tag that is within 20 source lines from the line with the body tag to point to the top of the file (in other words, no need for an fragment).

Normally this is a good thing and is what most people want since it makes for fewer fragments and less code in general.

If there is a good argument / logic why this creates an issue with NCX or Nav generation, I would be happy to consider making the 20 source line difference a user controllable setting (or disabling this check completely).

But I would have to understand the use case or problem (and I am not sure I do now) and would have to see that a simple search and replace to insert a hidden "dummy" header, right after the body tag wouldn't work just as well. After generating the NCX/TOC/NAV, a second Search & Replace can remove the hidden header tags easily to prevent code bloat.

In general, I really hate adding new settings that would just confuse most novice users when a simple workaround by experienced users would prevent the entire issue. But if it is needed to prevent bugs in TOC/NCX/Nav creation under some circumstances, then a setting is probably needed.

Thanks

KevinH · 03-31-2017, 10:22 AM

To add to this discussion of the "20 lines" setting I ran the following test and it nicely used the fragment when it built the toc.ncx

Code:

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
  "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title></title>
</head>
<body>
<!-- this is a comment

















-->
<h1 id="huh">This is a heading</h1>
<p>&nbsp;</p>
</body>
</html>

So if we make that a "setting" then changing that setting to 2, would surely make it easy to add a workaround by just adding a comment or even just a blank line after body to control things.

JSWolf · 03-31-2017, 10:34 AM

Quote:

Originally Posted by GrannyGrump

@JSWolf --- sorry, tail end of a long day, I used the wrong terminology. Error message I meant is: Fragment identifier is not defined. When I have multiple targets all on the same page/html file, each target must have a unique id. If each clickable link does not have a unique target id to go to, things go south real soon.

So why not have links that go to the top of the flow not use an ID and all the rest use an ID? Some Readers/apps do slow down when the ToC uses IDs. So the less IDs you can use, the better.

theducks · 03-31-2017, 10:43 AM

Quote:

Originally Posted by JSWolf

So why not have links that go to the top of the flow not use an ID and all the rest use an ID? Some Readers/apps do slow down when the ToC uses IDs. So the less IDs you can use, the better.

Because, if someone who favors huge, monoliths

joins all the HTML sections, it is broke. Merge only adjusts the filename.

JSWolf · 03-31-2017, 10:52 AM

Quote:

Originally Posted by theducks

Because, if someone who favors huge, monoliths

joins all the HTML sections, it is broke. Merge only adjusts the filename.

And in that case, it's up to that person to fix the problem he/she has now caused. There's no reason to punish others for something someone might do. Also, this means changing the Sigil code to do something that's not optimal.

DiapDealer · 03-31-2017, 11:05 AM

At one point (long ago) Sigil didn't default to the top of the file (ToC generation). Then I remember a discussion taking place, and the behavior being changed to what it is now. I don't remember the driving force behind the change.

I admit I'm having difficulty understanding GrannyGrump's workflow, and how Sigil's default behavior is hampering it. I understand the need to be able to copy/paste urls en masse. I'm just not quite understanding how the undefined fragment errors might be introduced when doing so. Sigil's TOC generation routines don't remove any IDs, so with or without an url fragment, the target should still exist.

Is it possible the "undefined url fragment" errors are completely unrelated to the task at hand?

There are avoidable conditions where splitting/merging with Sigil will create broken urls that will produce such an error when validating. I know because I recently modified Sigil to warn a user that those conditions are present before performing the split/merge.

Perhaps a small, step-by-step example of the workflow involved (with a sample epub) would help us understand better?

03-30-2017, 07:14 AM	#5
GrannyGrump Obsessively Dedicated... Posts: 3,200 Karma: 34977896 Join Date: May 2011 Location: JAPAN (US expatriate) Device: Sony PRS-T2, ADE on PC	Oh, understood. Sigil is doing exactly what it is designed to do, to provide the most "economical" ncx. But in this case, I am essentially using "copy & paste" to use those links elsewhere than in the TOC. So I really do need the IDs for those pasted links in my alphabetical "Index of Titles", which is really just a page full of links. Otherwise, Pagina EpubCheck will warn me about unidentified URL fragments (missing IDs), and the links will fail. Last edited by GrannyGrump; 03-30-2017 at 07:16 AM.

03-31-2017, 09:26 AM	#10
KevinH Sigil Developer Posts: 7,644 Karma: 5433388 Join Date: Nov 2009 Device: many	Hi All, Sorry to be late to this discussion but Doitsu just pointed it out to me ... ;-) I am not sure I understand exactly what is being asked for here. Would you please provide one xhtml code snippet from the area around the body tag that shows exactly why or how you need to NOT use an id that defaults to the top of the file? That will help me better to understand what is going on that the current TOC algorithm is not doing well with. Right now, the code in Sigil sets any first heading in a file after after the body tag that is within 20 source lines from the line with the body tag to point to the top of the file (in other words, no need for an fragment). Normally this is a good thing and is what most people want since it makes for fewer fragments and less code in general. If there is a good argument / logic why this creates an issue with NCX or Nav generation, I would be happy to consider making the 20 source line difference a user controllable setting (or disabling this check completely). But I would have to understand the use case or problem (and I am not sure I do now) and would have to see that a simple search and replace to insert a hidden "dummy" header, right after the body tag wouldn't work just as well. After generating the NCX/TOC/NAV, a second Search & Replace can remove the hidden header tags easily to prevent code bloat. In general, I really hate adding new settings that would just confuse most novice users when a simple workaround by experienced users would prevent the entire issue. But if it is needed to prevent bugs in TOC/NCX/Nav creation under some circumstances, then a setting is probably needed. Thanks Last edited by KevinH; 03-31-2017 at 09:41 AM.

03-31-2017, 10:22 AM	#11
KevinH Sigil Developer Posts: 7,644 Karma: 5433388 Join Date: Nov 2009 Device: many	To add to this discussion of the "20 lines" setting I ran the following test and it nicely used the fragment when it built the toc.ncx Code: <?xml version="1.0" encoding="utf-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <title></title> </head> <body> <!-- this is a comment --> <h1 id="huh">This is a heading</h1> <p> </p> </body> </html> So if we make that a "setting" then changing that setting to 2, would surely make it easy to add a workaround by just adding a comment or even just a blank line after body to control things.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
automatic TOC generation	Neurone	Conversion	1	06-03-2015 06:36 AM
CSS visiblility setting throws off TOC generation in MOBI	Guido Henkel	Calibre	1	11-02-2010 05:16 AM
CSS visiblility setting throws of TOC generation in MOBI	Guido Henkel	Calibre	0	11-01-2010 06:13 PM
ToC generation issue	tecgeo	Calibre	6	09-21-2010 08:44 PM
How to force TOC generation out of scanned PDF	magphil	Calibre	5	10-01-2009 04:53 PM

03-30-2017, 02:45 AM	#3
GrannyGrump Obsessively Dedicated... Posts: 3,200 Karma: 34977896 Join Date: May 2011 Location: JAPAN (US expatriate) Device: Sony PRS-T2, ADE on PC	Yup, still using v 0.8.6 I am going to upgrade Any Day Now (TM). (as I have been saying for months....) The dummy heading is a fast and easy workaround, thank you so much for that! That will see me through.

03-30-2017, 07:03 AM	#4
JSWolf Resident Curmudgeon Posts: 73,976 Karma: 128903378 Join Date: Nov 2006 Location: Roslindale, Massachusetts Device: Kobo Libra 2, Kobo Aura H2O, PRS-650, PRS-T1, nook STR, PW3	Just a quick note, on some Readers, the more ID's you have in the NCX ToC, the slower it will be to pull up the NCX ToC. So what Sigil is correctly doing is not using the ID when all you are doing is having the ToC entry go to the top of the file.

03-30-2017, 10:03 AM	#6
theducks Well trained by Cats Posts: 29,801 Karma: 54830978 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	GG You might generate an Inline (TOC) Index with your id markers and keep the traditional NCX (for better performance), just add you 'Index" to the regular TOC (NCX)

03-31-2017, 12:35 AM	#8
GrannyGrump Obsessively Dedicated... Posts: 3,200 Karma: 34977896 Join Date: May 2011 Location: JAPAN (US expatriate) Device: Sony PRS-T2, ADE on PC	@theDucks --- that is the way I will go. Thanks! @JSWolf --- sorry, tail end of a long day, I used the wrong terminology. Error message I meant is: Fragment identifier is not defined. When I have multiple targets all on the same page/html file, each target must have a unique id. If each clickable link does not have a unique target id to go to, things go south real soon. The page I am constructing has approx. 600 items, so I want to avoid manual editing if I possibly can.

03-31-2017, 11:05 AM	#15
DiapDealer Grand Sorcerer Posts: 27,549 Karma: 193191846 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	At one point (long ago) Sigil didn't default to the top of the file (ToC generation). Then I remember a discussion taking place, and the behavior being changed to what it is now. I don't remember the driving force behind the change. I admit I'm having difficulty understanding GrannyGrump's workflow, and how Sigil's default behavior is hampering it. I understand the need to be able to copy/paste urls en masse. I'm just not quite understanding how the undefined fragment errors might be introduced when doing so. Sigil's TOC generation routines don't remove any IDs, so with or without an url fragment, the target should still exist. Is it possible the "undefined url fragment" errors are completely unrelated to the task at hand? There are avoidable conditions where splitting/merging with Sigil will create broken urls that will produce such an error when validating. I know because I recently modified Sigil to warn a user that those conditions are present before performing the split/merge. Perhaps a small, step-by-step example of the workflow involved (with a sample epub) would help us understand better?

Advert

Advert