Add title="" to h* based on existing TOC -- suggestion for new feature (or plugin?)

[Tex2002ans]

Originally Posted by Hitch

I do wish to jump in here--if by "professionals" you mean formatters, formatters are absolutely not responsible for title casing.

[...]

(Tex here does do that, for one of his clients in particular, but that's a unique situation.)

I'm wondering, what happens in the case of ALL CAPS chapter titles, but you have to Proper Title Caps in the TOC? You just go with the simple First Cap Every Word?

[DNSB]

Personally, I will open the toc.ncx, nav.xhtml or the embedded HTML TOC in an external editor and then copy/paste into the header tag. Batch edit the existing headings to something like <h3 class="h3class" title=""> as an example and then copy/paste from the external editor in the "". Also useful when some genius decided that images are the way to go for chapter numbering. I played with doing this programmatically but the results generally took as much or more effort to clean up than simply doing it manually from the start. So damn many special cases needed to be accounted for.

[Mister L]

Originally Posted by Hitch

I do wish to jump in here--if by "professionals" you mean formatters, formatters are absolutely not responsible for title casing. We're not editors or proofreaders and we are not paid to do that work. In fact, if we "forget our place" and do make corrections, we're typically told off for it. I once had to listen to an ass-chewing by a rather jumped-up self-published author, who informed me that if she wanted her book ruined by "a bunch of self-important clerks," she'd hire some.

(Tex here does do that, for one of his clients in particular, but that's a unique situation.)

So, if you happened to have meant formatters, please know that formatters do NOT make those choices. Believe me, at least once a week I get a manuscript with "forward" in it and punctuation outside of quotation marks where it oughtn't be, incorrect emdash use, and on and on and on, but formatters don't earn remotely enough money to also proofread and correct what we see. And trade publishers? They hire out Indian firms, so...fuhgeddaboudit.

Hitch

I'm not talking about title casing at all, although this thread seems to be determined to be about title casing for some reason as people keep getting the impression that is what I'm talking about.

The examples I gave in the first post of the thread are specifically concerning the html code and even more specifically the difference between how chapter titles are presented in the html files vs. how they should be shown in the toc. Using h1 for the chapter number, then h2 for the title, instead of h1 with a br to send the title to the next line, or multiple spans to make fake smallcaps or something like that. I am a formatter too, I know the quality of the text is not my responsibility.

Originally Posted by slowsmile

I think in my original post to you I mentioned that you will not get good results with this plugin if your epub is using fake titlecase or fake smallcaps in your headings. In the code above -- you're using fake titlecase. In the above html you're using span classes to capitalize the first letter and then using another span class to make all text after lower case.

However, the good news is that I had another quick look at the above problem and I think I've found a way to resolve it. Try running the new plugin below.

I must have misunderstood when you said the plugin "Gathers all the toc item heading strings from the epub TOC page and puts them into a list." What did you mean by that exactly? What does it do with the list of titles from the toc page? The titles from the toc page are exactly the text I am trying to get, NOT the titles which are in the html pages.

The use of fake smallcaps or other weird code should not matter; the whole point of the plugin is to be used precisely in cases such as that, when it is excessively complicated to extract the title from the h* tags using the habitual methods like regex but there is already a correct toc page in the file. Otherwise it's easy enough to just do a regex for this.

The plugin shouldn't automate the regex; it should not take any text from the html file, no matter what kind of tag it is in, precisely because that text is not presented the way it should be in the toc, whereas the text in the toc page IS already correct. The plugin should copy the titles from the toc page, exactly as they are, and simply PASTE them back to each html file, without modifying the case, or the text itself, in any way. Is it possible to modify this plugin to do that?

If it's not possible, no worries. If it is possible but you don't want to spend more time on it, let me know, and I will try to figure it out myself.

I tried the new version of the plugin, unfortunately it is doing something very strange now, I am not sure exactly what. Here is the result after running it:

<div title="Le Lion Sur La Collinenonenone*Unprends Les Ronces À Pleines Mains, Et Tu Te Piqueras…Far Dareis Maicar’A’Carncar’A’Carnfais Flèche De Tout Bois, Ou Laisse Les Ténèbres S’Abattre Sur Le Monde…Leitmotivsaidinsaidinsaidinsaidinidemla Seule Façon De Vivre C’Est De Mourir…Je Dois Mourir. La Mort, Voilà Tout Ce Que Je Mérite…Car’A’Carnsaidinsaidinsammael, Rahvin, Moghedien Et…Non, S’Ils Étaient Tous Des Suppôts Des Ténèbres, Tu Les Utiliserais Quand Même.Shoufacar’A’Carncar’A’Carnet Où Qu’Elles Soient… Des Aes Sedai… Au Service De Tous… Mais Le Hall Des Serviteurs Est Détruit, Désormais… Détruit Pour Toujours… Ilyena, Mon*Amour…Car’A’Carnnonenone">
    <h1 id="toc_marker-6">1</h1>
    <h2><span class="Cap">L</span><span class="SmallCap">E LION SUR LA COLLINE</span></h2>
    <p class="Center"><span class="SmallCap"><span><img alt="06jordan-1.jpg" src="../Images/06jordan-1.jpg" width="20%"/></span></span></p>
    <p class="Center"><span class="SmallCap">*</span></p>
    <p>La Roue du Temps tourne et les Âges naissent et meurent, laissant dans leur sillage des souvenirs destinés à devenir des légendes. Puis les légendes se métamorphosent en mythes qui sombrent eux-mêmes dans l’oubli longtemps avant la renaissance de l’Âge qui leur donna le jour.</p>

This text is not the text from the toc file, but it doesn't seem to be the text from the html file either (I left the first paragraph of the chapter so you can see), so I am not sure where it came from.

[Mister L]

Originally Posted by DNSB

Personally, I will open the toc.ncx, nav.xhtml or the embedded HTML TOC in an external editor and then copy/paste into the header tag. Batch edit the existing headings to something like <h3 class="h3class" title=""> as an example and then copy/paste from the external editor in the "". Also useful when some genius decided that images are the way to go for chapter numbering. I played with doing this programmatically but the results generally took as much or more effort to clean up than simply doing it manually from the start. So damn many special cases needed to be accounted for.

Yes. Exactly this. This is what I am trying to automate: copying the titles from the toc.ncx or nav.html or embedded html toc back into the individual chapters, without having to do it by hand.

[Mister L]

Originally Posted by Tex2002ans

I'm wondering, what happens in the case of ALL CAPS chapter titles, but you have to Proper Title Caps in the TOC? You just go with the simple First Cap Every Word?

You shouldn't have to think about the case of the title at all; the whole point is the titles have already been properly formatted in the existing toc and they are displayed differently in the html files, so regenerating the toc will require a lot of extra work; I just want to copy the text from the existing toc and past it back into the html files so I can regenerate the toc with the correct titles which are already in the toc.

Edit for clarity: I mean, I want to copy the text from the existing toc into a title attribute for each heading, without in any way relying on the text in the h* tags of each file.

So for example:

HTML file:

<h1 id="toc_marker-26">1</h1>

<h2><span class="Cap">E</span><span class="SmallCap">N CHEMIN POUR</span> <span class="Cap">S</span><span class="SmallCap">HADAR</span> <span class="Cap">L</span><span class="SmallCap">OGOTH</span></h2>

Corresponding title IN THE EXISTING TOC:

 <navPoint id="navPoint-7" playOrder="7">
      <navLabel>
        <text>1. En chemin pour Shadar Logoth</text>
      </navLabel>
      <content src="Text/9782820516909-5.xhtml#toc_marker-26"/>
    </navPoint>

The hypothetical plugin should copy the text from the toc: "1. En chemin pour Shadar Logoth" and stick it back in the html file at the point the link goes to, either by putting it directly into a title="" attribute inside the h* tag or, if that is too complicated as DiapDealer has suggested, by inserting it in an html comment <!-- 1. En chemin pour Shadar Logoth --> so that it can be regexed into place.

The final result (after the plugin and maybe a regex to place the text where it belongs) should be something like this:

HTML file:

<h1 id="toc_marker-26" title="1. En chemin pour Shadar Logoth">1</h1>

<h2><span class="Cap">E</span><span class="SmallCap">N CHEMIN POUR</span> <span class="Cap">S</span><span class="SmallCap">HADAR</span> <span class="Cap">L</span><span class="SmallCap">OGOTH</span></h2>

(I am ignoring for the reasons of this post the poorly constructed html code which should obviously also be fixed and which can easily be fixed with a simple regex).

[slowsmile]

I've altered the plugin again, fixed some problems. I've changed the following:

• The heading list now obtains all the headings from the NCX file that you wanted.

• Also copies the NCX headings in the list -- unchanged -- directly into the the title tags of the html headings.

• Fixed the div section problem.

I'm still working on the span kludge problem. I'll give you a new plugin after I fix that.

[Mister L]

Originally Posted by slowsmile

I've altered the plugin again, fixed some problems. I've changed the following:

• The heading list now obtains the headings from the NCX file as you wanted.

• Also copies the NCX headings in the list -- unchanged -- directly into the the title tags of the headings.

• Fixed the div section problem.

I'm still working on the span kludge problem. I'll give you a new plugin after I fix that.

Oh, brilliant! Thank you so much!!
I don't know what the word "kludge" means but the first two steps are exactly what I am hoping for.
Can't wait to try out the new plugin.

[Hitch]

Originally Posted by Tex2002ans

I'm wondering, what happens in the case of ALL CAPS chapter titles, but you have to Proper Title Caps in the TOC? You just go with the simple First Cap Every Word?

Hey, sugar! (FWIW, "Tex" and I are friends outside of here and professional colleagues as well.)

I ask the customer. With most scenarios, I do as requested by the customer--even when it's wrong. I try my best, within the bounds of the contractual relationship and my tolerance for BS, to keep them from going completely off the rails in terms of grammar and proper punctuation usage, (if asked), but after that, it's on them. (For example, gently explaining that emdashes and ellipses don't actually have spaces before and aft, in print, sigh...)

For TOCs title case, I use the usual if "converting" from all-caps. (APA Title case with l/c for articles and the like.) If we're asked to do so and then we tell the customer to check it. (And 99% of the time, they don't, but whatchagonnado?)

Hitch

[slowsmile]

@Mister L...Here's the latest plugin. I think the new plugin is close to what you want now. Anyway, we'll see.

By the way, "kludge" refers to unnecessary dross code. When you use spans with classes to achieve titlecase or smallcaps for headings it makes it awkward and more difficult for the plugin code to actually find the html heading names because the heading name characters are all split up and insulted with multiple span styling. Why they don't just type in the heading the way they want to see it in their original doc(without using spans) just boggles the mind. How easy is that? That's span "kludge". For my own ebooks I never use span styling in my html headings which are always styled as a standard either as allcaps(typed in) or as proper titlecase(typed in) and that's it. See below.

[Hitch]

Originally Posted by slowsmile

@Mister L...Here's the latest plugin. I think the new plugin is closer to what you want now. Anyway, we'll see.

By the way, "kludge" refers to unnecessary dross code. When you use spans with classes to achieve titlecase or smallcaps for headings it makes it awkward and more difficult for the plugin code to actually find the html heading names because the heading name characters are all split up and insulted with multiple span styling. Why they don't just type in the heading the way they want to see it in their original doc(without using spans) just boggles the mind. How easy is that? That's span "kludge". For my own ebooks I never use span styling in my html headings which are always styled as a standard either as allcaps(typed in) or as proper titlecase(typed in) and that's it. See below.

That sort of code is used for the obvious reasons. If you "only" use smallcaps, then the text won't look right on older devices. You have to reverse-engineer the font sizes (small), (etc), so that it looks okay on KF7 and other devices. That's the simple eplanation.

Hitch

[Mister L]

Originally Posted by slowsmile

@Mister L...Here's the latest plugin. I think the new plugin is close to what you want now. Anyway, we'll see.

I tried it out but it doesn't seem to make any changes to the file. I used the same file as previously, generated an html toc with h1 "Table of Contents", then ran the new plugin; it changed the indents of the code in the html files but no new text was added. I tried running it with no html toc as well, just the ncx, but same result. Do I need to do anything particular to the file before running it?

Originally Posted by slowsmile

By the way, "kludge" refers to unnecessary dross code. When you use spans with classes to achieve titlecase or smallcaps for headings it makes it awkward and more difficult for the plugin code to actually find the html heading names because the heading name characters are all split up and insulted with multiple span styling. Why they don't just type in the heading the way they want to see it in their original doc(without using spans) just boggles the mind. How easy is that? That's span "kludge". For my own ebooks I never use span styling in my html headings which are always styled as a standard either as allcaps(typed in) or as proper titlecase(typed in) and that's it. See below.

Right, I see, thanks. I don't remember seeing that word before so I wasn't sure what it meant, too lazy to google it this week-end.

I am a code purist, like you seem to be, so I avoid extra spans and styles as much as possible, don't put the chapter number and title in 2 separate tags... The fake small-caps code drives me crazy but there are a couple of clients who use it in their house styles (which were set up before I started making their books, and since I'm not the only person who works for them it's not easy getting changes made all through the workflow). They have small-caps on the titles in the chapters, but they want the toc to show the titles in lower case (also part of their house styles). Nobody else who works for them ever puts the "toc display" title text in the files so whenever they give me an old file to modify, fixing the toc alone sometimes takes more time than whatever specific task I'm meant to be doing (thus the request for this plugin). I have no idea how the person who originally made the file manages it; do they hand-code the entire toc, every single time they work on a file?? These are epub2 books, if you modify the order of the files or add one or split or merge anything you have to renumber every single navpoint PLUS any modifications to the text... What a colossal waste of time.

Originally Posted by Hitch

That sort of code is used for the obvious reasons. If you "only" use smallcaps, then the text won't look right on older devices. You have to reverse-engineer the font sizes (small), (etc), so that it looks okay on KF7 and other devices. That's the simple eplanation.

Hitch

That's more or less the situation I am dealing with; those clients want epub2 files so advanced formatting is limited. But I dislike those sort of "kludge" for many reasons, not only because it makes dealing with the file more complicated but also from a usability standpoint it is not great as it can break the search function for instance. I'm trying to make gentle changes with them little by little but it's one of those "small cog big machine" situations.

When I have more liberty to decide though, for the majority of my clients, I make only fully backwards-compatible epub3 files and make typographical smallcaps using font-variant: smallcaps. I put the text either in allcaps (acronyms, roman numerals, etc.) or title case, depending on the situation. Very few of my clients object. One of them requests fake smallcaps on roman numerals / acronyms so they will be sure of the result everywhere, but those are outliers and as that still requires a span on the full word it doesn't really make much difference.

The way I see it, smallcaps are a purely esthetic choice; they look pretty but if the text is displayed in title case on older software, no meaning is lost and it's not hugely less beautiful to read. And it makes cleaner code that is fully searchable (and does not cause trouble when generating the toc although anyway when I make a book, I add the title="" attribute from the start wherever necessary so I never have these problems on my own books).

Also, it's really frustrating to me that the reading software developped by companies who easily have the ressources to improve it if they wanted to is still so primitive after all these years, and the way I see it, if we keep limiting ourselves to what is currently supported by all (or at least the majority) of reading sw, the sw devs have no reason to improve, because "there's no demand". Whereas if it's possible to point to X number of ebooks with typographical smallcaps and dropcap initials made with "first-letter" (etc.) and say "these books are not displaying as intended on your software", then at least there is a concrete argument in favour of improving, at least on all the new machines; as it's absolutely possible to make epub3 books which are completely compatible with epub2 rs (they won't have smallcaps or dropcap initials but they'll still be absolutely readable and look good), nobody loses.

[slowsmile]

@Mister L...I've just downloaded the plugin again and tested it on 3 different epubs and it worked fine. It uses the file headings from the toc.ncx file to find the xhtml file headings. When I tested the plugin, it always found the xhtml headings and always inserted the title attribute with the correct NCX heading name without a problem.
'
Things to try:

1. Download the plugin again. I did hurriedly change it a couple of times after initially uploading over small things that I wasn't happy with in the plugin. So I changed the plugin and re-uploaded several times. Trouble is I don't know when you downloaded the plugin. So best to download the plugin again to be sure that you have the latest working version. I'm sorry about that but I was tired of making new plugins versions for every small plugin change.

2. Before you run the plugin you should always generate a new epub NCX file by using Tools > Table of Contents > Generate Table of Contents. Doing that will make sure that all the appropriate and correct html headings(containing any h1 to h6 tags) will be in the toc.ncx file. Then after that try running the plugin. Basically what I'm saying here is that if you have no heading names in your toc.ncx file then the plugin won't be able to find any headings in the xhtml files. Just to also mention that if your epub has no epub TOC page then it's also quite possible that your epub's NCX file will contain no heading names as well.

3. Last point, be sure that you don't have any double quotes or apostrophe's in your NCX headings. In test I found the '&apos;' html entity in an NCX heading name with an apostrophe. That html entity will prevent that heading being found using my plugin. I got this problem despite having no entries in Edit > Preferences > Preserve Entities(which surprised me). Also neither the Reformat > Mend.. nor Mend and prettify... will fix that html entity problem in the NCX file but you can fix it easily enough by manually deleting the html entity and re-typing the apostrophe as proper text into the relevant NCX heading name.

...And I promise no further plugin changes until after I get your next reply...

[Mister L]

Originally Posted by slowsmile

@Mister L...I've just downloaded the plugin again and tried it out on 3 different epubs and it worked fine. It uses the file headings from the toc.ncx file to find the xhtml file headings. When I tested the plugin, it always found the xhtml headings and always inserted the title attribute with the correct NCX heading name without a problem.

Arg, I downloaded it again just to be sure and reinstalled it but it still didn't add any titles at all.

I set a checkpoint to compare the before and after, it appears to modify the indentations of all the tags and unexpectedly moved around a few unrelated things in the doctype but I don't see any modifications to the titles.

Here's a screencap of the diff of one file.

Originally Posted by slowsmile

Things to try:

1. Download the plugin again. I did hurriedly change it a couple of times after initially uploading over small things that I wasn't happy with in the plugin. So I changed the plugin and re-uploaded several times. Trouble is I don't know when you downloaded the plugin. So best to download the plugin again to be sure that you have the latest working version. I'm sorry about that but I was tired of making new plugins versions for every small plugin change.

No worries. I downloaded and installed again just to be sure.

Originally Posted by slowsmile

2. Before you run the plugin you should always generate a new epub NCX file by using Tools > Table of Contents > Generate Table of Contents. Doing that will make sure that all the appropriate and correct html headings(containing any h1 to h6 tags) will be in the toc.ncx file. Then try running the plugin after that. Basically what I'm saying here is that if you have no heading names in your toc.ncx file then the plugin won't be able to find any headings in the xhtml files.

Good god no this is absolutely what you should NOT do!! The reason for the plugin is to preserve the titles from the existing toc BEFORE you regenerate the toc! I need this plugin for the files where the toc has been modified by hand and cannot be regenerated automatically without significant losses. If you regenerate the toc before you run the plugin, then you've already lost all the previous formatting that you wanted to keep thanks to the plugin! The ncx is definitely populated with the correct heading names, however they do not match the headings in the html files. That's why I want to copy them from the ncx back to the html files, before making any changes which require regenerating the toc.

Originally Posted by slowsmile

3. Last point, be sure that you don't have any double quotes or apostrophe's in your NCX headings. In test I found the '&apos;' html entity in an NCX heading name with an apostrophe. That html entity will prevent that heading being found using my plugin. I got this problem despite having no entries in Edit > Preferences > Preserve Entities(which surprised me). Also neither the Reformat > Mend.. nor Mend and prettify... will fix that html entity problem in the NCX file but you can fix it easily enough by manually deleting the html entity and re-typing the apostrophe as proper text into the relevant NCX heading name.

There are some apostrophes but all as Unicode characters, no html entities at all in the ncx.

Originally Posted by slowsmile

...And I promise no further plugin changes until after I get your next reply...

Would it help you if I gave you my sample file to use as a test case? Then you could see exactly the conditions I am talking about. I can't give it to you as it is because of copyright but maybe if I make an extract of just a few chapters and borkify the text that seems like it should be okay.

[DNSB]

Originally Posted by Mister L

Would it help you if I gave you my sample file to use as a test case? Then you could see exactly the conditions I am talking about. I can't give it to you as it is because of copyright but maybe if I make an extract of just a few chapters and borkify the text that seems like it should be okay.

You could run the Scramble Epub plugin on it and then either attach it to a message here or other arrange for @slowsmile to access it.

[Mister L]

Originally Posted by DNSB

You could run the Scramble Epub plugin on it and then either attach it to a message here or other arrange for @slowsmile to access it.

Is there specifically a Scramble Epub plugin? I was thinking of Toxaris' Borkify plugin, is that the same you had in mind?