Table of contents - tweaking

[webfolk]

I'm converting the 2nd of 6 original books/volumes from Word to MOBI and EPUB files which we will post on our website for free downloading. This book has 5 Parts, each of which contains a portion of 23 Chapters. Every chapter contains 8 titled Sections. The source files were created in Word 2008 for Mac on a MacBook Pro OS 10.5.8. Only parts are designated style "Heading 1", level 1 in Word; only chapters are H2, L2; only sections are H3, L3.

Within the text of the book, each chapter starts with the book title, part, chapter, and section centered on the page. For example:

Sacred Memories
Part 1 ~ Earth Time
Chapter 1: The Awakening
A Gift for Someone

Right now, only the parts and chapters are showing up in the table of contents. Within the book, "Sacred Memories" and the part number/name each appear alone on a page with no other text. "Sacred Memories" is not designated as a heading, just part of the body of the text.

I would like each chapter to appear in the Calibre (0.7.24) generated table of contents like this:

Part X ~ Name of Part
Chapter X: Name of Chapter
Name of section 1
Name of section 2
...
Name of section 8

How do I get Calibre to detect the sections and place them in the T of C? Each has just an individual name and isn't labeled with the word "section" (or any other designation).

Within the text of the book, how do I get Calibre to not place a page break after "Sacred Memories" and the part number/name?

From studying the manual and searching this forum, I have an idea how to do this, but the actual procedure still eludes me. Any help or suggestions will be deeply appreciated. Thanks!

[jackie_w]

Hi,

Let me say up front that I may not be the ideal person to help you as I am a Windows user with an old version of Word (ver 10). However I can try.

Before I get into details perhaps you can look at the attached EPUB. If this is something like what you're aiming for then I can give you the sample Word doc and the Calibre conversion settings used.

[webfolk]

Originally Posted by jackie_w

Hi,

Let me say up front that I may not be the ideal person to help you as I am a Windows user with an old version of Word (ver 10). However I can try.

Before I get into details perhaps you can look at the attached EPUB. If this is something like what you're aiming for then I can give you the sample Word doc and the Calibre conversion settings used.

Hi Jackie,

I think I'll be able to read your Word file: have opened other older Windows versions successfully in the past.

I looked at the EPUB file you attached in the SONY Reader Library software and the table of contents looks just as I'd like: with all the parts, chapters, & sections. Bravo!

Within the text:

The title phrase "Sacred Memories" at the beginning of each chapter no longer has a page break after it (although it appears once at the beginning).

The title, part, chapter, and section names appear together on the same page at the beginning of each chapter, just as they should. However, the part number/names appear a second time at the beginning of each part, each on an individual page with no other text, as do the chapter number/names at the beginning of each chapter. Can this be prevented?

Thanks so much for your help!

[jackie_w]

Here is the source sample Word doc. The brightly coloured Heading styles I've used are only to try to make it clearer which Headings are in use.

As Calibre will not convert doc files, I prefer to save the .doc file as an HTML file before importing to Calibre, but in this case I saved as RTF as I wasn't sure whether you were comfortable with HTML.

In either case the conversion settings for the earlier EPUB were:

Structure Detection - Detect Chapters at - //*[name()='h1' or name()='h2']
Structure Detection - Chapter mark - pagebreak
Structure Detection - Insert pagebreaks before - //h:h3

Table of Contents - Level 1 TOC - //h:h1
Table of Contents - Level 2 TOC - //h:h2
Table of Contents - Level 3 TOC - //h:h3

Look & Feel - Extra CSS - h3 {display:none}

If you want to remove the Part and Chapter 'leader' pages then try substituting the following for the blue equivalents:

Structure Detection - Chapter mark - none

Look & Feel - Extra CSS - h1, h2, h3 {display:none}

The slight problem with this is that if you click on a Part (level1) or Chapter (level2) in the TOC in the Calibre Viewer then, because there is no page to go to, you end up on the page before the one you want. However, in practice on both my ereaders, this is not a problem because if I click on a Level1 or Level2 item in a 3-level TOC, all that happens is the higher TOC item expands to show the lower level TOC items, i.e I can't get out of the ereader's TOC until I click on a Level3 TOC item and this sends me to the correct page.

I've attached an updated EPUB which used the red conversion settings.

Anyway I'm sure you'll have fun playing around until you settle on your preferred solution. Good luck

[webfolk]

Originally Posted by jackie_w

As Calibre will not convert doc files, I prefer to save the .doc file as an HTML file before importing to Calibre, but in this case I saved as RTF as I wasn't sure whether you were comfortable with HTML.

I'm fine working with HTML. Being new to it, it's XPath that I find challenging.

Originally Posted by jackie_w

Anyway I'm sure you'll have fun playing around until you settle on your preferred solution. Good luck

Fun is what I'm in the process of ... and playing with the file and your suggestions have led me to some adaptations that should be made to the Word file I hadn't considered heretofore. In the print version, there is a design aesthetic that I'm seeing isn't suitable for an e-book. For example, the repetition of the title and part at the beginning of each chapter works nicely in our print version, but seems cumbersome in an e-book, especially in the table of contents.

Your conversion settings have shed some light on my understanding of XPath and I'm planning to revisit the Calibre manual in light of this light and the adaptations to the source file. I will post my results (or possibly more questions!) here.

Thanks for your help, Jackie. I appreciate the time and effort you took creating the test file. Color coding the conversion settings was very helpful.

[webfolk]

Originally Posted by webfolk

In the print version, there is a design aesthetic that I'm seeing isn't suitable for an e-book. For example, the repetition of the title and part at the beginning of each chapter works nicely in our print version, but seems cumbersome in an e-book, especially in the table of contents.

This isn't true. The problem isn't with the design aesthetic, it's getting it to work out in the conversion.

Originally Posted by webfolk

Fun is what I'm in the process of ...

Frustration & confusion have eclipsed the fun. I've tried many ways to get this to work, but it's not happening.

To precisely show what I'm going after, I'm attaching a PDF file intended for reading and printing from a computer. The internal table of contents (which is deleted in the HTML file I'm using for the ebook) looks like what the Calibre generated T of C should look like. Within the text, the headings at the beginning of each chapter are all on one page, no page breaks in between as is happening in the e-book files. The top heading ("Sacred Memories") that begins each chapter wasn't designated as a heading in Word, simply assigned a style with a body text outline level.

There are illustrations at the beginning of Chapters 5, 10, 11, 14, 16, 17, 20 and 21 that should be at the top of the page followed by the headings, but in the conversion are isolated on a page.

I'm also attaching the ZIP file created from the Word source file.

Thanks for any help or light that can be shed on this!

[jackie_w]

Hi webfolk, Here's an interim reply...

The reason mine worked and yours doesn't is that you will notice that I had far fewer <h1>, <h2> and <h3> tags in my sample doc than you do. I only placed one <h1>...</h1> para at the start of each Part and one <h2>...</h2> para at the start of each Chapter and one <h3>...</h3> para at the start of each Section. These "headings" are never displayed in the EPUB body text because the ExtraCSS I detailed (h1, h2, h3 {display:none}) makes them "invisible". However the TOC will see them and uses the labels correctly.

I actually styled the 4 lines of each "visible" Section heading as 'Heading 4' (i.e. <h4>) and these are not used in the TOC and not referenced in the Calibre conversion.

I can see that you may want each of the 4 lines to be styled slightly differently so I'll have a think about how you can achieve what you want with as few changes to your HTML as possible.

P.S. Another tip, when you save your Word doc as HTML try using the option SaveAs WebPage-filtered. You will get less 'MS excess baggage' in the resulting HTML file.

[webfolk]

Hi Jackie,

Originally Posted by jackie_w

The reason mine worked and yours doesn't is that you will notice that I had far fewer <h1>, <h2> and <h3> tags in my sample doc than you do. I only placed one <h1>...</h1> para at the start of each Part and one <h2>...</h2> para at the start of each Chapter and one <h3>...</h3> para at the start of each Section.

I tried this in one of the many files I created and it didn't turn out well ... but that was early on when I knew even less what I was doing than I do now. I'm going to try this again.

Originally Posted by jackie_w

I actually styled the 4 lines of each "visible" Section heading as 'Heading 4' (i.e. <h4>) and these are not used in the TOC and not referenced in the Calibre conversion.

I can see that you may want each of the 4 lines to be styled slightly differently so I'll have a think about how you can achieve what you want with as few changes to your HTML as possible.

I think the way to deal with this within the text is to make the 4 lines "Heading 4" and manually format each of the 4 in each chapter.

Originally Posted by jackie_w

P.S. Another tip, when you save your Word doc as HTML try using the option SaveAs WebPage-filtered. You will get less 'MS excess baggage' in the resulting HTML file.

Word for Mac doesn't have this option, but uses one that I think is similar, if not the same: "Save as Web Page/Save only display information into HTML." Still a lot of baggage, though.

I'm going to try the above as soon as I can, juggling my time with a web forum that also relates to our work.

Thank you, Jackie, for continuing with this!

[jackie_w]

Well I think I finally got there. I've attached updated HTML and the resulting EPUB.

1. In the HTML file added some new <h1> and <h2> paragraphs. Look for all references to the word invisible in both the CSS and HTML sections. I also had to remove 5 forced page-breaks, 1 just before each of the 5 Parts begins.
   
   
2. Use these Calibre conversion settings for ZIP to EPUB:
   

   Structure Detection - Detect Chapters at - //h:h1[re:test(@class, "invisible", "i")]
   Structure Detection - Chapter mark - none
   Structure Detection - Insert pagebreaks before - //h:h2[re:test(@class, "invisible", "i")]
   
   Table of Contents - Level 1 TOC - //h:h1[re:test(@class, "invisible", "i")]
   Table of Contents - Level 2 TOC - //h:h2[re:test(@class, "invisible", "i")]
   Table of Contents - Level 3 TOC - //h:h3
   
   Look & Feel - Extra CSS - nothing required here anymore, required CSS now inside HTML file

3. Tweak the EPUB. I'm not sure why this step is necessary. It may be a slight bug in the conversion process??? It would need to be reported in the Calibre ticket system to have an expert diagnose the problem. However, the above conversion produces an EPUB which looks like your PDF (I think) but as previously, when viewed in the Calibre viewer or the Firefox epubreader addon, if you select a Level 1 TOC item then it points to the wrong place (Level2s and Level3s are fine). I rectified this by using Calibre's EPUB-tweak feature to manually edit the toc.ncx file in a text editor. Only 5 small changes required to correct the destination of each of the Level 1 items (or Parts in your terms) so it points to the first Chapter in the Part, i.e. the same piece of HTML as the next item down in the toc.ncx file.
   
   I'm guessing that setting Structure Detection - Chapter mark to none is not that common and pagebreak is what most people use. You could then make a 'feature' of the leader page which contains only the 'Part x ~ ...' name, maybe with special formatting or an extra image. If you did it this way no epub-tweaking would be required.

Anyway, see what you think of the new EPUB

[webfolk]

Thanks Jackie!

I've been zipping around today doing a lot of multitasking ... and there's more still. I'm going to work with this a little later, so perhaps I'll have something more substantial to add by the end of my evening/the beginning of your new day.

With gratitude,
webfolk

[N13L5]

You may want to use something other than Word to generate your intermediate html, because Word is known to insert a LOT of extra tags and redundant full font instructions, causing considerable bloat to your final epub files.

For example: instead of just using <p> for each paragraph (linebreak in word) it uses <p fontface=blabla fontsize=blablabla fontcolor=blablabla> even though that's completely uneccessary, unless you used a different font specification in between.

That and other things add up sufficiently to potentially double your epub file sizes.

Opening your doc file in Open Office and saving as html makes cleaner html. Also, Open Office has a plugin available to save files directly to epub.

There's other ways to convert an office doc file to html, but I don't know what's available on the Mac

[webfolk]

Originally Posted by N13L5

Opening your doc file in Open Office and saving as html makes cleaner html. Also, Open Office has a plugin available to save files directly to epub.

I wasn't aware of Open Office ... just looked at the website and this might be a very good idea. I admire and appreciate open source software and the philosophy and motivation of the people behind it, very much in concert with The Folk of Yore.

Thanks for the suggestion!

[webfolk]

Originally Posted by jackie_w

Anyway, see what you think of the new EPUB

Jackie,

It looks just like the PDF. The illustrations are right there on the same page as the headings they accompany in their respective chapters and all the headings are also together on the same page. Excellent! Thank you again, Jackie.

One thing still happening, though, is "Sacred Memories" is appearing on a separate page before each chapter and towards the bottom of the page. I've looked at the file in Sigil and don't see any indication of a page break either in CSS or the HTML, but then I'm not very experienced in this area either.

This is not happening in those chapters with illustrations, which look exactly like they should. One way to correct this is simply to delete "Sacred Memories" from the beginning of each chapter, a sacrifice of aesthetics I'm willing to make at this point.

For those chapters with illustrations, clicking on the level 2 chapter name in the table of contents takes you right to that page. Where there is no illustration, clicking on the chapter takes you to the prior page with the solitary "Sacred Memories".

I noticed there's a lot less code. Did you clean it up? One of the things Word does when you "save as HTML" is include a cornucopia of font CSS and I don't see that anymore.

Thank you for all your efforts, Jackie. I deeply appreciate it!

[jackie_w]

Originally Posted by webfolk

One thing still happening, though, is "Sacred Memories" is appearing on a separate page before each chapter and towards the bottom of the page. I've looked at the file in Sigil and don't see any indication of a page break either in CSS or the HTML, but then I'm not very experienced in this area either.

You'll be pleased to know the answer to this one is very simple. Using Chapter 3 as an example, you have 13 blank lines hard-coded into the HTML between the "invisible" heading and 'Sacred Memories'. They look like this:

<p class="BeginChSpace"> </p>

If you remove some/all of them 'Sacred Memories' will move higher up the page. I considered doing this for you but decided it wasn't for me to decide. I tried to leave your HTML as untouched as I could.

Originally Posted by webfolk

This is not happening in those chapters with illustrations, which look exactly like they should.

Same reason. You appear to have removed most of the hard-coded blank lines for these chapters.

Originally Posted by webfolk

I noticed there's a lot less code. Did you clean it up? One of the things Word does when you "save as HTML" is include a cornucopia of font CSS and I don't see that anymore.

If you mean entries like this in the CSS:

@font-face
	{font-family:"Courier New";
	panose-1:2 7 3 9 2 2 5 2 4 4;}

Yes, there were 3 of them. I did remove them, force of habit, as I've never found that they did anything other than make the place look untidy I should have mentioned it. Sometimes there are hundreds of them. Other than that I am not aware of any changes I made other than the ones I listed in my previous post.

Compared to many Word-generated HTML files I've seen I thought yours was relatively clean. I suspect that save-as Webpage-filtered, or whatever the Mac equivalent is, would also have got rid of a lot of the mso-bidi-... type stuff.

Word will generate very clean HTML if you apply named styles. It looked to me as if this was mainly what you'd done.

However, its CSS is a lost cause in my experience. For my own books I remove all the Word-generated CSS and add a link to a standard ebook CSS file.

Originally Posted by webfolk

Thank you for all your efforts, Jackie. I deeply appreciate it!

You're welcome.

[webfolk]

Jackie,

Originally Posted by jackie_w

You'll be pleased to know the answer to this one is very simple. Using Chapter 3 as an example, you have 13 blank lines hard-coded into the HTML between the "invisible" heading and 'Sacred Memories'. They look like this:

<p class="BeginChSpace"> </p>

When I did my last post, I had remembered there were 13 blank lines at the beginning of each chapter not containing illustrations--among the few places I manually formatted the Word file. I was going to ask if they might be the reason for the unwanted page breaks, but didn't want to complicate that post even more than it was. Clearly, you didn't need this clue! At your suggestion, I removed the code from the "offending" chapters and--voila!--they look perfect!

I did this in Sigil and "Saved as" with an alternate file name (preserving the original file unchanged as backup). The new file, however, has a blank page inserted before the book cover. I've noticed this before when I saved from Sigil. How do I get rid of this and, for that matter, how do I prevent it from happening?

Originally Posted by jackie_w

If you remove some/all of them 'Sacred Memories' will move higher up the page. I considered doing this for you but decided it wasn't for me to decide. I tried to leave your HTML as untouched as I could.

It would have been fine. I deeply appreciate the sensitivity and respect you expressed through not doing it.

Originally Posted by jackie_w

If you mean entries like this in the CSS:

@font-face
	{font-family:"Courier New";
	panose-1:2 7 3 9 2 2 5 2 4 4;}

Yes, there were 3 of them. I did remove them.

Thanks for doing this!

The EPUB file required a bit more tweaking, some aesthetic formatting issues: for example, I reduced the indentation of each paragraph (and also 2 places where quoted phrases were indented) by altering the CSS style sheet in Sigil. I had also noticed the font was smaller and sans serif after the opening verse of some of the chapters. The style I had assigned to these in Word didn't carry over. I'm speculating this is because the style name I had used started with the numeral "1". I resolved this by creating a new CSS style and renaming the respective text block in HTML. The text looks a lot more balanced now in the Reader on my computer. I've attached the revised version for you to take a look. Please let me know if you agree or if there is some issue I'm not aware of with this.

I think I've resolved the table of contents by going into the T of C editor in Sigil ... which I just found. The Part names/numbers and the Chapter names/numbers were listed twice. I deselected the first occurrence of each so now, for example, clicking on Part 3 or clicking on Chapter 4 (which is the first chapter in Part 3) or clicking on "The Dawn is Nigh..." (the first section in Chapter 4) all takes you to the same text page, where they all appear together ... which is what I had wanted.

The T of C looks a bit strange when you open it, in that some chapters don't appear until you reveal them by clicking on the arrow preceding the part they're in, but--all things considered--I don't think this is a problem and preferable, in my opinion, to what was happening before.

I had framed this last bit as a question, but before hitting the "Submit Reply" button I got the inspiration to check out Sigil one more time.

Unless you see or think of something else, the only thing left to do is remove the blank page before the cover.

Again, thanks for all your help, Jackie!