Calibre FB2 output

[kmashlea]

I have been trying the varous formats on my new Bebook one 2010 and found that the FB2 format seems the best having title, page/pages, time, battery and chapter line.
I manually converted a few books using BookDesigner which is great but a slow process and doesn't seem to copy the images.

I tried Calibre to convert which is fast and simple to use except it doesn't do the chapters.

I used the debug facility that I found on this forum and the output does have the chapters marked correctly. It is just FB2 output that doesn't seem to pick it up. It marks <strong> around the chapters but (as I found by comparing the bookdesigner output) it only puts section around the book whereas Bookdesigner puts section and title around the chapters.

I am really impressed with everything else in Calibre, thanks but it would be fantastic if it could do chapters.

Is there something that I am not doing or is there a problem with the converter?

[kovidgoyal]

IIRC, the FB2 output plugin doesn't do this, but user_none the author of the plugin will have to confirm.

[kmashlea]

Thanks Kovid for quick reply, shame about that though, Calibre does everything else so much better and faster.

Guess i'll just have to do it the slow/laborious way through book designer.

[user_none]

calibre's FB2 output creates on section for the entire book. While this is valid, creating individual sections for each chapter is preferred. Most FB2 readers will auto generate a TOC from the individual sections within the document.

Due to incompatibilities (not limitations) between the FB2 format and how calibre handles chapters creating sections for each chapter is near impossible.

FB2 uses the text from the chapter itself:

<section>
  <title>
    <p>I am what will be used for the TOC as well as the chapter header in the document itself</p>
  </title>
<p>...</p>
</section>
...

calibre has a very flexible TOC generator which can create TOC items from text, and tags. TOC items can have different text than what is contained in the book. Also, TOC items can point to location not just content. This works fine with XHTML which calibre uses internally during conversion to represent the book. However, there is no good way to convert something like the following to FB2 while marking the chapter properly.

<a id="toc_1" />
<div>
  <p>The toc_1 has different text than me. I'm in a p but I could be in pretty much anything. I could even be something like <p><b>Title</b><br />Start the text.</p>
</div>

Basically there is no good and reliable way to turn that anchor point into a proper FB2 section. We can't use the text from the TOC item itself because we don't want to modify the actual book content. We also can't reliability tell what to include in the <title> tag of the section.

[kmashlea]

I am not quite sure what you are saying?

The fb2 output currently has
<a id="calibre_link-2"/><strong><p>CHAPTER 1</p></strong><p>
.......
</p><a id="calibre_link-3"/><strong><p>CHAPTER 2</p></strong><p> etc

What is wrong with creating it like:
</section>
<section><title><a id="calibre_link-2"/><strong><p>CHAPTER 1</p></strong></title><p>
.......
</p></section>
<section><title><a id="calibre_link-3"/><strong><p>CHAPTER 2</p></strong></title><p>
etc.

This gives the page break at each chapter and also splits into fb2 sections.

The title is whatever is found by Calibre, at worst you will end up with the odd extra chapter/title that may be picked up by mistake but that would be much better than no chapters/titles.

[user_none]

Originally Posted by kmashlea

The title is whatever is found by Calibre, at worst you will end up with the odd extra chapter/title that may be picked up by mistake but that would be much better than no chapters/titles.

The TOC (remember users and setup their own xpaths to find elements and use custom text) just points to an element in the intermediate document. That element could be anything. So there is no reliable way to detect what is the actual chapter from that element because the element could be an anchor point.

Originally Posted by kmashlea

The fb2 output currently has
<a id="calibre_link-2"/><strong><p>CHAPTER 1</p></strong><p>
.......
</p><a id="calibre_link-3"/><strong><p>CHAPTER 2</p></strong><p> etc

In this case the FB2 output isn't detecting the chapter or doing anything special. The source book has the chapter heading in a tag that is translated to strong in the FB2 output.

The FB2 output has no idea what a chapter or chapter heading is. It is possible to use h tags to determine this but that isn't compatible with calibre TOC system because TOC items don't have to be h tags. A TOC item can be defined as pretty much anything.

[capidamonte]

user_none, could you give an example where this would fail?

I'm having similar difficulty understanding what the impossibility is: if you are defining the TOC entries in the xpath, why can't you use the result? Is it that you cannot determine what the content for the FB2 <title> element would be? Or that you cannot determine (even implicitly) where the section ends?

[user_none]

Originally Posted by capidamonte

user_none, could you give an example where this would fail?

<a id="toc_1" />
<p>Chapter 1:</p>
<p>I am the first chapter.</p>

<h1>Chatper 2:</h1>
<p id="toc_2">I am the second chapter. Notice the toc marker is on the content not the heading.</p>

<div id="toc_3">I am the third chapter. Notice I don't have a heading. Yet because the TOC can just point to a location and stores the text separately I will show in the TOC as "Chapter 3".</div>

[kmashlea]

Forgive me if I am being naive on this as I do not really know xpath, or fb2 etc apart from what I am seeing / learning.

You say Calibre has not found the 'CHAPTER's, I though that is what the preferences were for? if it is not found, how come in the intermediate html file that gets created it shows id="calibre_toc_" style="page-break-before:always" before each chapter? Doesn't this mean that it has found the chapters?

If I manually search the fb2 output looking for the 'CHAPTER' e.g. "id="calibre_link" and insert the section / title as I stated, it works perfectly, showing page breaks and sections and looks great.

I am not trying to do anything fancy (wouldn't know how anyway), I just would like the output to be broken into sections with page breaks for each section. The only reason for this is to make it easier to read.

[user_none]

Originally Posted by kmashlea

if it is not found, how come in the intermediate html file that gets created it shows id="calibre_toc_" style="page-break-before:always" before each chapter?

It's not an issue with finding the chapters during the conversion process. It's an issue of transforming the XHTML with an external TOC into the FB2 format. The intermediate can mark explicitly what a chapter is but it does not have to. It can just point to an arbitrary element in the content. Also, the TOC text is external to the document and does not have to relate to the content in the book.

Look at the examples I posted previously. In example 2 the chapter is marked after the chapter title. In 3 the chapter has no title in the document.

FB2 requires sections to be in a specific format for the application to generate a TOC:

<section>
  <title>
    <p>TITLE TEXT</p>
  </title>
  <p>CONTENT</p>
  ...
</section>

Again, there is no reliable way to determine what is supposed to be in the title and what is supposed to be in the content. In example 2 the title will always be added to the end of the previous chapter.

In example 3, well what do we use as the title? Do we use the first sentence? The first X characters of the sentence? If there is no chapter title in the book's content then breaking it up like this will not look correct.

Originally Posted by kmashlea

If I manually search the fb2 output looking for the 'CHAPTER' e.g. "id="calibre_link" and insert the section / title as I stated, it works perfectly, showing page breaks and sections and looks great.

In some cases this works fine. But not all source material will end up looking like this. Again, look at the examples of possible chapter markings that are possible going into the FB2 output.

[kmashlea]

Thanks, I finally see what you are saying (about time he says!).

As an experiment, after yesterdays discussion, I wrote a little program to call the converter on a directory of books. After converting each the books to FB2, I got it to search through the fb2 output and insert the section/title as I had said.

You were right about the different scenarios, only about 6 out of 10 (in a sample of a hundred books) came out as I expected. Some had extra chapters/page breaks where the odd sentence got picked up as a chapter, but doesn't really mess the reading up much.
The others didn't seem to pick up the markers at all even though some had 'chapter's in them? The ones that didn’t work just remained the same as if I had left the output alone so, there doesn't seem to any detriment to what I did though. I turned the TOC off from the output as I am not really interested in the actual table at the beginning of the book which was good as well.

On the bright side, I now have 60ish books looking good without having to manually modify them with book designer etc and an automated way of converting the bulk of my books to this format.

[user_none]

Originally Posted by kmashlea

As an experiment, after yesterdays discussion, I wrote a little program to call the converter on a directory of books. After converting each the books to FB2, I got it to search through the fb2 output and insert the section/title as I had said.

You were right about the different scenarios, only about 6 out of 10 (in a sample of a hundred books) came out as I expected. Some had extra chapters/page breaks where the odd sentence got picked up as a chapter, but doesn't really mess the reading up much.

Would you be willing to share your program? Since this is so sought after I'm willing to integrate it as an option.

[kmashlea]

It is fairly crude and simple, I don't think you would want the whole thing as I set it up as a right-click menu option to convert a directory at a time. It converts based on a priority order of types so that I don't have to do anything.

I can send you the conversion routine though if you want and if you let me know how or where to send it.

[user_none]

Originally Posted by kmashlea

I can send you the conversion routine though if you want and if you let me know how or where to send it.

Send it to john AT nachtimwald.com

[kmashlea]

It's on its way to you, thanks.