please help a newbie recipe hacker :)

[Alexis]

Hi there,

Firstly, big thanks to the developer(s) for Calibre.

In an effort to learn about recipes, I thought I'd start by tweaking the already-existing Ars Technica one. There were two main issues I wanted to address:
1. Formatting: captions, images, authors, dates etc. could do with prettier formatting, and I have not had too much trouble customising the existing recipe to achieve this.
2. Structure / TOC in *.mobi: this is where I am stuck at the moment. I would appreciate help & advice on this. I am obviously new at this, so wouldn't be suprised if I'd missed something obvious.

This is what I do (using 0.7.40):

ebook-convert $CALIBRE_PATH/resources/recipes/ars_technica.recipe ars.mobi -vv --debug-pipeline ars_debug/ --test --output-profile=kindle

And then:

ebook-viewer ars.mobi

The problem is that when I click on the TOC in ebook-viewer, the main view moves to after the article title, image and caption, right at the beginning of the main article text. The behaviour I was expecting was that it should move to the beginning of the article, including the title/headline (this is what happens when I run the same command, but using the nytimes_sub recipe instead).

So I guess structure detection may be going wrong?

The article headline/titles are tagged with <h2> for both NYTimes and Ars Technica processed html, so I don't understand what is going on. I have tried to look at the processed HTML from the NYT recipe in detail, comparing to the Ars Technica processed HTML but I haven't found a clue as to what might be different which would explain bad structure detection in one case but not the other.

As a first attempt at forcing the structure detection, I tried adding

--level3-toc=//h:h2

to the command line, but to be honest I am a bit confused as to how I should go about debugging the problem. In fact, I'm still not sure whether it has to do with structure detection at all.

Any pointers, advice etc would be gratefully received!

Thanks!

[kovidgoyal]

I'd suggest using the kindle previewer instead of the ebook viewer to test mobi files. The news downloads in MOBI format have various features specifically for the Kindle that the viewer doesn't support, including navigation with the 5 way controller on the Kindle. If you want to use the ebook viewer use epub output instead.

Most probably, on the Kindle you will not see this issue.

[Alexis]

Hello,

Thanks for your prompt reply.

I should have specified that I have already checked the behaviour on the Kindle, and I see the same issue.

I like the sound of the Kindle previewer - didn't know about it, will look it up.

[kovidgoyal]

Have you tried it with the periodicals menu on the Kindle, i.e press the center button on the 5-way controller and see the list of articles and click an article?

[Alexis]

Yes I have. When I do this, the main text is displayed, but I have to press the < button (to go one page back) in order to move back to the headline.

[kovidgoyal]

Use the --debug-pipeline option and look at the entries in input/toc.ncx file to see where they point.

[Alexis]

Well the <content> tags within the <navpoint>s look to me like they're pointing to the correct html. I'm not sure what I'm looking for exactly... Is there some way the toc.ncx could be pointing to some specific place within the html rather than just the start?
What am I looking for?

[kovidgoyal]

If they are pointing to some part of the html the filename will be followed a # and a word

[Alexis]

Well that doesn't seem to be the case. I have copied it below, maybe you will notice something unusual?

The fact that other recipes (e.g. Nature News) seem to work fine with me makes me think it isn't something I've done

Also, I have tried to output to EPUB and view with ebook-viewer, and the TOC looks much better (with nested content), and clicking on the items takes me to the right places. So maybe the conversion to MOBI isn't working well with this particular markup/structure?

<?xml version='1.0' encoding='utf-8'?> 
<ncx xmlns:calibre="http://calibre.kovidgoyal.net/2009/metadata" xmlns="http://www.daisy.org/z3986/2005/ncx/" xml:lang="en"> 
  <head> 
    <meta content="a3eaa47e-598f-49d4-84a4-377f20c87c03" name="dtb:uid"/> 
    <meta content="3" name="dtb:depth"/> 
    <meta content="calibre (0.7.40)" name="dtb:generator"/> 
    <meta content="0" name="dtb:totalPageCount"/> 
    <meta content="0" name="dtb:maxPageNumber"/> 
  </head> 
  <docTitle> 
    <text>Table of Contents</text> 
  </docTitle> 
  <navMap> 
    <navPoint id="dbdd347d-91b4-4b3b-a245-d76b5960690d" playOrder="1"> 
      <navLabel> 
        <text>Infinite Loop (Apple content)</text> 
      </navLabel> 
      <content src="feed_0/index.html"/> 
      <calibre:meta name="description">The Art of Technology</calibre:meta> 
      <navPoint id="ac4170c4-b6fc-4a26-9a37-27284b09d217" playOrder="2"> 
        <navLabel> 
          <text>Week in Apple: it's all about the Verizon iPhone (and iOS 4.3)</text> 
        </navLabel> 
        <content src="feed_0/article_0/index.html"/> 
        <calibre:meta name="author">jacqui@arstechnica.com (Jacqui Cheng)</calibre:meta> 
        <calibre:meta name="description"> It goes without saying that the event invitation that went out last Friday was indeed a legit hint that the Verizon iPhone was on its way. This week's top Apple stories naturally gravitated towards news about the CDMA device, but we also discussed what's in the new iOS 4.3 beta, why Apple won't get rid of the hardware Home button anytime soon, what happened to VLC on the App Store, and more. Verizon iPhone: no longer a myth, available in February: How long have you been wait…</calibre:meta> 
      </navPoint> 
      <navPoint id="d5ebabca-286a-4244-93ec-2f9c5d1f3102" playOrder="3"> 
        <navLabel> 
          <text>Project HiJack uses iPhone audio jack to make cheap sensors</text> 
        </navLabel> 
        <content src="feed_0/article_1/index.html"/> 
        <calibre:meta name="author">chris.foresman@arstechnica.com (Chris Foresman)</calibre:meta> 
        <calibre:meta name="description"> Making accessories that tie into an iOS device's Dock connector is an expensive proposition: it requires getting certain components from Apple and applying for a costly "Made for iPhone" (or iPod or iPad) license. However, it is possible to use the headphone jack for two-way data communication with an iPhone and also to power small electronic circuits. A group of students and faculty from the University of Michigan's Electrical Engineering and Computer Science Department have …</calibre:meta> 
      </navPoint> 
    </navPoint> 
    <navPoint id="aa2f09c0-832e-45af-9dc3-fda438c7cfa8" playOrder="4"> 
      <navLabel> 
        <text>Opposable Thumbs (Gaming content)</text> 
      </navLabel> 
      <content src="feed_1/index.html"/> 
      <calibre:meta name="description">The Art of Technology</calibre:meta> 
      <navPoint id="329ce852-47c8-4729-8615-f5e60a88ab10" playOrder="5"> 
        <navLabel> 
          <text>Week in gaming: 3D PCs, Killzone 3, PS3 lawsuits, Bulletstorm</text> 
        </navLabel> 
        <content src="feed_1/article_0/index.html"/> 
        <calibre:meta name="author">bkuchera@arstechnica.com (Ben Kuchera)</calibre:meta> 
        <calibre:meta name="description"> Did you hear the 3DS battery is terrible? It's a scandal! We're looking forward to doing our own tests to see just how bad things are in real-world conditions, but this doesn't give us a ton of hope. In other news, the PlayStation 3 is now completely open due to the work of some dedicated hackers, and Sony is taking it to the courts. This week we also sounded off on Killzone 3, Dead Space 2, and Bulletstorm. A good week for gaming news? You betcha. …</calibre:meta> 
      </navPoint> 
      <navPoint id="47b0efc3-02b7-41ab-bb1a-a649bd9b2736" playOrder="6"> 
        <navLabel> 
          <text>Xbox rising: December sales up 42% from 2009</text> 
        </navLabel> 
        <content src="feed_1/article_1/index.html"/> 
        <calibre:meta name="author">bkuchera@arstechnica.com (Ben Kuchera)</calibre:meta> 
        <calibre:meta name="description"> Nintendo had a good December, selling 2.5 million DS systems and 2.3 million Wii systems, taking the crown for most hardware sold in calendar 2010. That sounds great, until you realize the DS and Wii sold 3.31 million and 3.81 million units respectively in December 2009. Microsoft has much more to brag about, as the Xbox 360 is the only console to see growth year over year, with 42 percent higher sales in 2010 than 2009. $6.2 billion was spent at retail on Xbox 360 hardware,…</calibre:meta> 
      </navPoint> 
    </navPoint> 
  </navMap> 
</ncx>

[kovidgoyal]

That looks correct, post your recipe, I'll have a look at it when I have a moment

[Alexis]

The problem occurs with the recipe distributed with Calibre: ars_technica.recipe

Thanks for your help!

[kovidgoyal]

This was indeed a bug in the MOBI output plugin, triggered by the presence of an empty id attribute in the input document. Fixed in next release.

[Alexis]

Thanks Kovid for this.

I will get back to improving the Ars Technica recipe soon, and will try to eliminate the empty id element anyway.