BBC Recipe conversion_options explaination needed.

[mattst]

Currently the various BBC News recipes either do not work at all or do not work very well (including the one posted on 07-23-2010 by Starson17 on this board). Part of the reason is changes to the site and partly that the HTML used by the news site developers often seems to have minor errors in, Eg. a double space used in a tag, a trailing space at the end of it, a missing '-' in a tag and so on.

I am in the process of completely re-writing the recipe, including using regular expressions for all the elements in the keep_only and remove tags lists in an effort to make the recipe robust.

Being a recipe noob, though with a strong programming background, there is one aspect of the existing BBC News recipes which I do not understand.

Eg. From the Darko Miletic / Starson17 recipe (link below):
https://www.mobileread.com/forums/showpost.php?p=1022483&postcount=14

conversion_options = {
'comments'         : description,
'tags'             : category,
'language'         : language,
'publisher'        : publisher,
'linearize_tables' : True }

Of course I've read the help on it here - http://manual.calibre-ebook.com/news_recipe.html - but frankly that has not helped me understand what is happening with the conversions made in the code above. Can someone explain to me what the various conversion_options above are actually doing please?

Many thanks.

[kovidgoyal]

Conversion options are passed to the conversion system. See

ebook-convert dummy.epub dummy2.epub -h

for help on all the conversion options.

[mattst]

Thanks very much kovidgoyal, now I understand what they do.

Some variables seem to be passed to ebook-convert automatically from the recipe without the need for an entry in the conversion_options list.

The class variables 'title', 'extra_css', '__author__', all seem to be passed from the recipe to ebook-convert.

Also the ebook-convert command line option '--pubdate' is filled automatically by the string described in 'timefmt'.

Have I got that all correct?

Or should I be doing this?

conversion_options = {
'title'            : title,
'authors'          : __author__,
'comments'         : description,
'tags'             : category,
'language'         : language,
'publisher'        : publisher,
'extra-css'        : extra_css,
'linearize_tables' : False }

Thanks.

PS. Should I explicitly set 'linearize_tables' to False? Or is that the default?

PPS. Is there somewhere where I can read what gets passed to ebook-convert without the need of an entry in conversion_options?

Cheers.

[kovidgoyal]

You should not pass any metadata options in conversion_options. That field is really meant for options that affect how the ebook is generated. Any non-metadata option that you do not pass takes its default value, which you can check with ebook-convert -h

[mattst]

Thanks again kovidgoyal.

On your suggestion I've taken out the metadata I was setting using the conversion_options list in my recipe but now find that the resulting mobi ebook that's been created by ebook-convert has 'calibre' listed as both 'author' and 'publisher'.

Is this what is supposed to happen (designed that way) or perhaps it's because I'm using Calibre v. 0.7.50 on Ubuntu/Lucid (which was got via a PPA called ppa:n-muench/calibre-lucid) and not the latest version 0.8.25?

Thanks.

PS. BTW looking through a couple of dozen recipes I've just seen that quite a lot set metadata using the conversion_options list.

[Starson17]

Originally Posted by mattst

On your suggestion I've taken out the metadata I was setting using the conversion_options list in my recipe but now find that the resulting mobi ebook that's been created by ebook-convert has 'calibre' listed as both 'author' and 'publisher'.

Is this what is supposed to happen (designed that way)

It's designed that way.

PS. BTW looking through a couple of dozen recipes I've just seen that quite a lot set metadata using the conversion_options list.

I keep them in my standard recipe to remind me to see if changes are needed, and often forget to take them out, or just leave them in as harmless, or use them to set minor things I'd like set (publisher, linearize tables). I often set the publisher to the web site source, but leave author configured as "calibre" so all recipe created ebooks are together and for ease of searching. Some recipes try to set tags, but that fails.

[mattst]

Thanks for the info. Starson17, it's appreciated.

Originally Posted by Starson17

Some recipes try to set tags, but that fails.

My tag settings worked fine.

Cheers.

[kovidgoyal]

It is designed that way. The fact that you can override it via conversion_options is an accident and is not guaranteed to continue working (I just dont care about this issue enough to fix the accident).

[Starson17]

Originally Posted by mattst

Thanks for the info. Starson17, it's appreciated.
My tag settings worked fine.
Cheers.

That's odd. I've seen several recipes that have tag settings, but don't produce tags in the calilbre GUI. I tested one just before posting and it failed. I've never looked into it, but I vaguely recall a post saying it can't be done, and I know I tried it once before and it failed. Would you post a recipe that sets tags and let me try it?

[kovidgoyal]

The OP is probably talking about the tags in the output file when converted via the command line.

[Starson17]

Originally Posted by kovidgoyal

The OP is probably talking about the tags in the output file when converted via the command line.

OK, thanks. That's sort of what I suspected.

[mattst]

Originally Posted by Starson17

That's odd. I've seen several recipes that have tag settings, but don't produce tags in the calilbre GUI. I tested one just before posting and it failed. I've never looked into it, but I vaguely recall a post saying it can't be done, and I know I tried it once before and it failed. Would you post a recipe that sets tags and let me try it?

Originally Posted by kovidgoyal

The OP is probably talking about the tags in the output file when converted via the command line.

Yes kovidgoyal is exactly right.

class Whatever(BasicNewsRecipe):

...
    tags = 'news, sport'
...
    conversion_options = { 'tags' : tags }
...

ebook-convert Whatever.recipe out.mobi

The tags 'news, sport' are shown when out.mobi is imported to Calibre.

HTH.

PS. You replied while I was writing my reply!

[Starson17]

Originally Posted by mattst

Yes kovidgoyal is exactly right.

    tags = 'news, sport'
...
    conversion_options = { 'tags' : tags }
...

Yes, that's how it's attempted. It has no direct affect on the tags in the GUI, which is what most people want. The Adventure Gamers recipe, which I often point people to for a recipe that handles multipage (it's alphabetically the first with that type of code) has that exact form, but does not create tags in the GUI database. It will bring them in if imported in a separate step.

[mattst]

Originally Posted by kovidgoyal

It is designed that way. The fact that you can override it via conversion_options is an accident and is not guaranteed to continue working (I just dont care about this issue enough to fix the accident).

Thanks.

May I ask what the reasoning for setting both the 'author(s)' and 'publisher' metadata to 'calibre' is (when using ebook-convert to create an ebook from a recipe)? While I can see why calibre is being set as the author, I don't understand why it could be considered the publisher?

In news.py the class BasicNewsRecipe(Recipe) defines this:

#: Publication type
#: Set to newspaper, magazine or blog
publication_type = 'unknown'

Have you thought about adding some more categories, Eg. newsfeed, newsportal, academic and such like?

Finally am I setting 'smarten-punctuation' correctly below, I seem to be having some problems with it?

conversion_options = { 'smarten-punctuation' : True }

Thanks.

PS. (Being a noob) I've only just realized that you are the creator of calibre. Excellent, excellent, excellent piece of software - thanks so much for it.

[mattst]

EDIT: Ok got it now. When running the recipe from within the Calibre GUI, the tags do not get added. END EDIT

Originally Posted by Starson17

Yes, that's how it's attempted. It has no direct affect on the tags in the GUI, which is what most people want. The Adventure Gamers recipe, which I often point people to for a recipe that handles multipage (it's alphabetically the first with that type of code) has that exact form, but does not create tags in the GUI database. It will bring them in if imported in a separate step.

Maybe I am missing the point but when I've created a new mobi ebook using my recipe and ebook-convert, when I add the ebook to Calibre the tags are immediately shown, as the highlighted item in the screenshot linked below shows (I haven't edited the metadata manually).

http://www.zilefile.com/files/16978_...ot_Calibre.png

PS. I'm using Calibre v. 0.7.50 on Ubuntu/Lucid.