Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Recipes

Notices

Reply
 
Thread Tools Search this Thread
Old 11-06-2011, 07:39 AM   #1
mattst
Enthusiast
mattst began at the beginning.
 
Posts: 27
Karma: 10
Join Date: Nov 2011
Device: Kindle
BBC Recipe conversion_options explaination needed.

Currently the various BBC News recipes either do not work at all or do not work very well (including the one posted on 07-23-2010 by Starson17 on this board). Part of the reason is changes to the site and partly that the HTML used by the news site developers often seems to have minor errors in, Eg. a double space used in a tag, a trailing space at the end of it, a missing '-' in a tag and so on.

I am in the process of completely re-writing the recipe, including using regular expressions for all the elements in the keep_only and remove tags lists in an effort to make the recipe robust.

Being a recipe noob, though with a strong programming background, there is one aspect of the existing BBC News recipes which I do not understand.

Code:
Eg. From the Darko Miletic / Starson17 recipe (link below):
http://www.mobileread.com/forums/showpost.php?p=1022483&postcount=14

conversion_options = {
'comments'         : description,
'tags'             : category,
'language'         : language,
'publisher'        : publisher,
'linearize_tables' : True }
Of course I've read the help on it here - http://manual.calibre-ebook.com/news_recipe.html - but frankly that has not helped me understand what is happening with the conversions made in the code above. Can someone explain to me what the various conversion_options above are actually doing please?

Many thanks.
mattst is offline   Reply With Quote
Old 11-06-2011, 10:10 AM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 24,766
Karma: 4369667
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Conversion options are passed to the conversion system. See

ebook-convert dummy.epub dummy2.epub -h

for help on all the conversion options.
kovidgoyal is offline   Reply With Quote
Old 11-06-2011, 11:42 AM   #3
mattst
Enthusiast
mattst began at the beginning.
 
Posts: 27
Karma: 10
Join Date: Nov 2011
Device: Kindle
Thanks very much kovidgoyal, now I understand what they do.

Some variables seem to be passed to ebook-convert automatically from the recipe without the need for an entry in the conversion_options list.

The class variables 'title', 'extra_css', '__author__', all seem to be passed from the recipe to ebook-convert.

Also the ebook-convert command line option '--pubdate' is filled automatically by the string described in 'timefmt'.

Have I got that all correct?

Or should I be doing this?

Code:
conversion_options = {
'title'            : title,
'authors'          : __author__,
'comments'         : description,
'tags'             : category,
'language'         : language,
'publisher'        : publisher,
'extra-css'        : extra_css,
'linearize_tables' : False }
Thanks.

PS. Should I explicitly set 'linearize_tables' to False? Or is that the default?

PPS. Is there somewhere where I can read what gets passed to ebook-convert without the need of an entry in conversion_options?

Cheers.
mattst is offline   Reply With Quote
Old 11-06-2011, 10:04 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 24,766
Karma: 4369667
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
You should not pass any metadata options in conversion_options. That field is really meant for options that affect how the ebook is generated. Any non-metadata option that you do not pass takes its default value, which you can check with ebook-convert -h
kovidgoyal is offline   Reply With Quote
Old 11-07-2011, 12:44 PM   #5
mattst
Enthusiast
mattst began at the beginning.
 
Posts: 27
Karma: 10
Join Date: Nov 2011
Device: Kindle
Thanks again kovidgoyal.

On your suggestion I've taken out the metadata I was setting using the conversion_options list in my recipe but now find that the resulting mobi ebook that's been created by ebook-convert has 'calibre' listed as both 'author' and 'publisher'.

Is this what is supposed to happen (designed that way) or perhaps it's because I'm using Calibre v. 0.7.50 on Ubuntu/Lucid (which was got via a PPA called ppa:n-muench/calibre-lucid) and not the latest version 0.8.25?

Thanks.

PS. BTW looking through a couple of dozen recipes I've just seen that quite a lot set metadata using the conversion_options list.

Last edited by mattst; 11-07-2011 at 12:49 PM.
mattst is offline   Reply With Quote
Old 11-07-2011, 02:05 PM   #6
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by mattst View Post
On your suggestion I've taken out the metadata I was setting using the conversion_options list in my recipe but now find that the resulting mobi ebook that's been created by ebook-convert has 'calibre' listed as both 'author' and 'publisher'.

Is this what is supposed to happen (designed that way)
It's designed that way.
Quote:
PS. BTW looking through a couple of dozen recipes I've just seen that quite a lot set metadata using the conversion_options list.
I keep them in my standard recipe to remind me to see if changes are needed, and often forget to take them out, or just leave them in as harmless, or use them to set minor things I'd like set (publisher, linearize tables). I often set the publisher to the web site source, but leave author configured as "calibre" so all recipe created ebooks are together and for ease of searching. Some recipes try to set tags, but that fails.
Starson17 is offline   Reply With Quote
Old 11-07-2011, 05:33 PM   #7
mattst
Enthusiast
mattst began at the beginning.
 
Posts: 27
Karma: 10
Join Date: Nov 2011
Device: Kindle
Thanks for the info. Starson17, it's appreciated.

Quote:
Originally Posted by Starson17 View Post
Some recipes try to set tags, but that fails.
My tag settings worked fine.

Cheers.
mattst is offline   Reply With Quote
Old 11-07-2011, 09:40 PM   #8
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 24,766
Karma: 4369667
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
It is designed that way. The fact that you can override it via conversion_options is an accident and is not guaranteed to continue working (I just dont care about this issue enough to fix the accident).
kovidgoyal is offline   Reply With Quote
Old 11-08-2011, 09:43 AM   #9
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by mattst View Post
Thanks for the info. Starson17, it's appreciated.
My tag settings worked fine.
Cheers.
That's odd. I've seen several recipes that have tag settings, but don't produce tags in the calilbre GUI. I tested one just before posting and it failed. I've never looked into it, but I vaguely recall a post saying it can't be done, and I know I tried it once before and it failed. Would you post a recipe that sets tags and let me try it?
Starson17 is offline   Reply With Quote
Old 11-08-2011, 10:06 AM   #10
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 24,766
Karma: 4369667
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
The OP is probably talking about the tags in the output file when converted via the command line.
kovidgoyal is offline   Reply With Quote
Old 11-08-2011, 10:29 AM   #11
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by kovidgoyal View Post
The OP is probably talking about the tags in the output file when converted via the command line.
OK, thanks. That's sort of what I suspected.
Starson17 is offline   Reply With Quote
Old 11-08-2011, 10:30 AM   #12
mattst
Enthusiast
mattst began at the beginning.
 
Posts: 27
Karma: 10
Join Date: Nov 2011
Device: Kindle
Quote:
Originally Posted by Starson17 View Post
That's odd. I've seen several recipes that have tag settings, but don't produce tags in the calilbre GUI. I tested one just before posting and it failed. I've never looked into it, but I vaguely recall a post saying it can't be done, and I know I tried it once before and it failed. Would you post a recipe that sets tags and let me try it?
Quote:
Originally Posted by kovidgoyal View Post
The OP is probably talking about the tags in the output file when converted via the command line.
Yes kovidgoyal is exactly right.

Code:
class Whatever(BasicNewsRecipe):

...
    tags = 'news, sport'
...
    conversion_options = { 'tags' : tags }
...

ebook-convert Whatever.recipe out.mobi
The tags 'news, sport' are shown when out.mobi is imported to Calibre.

HTH.

PS. You replied while I was writing my reply!
mattst is offline   Reply With Quote
Old 11-08-2011, 10:34 AM   #13
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by mattst View Post
Yes kovidgoyal is exactly right.

Code:
    tags = 'news, sport'
...
    conversion_options = { 'tags' : tags }
...
Yes, that's how it's attempted. It has no direct affect on the tags in the GUI, which is what most people want. The Adventure Gamers recipe, which I often point people to for a recipe that handles multipage (it's alphabetically the first with that type of code) has that exact form, but does not create tags in the GUI database. It will bring them in if imported in a separate step.
Starson17 is offline   Reply With Quote
Old 11-08-2011, 10:51 AM   #14
mattst
Enthusiast
mattst began at the beginning.
 
Posts: 27
Karma: 10
Join Date: Nov 2011
Device: Kindle
Quote:
Originally Posted by kovidgoyal View Post
It is designed that way. The fact that you can override it via conversion_options is an accident and is not guaranteed to continue working (I just dont care about this issue enough to fix the accident).
Thanks.

May I ask what the reasoning for setting both the 'author(s)' and 'publisher' metadata to 'calibre' is (when using ebook-convert to create an ebook from a recipe)? While I can see why calibre is being set as the author, I don't understand why it could be considered the publisher?

In news.py the class BasicNewsRecipe(Recipe) defines this:
Code:
#: Publication type
#: Set to newspaper, magazine or blog
publication_type = 'unknown'
Have you thought about adding some more categories, Eg. newsfeed, newsportal, academic and such like?

Finally am I setting 'smarten-punctuation' correctly below, I seem to be having some problems with it?
Code:
conversion_options = { 'smarten-punctuation' : True }
Thanks.

PS. (Being a noob) I've only just realized that you are the creator of calibre. Excellent, excellent, excellent piece of software - thanks so much for it.
mattst is offline   Reply With Quote
Old 11-08-2011, 11:07 AM   #15
mattst
Enthusiast
mattst began at the beginning.
 
Posts: 27
Karma: 10
Join Date: Nov 2011
Device: Kindle
EDIT: Ok got it now. When running the recipe from within the Calibre GUI, the tags do not get added. END EDIT

Quote:
Originally Posted by Starson17 View Post
Yes, that's how it's attempted. It has no direct affect on the tags in the GUI, which is what most people want. The Adventure Gamers recipe, which I often point people to for a recipe that handles multipage (it's alphabetically the first with that type of code) has that exact form, but does not create tags in the GUI database. It will bring them in if imported in a separate step.
Maybe I am missing the point but when I've created a new mobi ebook using my recipe and ebook-convert, when I add the ebook to Calibre the tags are immediately shown, as the highlighted item in the screenshot linked below shows (I haven't edited the metadata manually).

http://www.zilefile.com/files/16978_...ot_Calibre.png

PS. I'm using Calibre v. 0.7.50 on Ubuntu/Lucid.

Last edited by mattst; 11-09-2011 at 12:46 PM.
mattst is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Recipe for Real-World Economics Review Blog (wordpress) needed marksoc Recipes 4 10-29-2011 04:33 AM
I KNEW that there was a logical explaination for Roswell! ardeegee Lounge 8 05-20-2011 01:26 AM
New recipe request - BBC News Ukrainian storkozos Introduce Yourself 7 10-25-2010 11:36 AM
Recipe for BBC Ukrainian storkozos Recipes 1 10-21-2010 07:01 AM
Recipe help needed for looping through sections of a website Acey Calibre 1 10-16-2008 01:09 PM


All times are GMT -4. The time now is 08:50 PM.


MobileRead.com is a privately owned, operated and funded community.