Custom recipes (archive, read-only)

[sark666]

I tried importing the string module using import string but it still didn't work.

I don't know what you mean by did it give an error. I haven't seen any errors throughout any of this. Do you mean launching calibre from the cmd line with a verbose setting? Or is there a built in error messaging that I'm not seeing? Same with the printing, you say you like to " print the results when the recipe runs, then I can see if it's doing what I expect." So you simply insert print statements on the variables? I'll try that.

But it runs further without the string there and using as you suggested. However, now I have just the table of contents of rss feed but each link simply leads to:

"This article was downloaded by calibre from http://www.metronews.ca/toronto/local/article/589013--caribana-brings-the-summer-spirit-to-toronto"

Well that's one specific example, but you see what I mean. I tried inserting a print statement. I simply added print s prior to the return, but I don't see where I'm supposed to see the output.

[Starson17]

Originally Posted by sark666

I don't know what you mean by did it give an error. I haven't seen any errors throughout any of this. Do you mean launching calibre from the cmd line with a verbose setting?

Yes. It's very hard to construct recipes inside of Calibre's GUI.
Read This.

is there a built in error messaging that I'm not seeing? Same with the printing, you say you like to " print the results when the recipe runs, then I can see if it's doing what I expect." So you simply insert print statements on the variables? I'll try that.

I do it this way:

ebook-convert myrecipe.recipe output_dir --test -vv>myrecipe.txt

using as you suggested. However, now I have just the table of contents of rss feed but each link simply leads to:

"This article was downloaded by calibre from http://www.metronews.ca/toronto/local/article/589013--caribana-brings-the-summer-spirit-to-toronto"

It looks like that worked. That shows your recipe created the correct link and Calibre went there. However, when I went there, with scripts blocked in FireFox, I got nothing but photos. Allowing scripts produced the article text. You may need to deal with this issue to get anything.

Try posting your recipe (Inside code and spoiler tags) - I'll look at it.

[Flexicat]

Hello all,

I am certainly enjoying the control that calibre gives me with sorting my books with my new Kobo and have decided to try the customization for newsites and the like. As you might guess, I do not understand Python.

I have started a recipe and so far have gotten:


class AdvancedUserRecipe1280684448(BasicNewsRecipe):
title = u'Twisting the Hellmouth'
oldest_article = 7
max_articles_per_feed = 100

feeds = [(u'Latest', u'http://www.tthfanfic.org/rss.php')]


by following the instructions. This is the RSS feed for the latest stories.

Here I have hit a snag, however. I get the first page of all the articles, but nothing beyond that. The suggestion is to use the print url command, which makes sense.

The articles do have a print address but it is quite different from the original story address.

Story address: http://www.tthfanfic.org/Story-22821...ly+Literal.htm

Print address: http://www.tthfanfic.org/wholestory....1&format=print

I can see that each article has a special ID identifyer, but I cannot figure out how to extract it and put it into the print url command. I can put the bit after the ID number, but don't know how to fix the front text or even get the ID number.

I have tried

def print_version(self, url):
return url + '&format=print'

without success. I have also tried

def print_version(self, url):
return 'http://www.tthfanfic.org/wholestory.php?no=' + artid + '&format=print'

also with out success.

I must be missing some vital step. Can someone here assist me?

Thank you.

[Starson17]

Originally Posted by Flexicat

I have started a recipe and so far have gotten:

Spoiler:

class AdvancedUserRecipe1280684448(BasicNewsRecipe):
    title          = u'Twisting the Hellmouth'
    oldest_article = 7
    max_articles_per_feed = 100

    feeds          = [(u'Latest', u'http://www.tthfanfic.org/rss.php')]

Notice Code tags and Spoiler tags.

I have also tried
def print_version(self, url):
return 'http://www.tthfanfic.org/wholestory.php?no=' + artid + '&format=print'
also with out success.

This should work if you put the code needed to extract artid from the url.

Try this:

    def print_version(self, url):
        split1 = url.split("/")
        xxx = split1[3]
        split2 = xxx.split("-")
        artid =  split2[1]
        print "artid is: ', artid
        return 'http://www.tthfanfic.org/wholestory.php?no=' + artid + '&format=print'

This isn't tested. Please don't some back with a simple "this doesn't work" - you'll need to set the numbers/subscripts if I've miscounted. I'll happily help, but make sure you post the complete recipe in code and spoiler tags.

[Flexicat]

Thank you!

Notice Code tags and Spoiler tags

I shall try to be tidier.

Here is my adjusted code:

Spoiler:

class AdvancedUserRecipe1280684448(BasicNewsRecipe):
    title          = u'Twisting the Hellmouth'
    oldest_article = 7
    max_articles_per_feed = 100

    feeds          = [(u'Latest', u'http://www.tthfanfic.org/rss.php')]

 def print_version(self, url):
        split1 = url.split("/")
        xxx = split1[3]
        split2 = xxx.split("-")
        artid =  split2[1]
        print "artid is: ', artid
        return 'http://www.tthfanfic.org/wholestory.php?no=' + artid + '&format=print'

I get the error message

"ERROR: Invalid input: Could not create recipe. Error: unindent does not match any outer indentation level (recipe8.py, line 11)" when trying to update it.

Am I missing some sort of closing tag?

Also, what are the "numbers/subscripts" you mention? Are they the "/" marks in the URL?

Thank you

[capidamonte]

Has anyone else noticed that the Honolulu Star Advertiser recipe produces double articles -- ie: articles are headed properly but contain the same text repeated twice?

[Starson17]

Originally Posted by Flexicat

Thank you!

You're welcome.

I shall try to be tidier.

I hope I didn't sound too grumpy

Here is my adjusted code:

Spoiler:

class AdvancedUserRecipe1280684448(BasicNewsRecipe):
    title          = u'Twisting the Hellmouth'
    oldest_article = 7
    max_articles_per_feed = 100

    feeds          = [(u'Latest', u'http://www.tthfanfic.org/rss.php')]

 def print_version(self, url):
        split1 = url.split("/")
        xxx = split1[3]
        split2 = xxx.split("-")
        artid =  split2[1]
        print "artid is: ', artid
        return 'http://www.tthfanfic.org/wholestory.php?no=' + artid + '&format=print'

It's almost perfect

I get the error message

"ERROR: Invalid input: Could not create recipe. Error: unindent does not match any outer indentation level (recipe8.py, line 11)" when trying to update it.

You had a space in front of the def print_version. Python is picky about indents. I made a typo that you copied when I pasted in the quotes. Because you pasted in the code tags, (which preserves indents) I was able to easily spot it.

Try this:

Spoiler:

class AdvancedUserRecipe1280684448(BasicNewsRecipe):
    title          = u'Twisting the Hellmouth'
    oldest_article = 7
    max_articles_per_feed = 100

    feeds          = [(u'Latest', u'http://www.tthfanfic.org/rss.php')]

def print_version(self, url):
    split1 = url.split("/")
    xxx = split1[3]
    split2 = xxx.split("-")
    artid =  split2[1]
    print 'artid is: ', artid
    return 'http://www.tthfanfic.org/wholestory.php?no=' + artid + '&format=print'

Also, what are the "numbers/subscripts" you mention? Are they the "/" marks in the URL?

I wasn't sure xxx = split1[3] was right, it might have been xxx = split1[4], but I've now checked them - the recipe ran OK.

[Flexicat]

Thank you!

I hope I didn't sound too grumpy

No you didn't. I appreciated the information. And, obviously, the code tags help fix problems too. Who knew a space could be such a tricky thing?

[Starson17]

Originally Posted by Flexicat

Thank you!

No you didn't. I appreciated the information. And, obviously, the code tags help fix problems too. Who knew a space could be such a tricky thing?

If you're happy with the results, you're done. If not, the next step is to remove junk that you don't want. The easiest way to do that is by identifying the junk using FireFox and FireBug, then putting a
remove_tags = [dict(name='div', attrs={'class':['classname']}),] or whatever into your recipe.

[fuquaforlife]

Originally Posted by Starson17

These are good questions. I've never looked closely enough to answer it. Why don't you test it and tell us all? I'd have to look at the code or test it to answer. I suspect it just compares the date, not the time, but that's just a guess. Personally, I never read every single recipe every day, so I like lots of overlap. I just read articles until I get to the ones I've already read.

Just did a test -- looks like if you put in one day, the recipe picks up the last 24hours worth of articles!

[sark666]

@Starson17

Thanks, if you can look at it, it would be appreciated.

But the strange thing is I just re-ran it since Friday, and now on some I get the article with no image (haven't verified on the site if those actually have an image), some with just an image, and some with just the header alone.

class AdvancedUserRecipe1280289069(BasicNewsRecipe):
    title          = u'Metro Daily'
    oldest_article = 7
    max_articles_per_feed = 100

    feeds          = [(u'metro local', u'http://www.metronews.ca/Toronto/local/rss')]


    def print_version(self, url):
        baseurl='http://www.metronews.ca/ArticlePrint/' 
        split1 = url.split("/")
        xxx=split1 [6]
        split2= xxx[0:5]
        s =  baseurl + split2
        print s
        return s

Here are more feeds if it helps in having more examples when generated:

    feeds          = [
    (u'Local', u'http://www.metronews.ca/Toronto/local/rss'),
    (u'Entertainment', u'http://www.metronews.ca/Toronto/entertainment/rss'),
    (u'World', u'http://www.metronews.ca/Toronto/world/rss'),
    (u'Canada',u'http://www.metronews.ca/Toronto/canada/rss'),
    (u'Business',u'http://www.metronews.ca/Toronto/business/rss')
    ]

I'm going to read the article you linked but anything you can suggest to get consistent results would be appreciated. I'm a little confused by your mention of firefox. I"m not sure if calibre somehow uses the gecko engine in downloading the articles.

[saverio.palmieri]

Hi everyone,

Here goes my first try to write a recipe for Calibre. If anyone wanna give it a try and send me some feedback I will appreciate it very much.

Regards,

Saverio.

[saverio.palmieri]

Originally Posted by capidamonte

Has anyone else noticed that the Honolulu Star Advertiser recipe produces double articles -- ie: articles are headed properly but contain the same text repeated twice?

Fixed recipe attached.

Regards,

Saverio.

[capidamonte]

Thanks Saverio!

[Starson17]

Originally Posted by sark666

I just re-ran it since Friday, and now on some I get the article with no image (haven't verified on the site if those actually have an image), some with just an image, and some with just the header alone.

I have no idea about the images, you'll have to check the source. As to those with just the header, this likely indicates that you aren't correctly calculating the article URL.

I'm a little confused by your mention of firefox. I"m not sure if calibre somehow uses the gecko engine in downloading the articles.

No, FireFox is not used by Calibre's recipes. There are two basic steps in building a good recipe. First, you have to get or calculate the article URLs. From the above, it sounds like you don't have that step done yet. Then you usually want to clean up the results, removing advertising and navigation buttons, while keeping articles and photos. The easiest way to do that is to open the article in FireFox, point to something you want to keep or remove and ask FireBug how to refer to it. Once you know how it's identified, you can tell your recipe to keep it or remove it.