Custom recipes (archive, read-only)

[carpii]

Originally Posted by kiklop74

Here goes:

Thanks so much for Linux Magazine, kiklop

[DougBarry]

I'm interested in a recipe for the Utne Reader: http://www.utne.com/

Thanks.

[kiklop74]

Originally Posted by DougBarry

I'm interested in a recipe for the Utne Reader: http://www.utne.com/

Thanks.

Here goes:

[ashadocat]

Can comics be done? because everyday decay is almost the perfect size for the reader and a good comic. do I need to do that thing with dosage?

[girlperson1]

http://www.sciencedaily.com/newsfeeds.htm

The RSS feeds are located here: http://www.sciencedaily.com/newsfeed.xml

It's updated six times a day!!!

[kiklop74]

Originally Posted by girlperson1

http://www.sciencedaily.com/newsfeeds.htm

The RSS feeds are located here: http://www.sciencedaily.com/newsfeed.xml

It's updated six times a day!!!

here goes:

[girlperson1]

Originally Posted by kiklop74

here goes:

Thank you!!

[ddavtian]

Hi guys.

I wrote this simple code for Forbes. It works, could use better cleaning of the text. There is even a low resolution cover page :-) But it doesn't create the menu items for articles: Next, Previous, Main, etc.

If somebody can improve this, please post back.

Thanks, David

PHP Code:

class AdvancedUserRecipe1226018223(BasicNewsRecipe):
    title          = u'Forbes'
    oldest_article = 30
    max_articles_per_feed = 100
    no_stylesheets = True
    html2lrf_options = ['--base-font-size', '10']     

    
    remove_tags_before  = dict(id='p_RightTop')
    remove_tags_after  = dict(id='contextualLinks')
    remove_tags = [dict(id=['controlsbox', 'sharePanel']),
           ]
    feeds          = [(u'Latest', u'http://www.forbes.com/news/index.xml'), 
        (u'Most Popular', u'http://www.forbes.com/feeds/popstories.xml'), 
        (u'Most Emailed', u'http://www.forbes.com/feeds/mostemailed.xml'), 
        (u'Faces', u'http://www.forbes.com/facesscan/index.xml'), 
        (u'Technology', u'http://www.forbes.com/technology/index.xml'), 
        (u'Personal Tech', u'http://www.forbes.com/personaltech/index.xml'), 
        (u'Wireless', u'http://www.forbes.com/wireless/index.xml'),
        (u'Business', u'http://www.forbes.com/business/index.xml'), 
        (u'Sports Money', u'http://www.forbes.com/sportsmoney/index.xml'), 
        (u'Sports', u'http://www.forbes.com/forbeslife/sports/index.xml'),
        (u'Vehicles', u'http://www.forbes.com/forbeslife/vehicles/index.xml'),
        (u'Leadership', u'http://www.forbes.com/leadership/index.xml'), 
        (u'Careers', u'http://www.forbes.com/leadership/careers/index.xml'),
        (u'Compensation', u'http://www.forbes.com/leadership/compensation/index.xml'),
        (u'Managing', u'http://www.forbes.com/leadership/managing/index.xml')]


    def get_cover_url(self):
        self.cover_url  = u'http://www.forbes.com/media/current_covers/forbes_120_160.gif'
        cover =  self.cover_url
        return cover 


[kiklop74]

Forbes.com has a bit complicated layout of their pages and in such cases it is better to use print version of the page if it exists and in this case it does so this is modified version of your recipe:

from calibre.ebooks.BeautifulSoup import BeautifulSoup

class AdvancedUserRecipe1226018223(BasicNewsRecipe):
    title          = u'Forbes'
    oldest_article = 30
    max_articles_per_feed = 100
    no_stylesheets = True
    html2lrf_options = ['--base-font-size', '10']     

    cover_url  = u'http://www.forbes.com/media/current_covers/forbes_120_160.gif'
    
    feeds          = [(u'Latest', u'http://www.forbes.com/news/index.xml'), 
        (u'Most Popular', u'http://www.forbes.com/feeds/popstories.xml'), 
        (u'Most Emailed', u'http://www.forbes.com/feeds/mostemailed.xml'), 
        (u'Faces', u'http://www.forbes.com/facesscan/index.xml'), 
        (u'Technology', u'http://www.forbes.com/technology/index.xml'), 
        (u'Personal Tech', u'http://www.forbes.com/personaltech/index.xml'), 
        (u'Wireless', u'http://www.forbes.com/wireless/index.xml'),
        (u'Business', u'http://www.forbes.com/business/index.xml'), 
        (u'Sports Money', u'http://www.forbes.com/sportsmoney/index.xml'), 
        (u'Sports', u'http://www.forbes.com/forbeslife/sports/index.xml'),
        (u'Vehicles', u'http://www.forbes.com/forbeslife/vehicles/index.xml'),
        (u'Leadership', u'http://www.forbes.com/leadership/index.xml'), 
        (u'Careers', u'http://www.forbes.com/leadership/careers/index.xml'),
        (u'Compensation', u'http://www.forbes.com/leadership/compensation/index.xml'),
        (u'Managing', u'http://www.forbes.com/leadership/managing/index.xml')]

    def print_version(self, url):
        raw = self.browser.open(url).read()
        soup = BeautifulSoup(raw.decode('latin1', 'replace'))
        print_link = soup.find('a', {'onclick':"s_linkTrackVars='prop18';s_linkType='o';s_linkName='Print';if(typeof(globalPageName)!='undefined')s_prop18=globalPageName;s_lnk=s_co(this);s_gs(s_account);"})
        if print_link is None:
            return ''
        return 'http://www.forbes.com' + print_link['href']

BTW as you can see if you have a url of cover page just specify it by assigning value to cover_url. No need to override anything.

[ddavtian]

Thank you kiklop! I looked at print page but did not know how to use it.

If you have time, can you add a recipe for Time magazine? RSS feeds are on this page: http://www.time.com/time/rss.

[kiklop74]

Originally Posted by ddavtian

Thank you kiklop! I looked at print page but did not know how to use it.

If you have time, can you add a recipe for Time magazine? RSS feeds are on this page: http://www.time.com/time/rss.

Here goes:

[ddavtian]

Thank you Kiklop, that was fast!

I tried twice, both times got this error:

Job: **Fetch news from Time**
**tuple**: ('IndexError', u'list index out of range')
**Traceback**:
Traceback (most recent call last):
File "parallel.py", line 935, in worker
File "parallel.py", line 893, in work
File "C:\Program Files\calibre\library.zip\calibre\ebooks\epub\from _feeds.py", line 64, in main
File "C:\Program Files\calibre\library.zip\calibre\ebooks\epub\from _feeds.py", line 54, in convert
File "calibre\ebooks\epub\from_html.pyo", line 328, in convert
File "calibre\ebooks\epub\from_html.pyo", line 173, in parse_content
File "calibre\ebooks\epub\from_html.pyo", line 113, in __init__
File "calibre\ebooks\html.pyo", line 532, in __init__
File "calibre\ebooks\html.pyo", line 413, in __init__
File "calibre\ebooks\html.pyo", line 472, in parse_html
File "lxml\html\__init__.pyo", line 114, in body
IndexError: list index out of range


Anytime I get an error (on any feed), after that I cannot fetch the same news again, I click on "download", it's not doing anything. I need to restart Calibre to try the errored out feed again.


EDIT: I changed the output format to lrf and it worked fine. Epub kept erroring out. Thanks again for the recipe.

[kiklop74]

I think you should report this bug so that it can be fixed. I always develop recipes assuming lrf usage since everything looks better in that format.

[DougBarry]

Originally Posted by kiklop74

Here goes:

Thank you.

[Vaseline]

If anyone could do something with the daily mail for me - without judging me - that would be fantastic ;p.

http://www.dailymail.co.uk/home/rssMenu.html

Motoring/Sport/Property/ on there not needed really.