Custom recipes (archive, read-only)

[Starson17]

Originally Posted by nook.life

Starson17, you are the best. The recipe works perfectly now.

Nook.life, I don't know if you are still here, but the Explosm feed occasionally has non-comic articles that point to a forum. I recently realized how to filter those out. All those articles have "showthread" in the link and they can be filtered out with this (untested) code:

    def print_version(self, url):
        match = re.search(r'showthread', url)
        if not match:
            return url

edit: I tested the code - it works great.

[iLeaveYou]

I was searching the entire thread from the top to the bottom.
I didn't find any recipe for a Romanian News Website.
The easiest one I was trying is: http://www.evz.ro/rss.xml
I am able to get the feeds but I can't get rid of some extra, unwanted content.
Could somebody give me a hand on this?
Thank you.

[smargo]

Instapaper script stopped working!

It stops on 34% and hangs. Please help, what's wrong?

Here is the output:

Downloaded article: Умная Россия | ИноСМИ - Все, что достойно перевода from http://www.inosmi.ru/social/20100517/159972702.html
http://www.instapaper.com/text?u=http%3A//www.echo.msk.ru/blog/nzlobin/679236-echo/ saved to c:\docume~1\margose\locals~1\temp\calibre_0.7.7_w4tvnj_plumber\feed_0\article_98\index.xhtml
Downloaded article: Радиостанция "Эхо Москвы" / Блоги / Николай Злобин, политолог / Четверть Путина / Комментарии from http://www.echo.msk.ru/blog/nzlobin/679236-echo/
Processing images...
Fetching http://beta.inosmi.ru/images/15997/75/159977595.gif
Fetching http://beta.inosmi.ru/images/15934/92/159349243.jpg
Fetching http://beta.inosmi.ru/images/15934/92/159349245.jpg
Recursion limit reached. Skipping links in http://www.instapaper.com/text?u=http%3A//inosmi.ru/army/20100517/159977679.html
http://www.instapaper.com/text?u=http%3A//inosmi.ru/army/20100517/159977679.html saved to c:\docume~1\margose\locals~1\temp\calibre_0.7.7_w4tvnj_plumber\feed_0\article_97\index.xhtml
Downloaded article: Провалившаяся военная реформа, модернизация и российская внешняя политика | ИноСМИ - Все, что достойно перевода from http://inosmi.ru/army/20100517/159977679.html
Processing images...
Fetching http://graphics8.nytimes.com/images/2010/05/09/magazine/09babies-span/09babies-t_CA0-articleLarge.jpg
Fetching http://www.nytimes.com/adx/bin/clientside/420f45a2Q2FQ3AQ7EQ60Q7CSQ3AuAQ23S@Q7BAz6Q7EuSQ2Al6t8Q2AgQ5B8Q5EQ2A9dQ23Q7EQ2A98tgQ5B
Fetching http://up.nytimes.com/?d=0/15/&t=2&s=0&ui=0&r=&u=www%2enytimes%2ecom%2f2010%2f05%2f09%2fmagazine%2f09babies%2dt%2ehtml%3f%5fr%3d1%26pagewanted%3dall
Recursion limit reached. Skipping links in http://www.instapaper.com/text?u=http%3A//www.nytimes.com/2010/05/09/magazine/09babies-t.html%3Fpartner%3Drss%26emc%3Drss%26pagewanted%3Dprint
http://www.instapaper.com/text?u=http%3A//www.nytimes.com/2010/05/09/magazine/09babies-t.html%3Fpartner%3Drss%26emc%3Drss%26pagewanted%3Dprint saved to c:\docume~1\margose\locals~1\temp\calibre_0.7.7_w4tvnj_plumber\feed_1\article_0\index.xhtml
Downloaded article: The Moral Life of Babies from http://www.nytimes.com/2010/05/09/magazine/09babies-t.html?partner=rss&emc=rss&pagewanted=print
Processing images...
Fetching http://upload.wikimedia.org/wikipedia/commons/thumb/a/ac/Amos_Oz_by_Kubik.JPG/200px-Amos_Oz_by_Kubik.JPG
Fetching http://upload.wikimedia.org/wikipedia/commons/thumb/4/42/Nily_Oz_and_Amos_Oz_in_New_York_City_2008.jpg/220px-Nily_Oz_and_Amos_Oz_in_New_York_City_2008.jpg
Recursion limit reached. Skipping links in http://www.instapaper.com/text?u=http%3A//en.wikipedia.org/wiki/Amos_Oz
http://www.instapaper.com/text?u=http%3A//en.wikipedia.org/wiki/Amos_Oz saved to c:\docume~1\margose\locals~1\temp\calibre_0.7.7_w4tvnj_plumber\feed_1\article_1\index.xhtml
Downloaded article: Amos Oz - Wikipedia, the free encyclopedia from http://en.wikipedia.org/wiki/Amos_Oz

[kiklop74]

New recipe for evz.ro:

[iLeaveYou]

Originally Posted by kiklop74

New recipe for evz.ro:

Awesome!
Thank you very much!!!

Would this one be more dificult?
http://www.realitatea.net/rss.html

[jasonfedelem]

Kiklop,

I have been unsuccesful in creating recipes for the two papers closest to me. If you get a chance can you look at them?

Austin American Statesman: http://www.statesman.com/
Waco Tribune: http://www.wacotrib.com/

[schnortz]

Originally Posted by Starson17

I thought of "another way."

    def print_version(self, url):
        match = re.search(r'PhotoGallery', url)
        if not match:
            return url

This did the trick for the Photo based links. Thanks a lot.

[Starson17]

Originally Posted by schnortz

This did the trick for the Photo based links. Thanks a lot.

You're welcome. I found a couple of other places to use it myself.

[kiklop74]

I do recipes when I have time and right mood, so there are no guarantees whatsoever on if I will do any of the requested recipes in this thread.

[einstuerzende]

rty,

I've been fumbling around with making a recipe for cn.wsj.com without an awful lot of success. If you have time and are taking any requests, I'd appreciate whatever help you could give. I'm trying to get the Traditional character edition, which I think means throwing "big5" in front of everything (ex: http://cn.wsj.com/big5/20100708/FRX003561.asp)

On another note, I made a couple of modifications to the BBC Chinese recipe to make it pull the Traditional version, otherwise it's pretty much as rty built it; sharing in case it's of use to anyone:

Spoiler:

class AdvancedUserRecipe1277443634(BasicNewsRecipe):
title = u'BBC 中文網'
oldest_article = 7
max_articles_per_feed = 100

feeds = [
(u'\u4e3b\u9801', u'http://www.bbc.co.uk/zhongwen/trad/index.xml'),
(u'\u570B\u969B\u65b0\u805e', u'http://www.bbc.co.uk/zhongwen/trad/world/index.xml'),
(u'\u5169\u5CB8\u4E09\u5730', u'http://www.bbc.co.uk/zhongwen/trad/china/index.xml'),
(u'\u91D1\u878D\u8CA1\u7D93', u'http://www.bbc.co.uk/zhongwen/trad/business/index.xml'),
(u'\u7DB2\u4E0A\u4E92\u52D5', u'http://www.bbc.co.uk/zhongwen/trad/interactive/index.xml'),
(u'\u97F3\u8996\u5716\u7247', u'http://www.bbc.co.uk/zhongwen/trad/multimedia/index.xml'),
(u'\u5206\u6790\u8A55\u8AD6', u'http://www.bbc.co.uk/zhongwen/trad/indepth/index.xml')
]
extra_css = '''
@font-face {font-family: "DroidFont", serif, sans-serif; src: url(res:///system/fonts/DroidSansFallback.ttf); }\n
body {margin-right: 8pt; font-family: 'DroidFont', serif;}\n
h1 {font-family: 'DroidFont', serif;}\n
.articledescription {font-family: 'DroidFont', serif;}
'''
__author__ = 'rty'
__version__ = '1.0'
language = 'zh-HANT'
pubisher = 'British Broadcasting Corporation'
description = 'BBC news in Chinese'
category = 'News, Chinese'
remove_javascript = True
use_embedded_content = False
no_stylesheets = True
encoding = 'UTF-8'
conversion_options = {'linearize_tables':True}
masthead_url = 'http://wscdn.bbc.co.uk/zhongwen/trad/images/1024/brand.jpg'
keep_only_tags = [
dict(name='h1'),
dict(name='p', attrs={'class':['primary-topic','summary']}),
dict(name='div', attrs={'class':['bodytext','datestamp']}),
]

[rty]

Originally Posted by einstuerzende

rty,

I've been fumbling around with making a recipe for cn.wsj.com without an awful lot of success. If you have time and are taking any requests, I'd appreciate whatever help you could give. I'm trying to get the Traditional character edition, which I think means throwing "big5" in front of everything (ex: http://cn.wsj.com/big5/20100708/FRX003561.asp)

Either I am getting senile today, but I think cn.wsj.com seems to have some kind of protection in place to prevent Calibre basic scraping method from the RSS page. We'll have to wait for experts to help.

[rty]

Originally Posted by jasonfedelem

Kiklop,

I have been unsuccesful in creating recipes for the two papers closest to me. If you get a chance can you look at them?

Austin American Statesman: http://www.statesman.com/
Waco Tribune: http://www.wacotrib.com/

Why create again for Stateman? It's already incorporated into the Calibre under "Fetch News"

[iLeaveYou]

Originally Posted by iLeaveYou

Awesome!
Thank you very much!!!

Would this one be more dificult?
http://www.realitatea.net/rss.html

Originally Posted by kiklop74

I do recipes when I have time and right mood, so there are no guarantees whatsoever on if I will do any of the requested recipes in this thread.

@kiklop74
Sorry! I didn't wanted to offend you.
I was just thinking somebody could help me as I am a noob in this and as I mentioned before couldn't find any Romanian Recipes.

[bhandarisaurabh]

CAN SOMEONE MAKE RECIPE FOR WHARTON INDIA@ KNOWLEDGE
http://knowledge.wharton.upenn.edu/india/rss/

AND FINANCIAL EXPRESS PRINT EDITION WITHOUT USING FEEDS AND USING THE LINK
http://www.financialexpress.com/print/

[jasonfedelem]

Originally Posted by rty

Why create again for Stateman? It's already incorporated into the Calibre under "Fetch News"

That's weird. I'm looking under Fetch News under both "Austin" and "The Austin" but it doesn't show up...

Is there some extended lib of recipes? I'm running v0.7.7. and have 237 recipes listed under "English"