Custom recipes (archive, read-only)

[OnwardAhead]

Am trying to run a test with the ForeignPolicy.com main feed. Have tried a number of variations, and think that this following snippet would be closest to what I want to get, but no love.

class AdvancedUserRecipe1248523694(BasicNewsRecipe):
    title          = u'Foreign Policy Test'
    oldest_article = 15
    max_articles_per_feed = 100
    keep_only_tags = [dict(name='div', attrs={'id':'art-mast'}),dict(name='div', attrs={'id':'art-body'})]

    feeds          = [(u'Main', u'http://www.foreignpolicy.com/node/feed')]

    def print_version(self, url):
        return url + '?print=yes&hidecomments=yes&page=full'

Not sure if my syntax is off, but I continually get 'IndexError: list index out of range' (see attached logs).

Any thoughts?

[kovidgoyal]

That's a weird set of errors. Off the top of my head I'd say the pages being downloaded are not even HTML

The IndexErrors basically come from finding no html tags in the files

[amerryman]

Thanks Kovid -

Okay just to be sure I re-read the FAQ and didn't see anything there about this maybe you can shed some light. Is there a way to (BESIDES creating a custom feed for each one) adjust the tags that are associated with recipes?

For instance instead of having each one tagged as news AND the title ie: under collections I have 8 news and each one self titled Washington Post, Engadget, etc.. is there a easy way of omitting the Self Title and just tagging it as News for the Collections?

[kovidgoyal]

Not at the moment

[joxxon]

I would really love to be able to read the swedish news paper Dagens Nyheter on my prs-505. Anyone that have cocked this one?

www.dn.se

feeds:

http://www.dn.se/toppnyheter-rss
http://www.dn.se/ekonomi-rss
http://www.dn.se/sport-rss
http://www.dn.se/debatt-rss
http://www.dn.se/ledare-rss
http://www.dn.se/kultur-rss

BTW, Im running a news website (free and add free so this is not a pitch ) that collects news feeds from various news papers etc. Everything from CNN to BBC. Its designed for mobile devices. Feel free to have a look.

http://getnews.mine.nu

[samgler]

Collection of Thai Newspapers.

http://www.norsorpor.com/chooseRSS.php

I browse within each category, news are there. However, I cannot make them to LRFs.

My codes are:

class AdvancedUserRecipe1248726179(BasicNewsRecipe):
    title          = u'NorSorPor'
    oldest_article = 7
    max_articles_per_feed = 100
    encoding              = 'utf_8'
    no_stylesheets        = True
    use_embedded_content  = False
    remove_javascript     = True

    remove_tags     = [dict(name='td', attrs={'align':'right'})]
    remove_tags     = [dict(name='td', attrs={'align':'left'})]

    html2lrf_options = ['--ignore-tables']
    html2epub_options = 'linearize_tables = True'

    feeds          = [
(u'Hot News', u'http://www.norsorpor.com/rss.php?category=1'), 
(u'Business', u'http://www.norsorpor.com/rss.php?category=3'), 
(u'Entertainment', u'http://www.norsorpor.com/rss.php?category=4'), 
(u'Around The World', u'http://www.norsorpor.com/rss.php?category=5'), 
(u'Sports', u'http://www.norsorpor.com/rss.php?category=6'), 
(u'Technology', u'http://www.norsorpor.com/rss.php?category=9'), 
(u'Premiere League', u'http://www.norsorpor.com/rss.php?category=21')
                         ]

Each category will have its own blank page with only the title inside. Am I missing something? TIA

[ssimon2000]

Has anyone developed a recipe for Our Daily Bread? The website is http://www.rbc.org/odb/odb.shtml, and the RSS feed is http://www.rbc.org/rss.ashx?id=50398.
I've tried loading the RSS address into a custom news source, but all I get is a title page...

Thanks!

[cartesio]

Please help me with a recipe for Project Syndicate:
http://www.project-syndicate.org
http://www.project-syndicate.org/about_us/rss

Thanks!

[OnwardAhead]

On the Foreign Policy feeds, very odd. Initially I thought there may have been an issue due to the javascript embedded in the print-friendly URLS. That said, if you try the feed direct, without rewriting the links (as my example did), you still get the same errors.

Has anyone tried this off of the ForeignPolicy Main feed (http://www.foreignpolicy.com/node/feed)?? Does not seem to be a problem with their other feeds. Would really like to get this working so any input is appreciated.

Cheers

[scwehrl]

Seems that after upgrade to 6.0 the first couple of times I downloaded Newsweek it worked fine, now get the attached parse error. Something on my end? Windows Vista 32bit. Thanks for any help.

[kovidgoyal]

Yeah newsweek started embedding some invalid XML content that is causing that error. Will be fixed in the next release.

[dieterpops]

I would like to get a recipe for the Minneapolis Star Tribune. Thanks!

[scwehrl]

Thanks very much Kovid

[Krapmeister]

If anyone with advanced recipe knowledge can do a recipe for http://www.mcsweeneys.net/ I'd appreciate it.

Basic mode gets the content but also pages and pages of links to archive content which isn't in the feed.

The site adds about 10 items every day so a recipe that get's the most recent 30 would suit me, and then I can just schedule it for every 3 days.

Thanks

K

[hackettt]

Kovid and Darko —

I am not skilled in recipes, and I believe I am doing something incorrect. What I wish to do is edit the UK papers' recipes to omit all categories except sport. (I am cricket and football fan living in the States.) However, when I try to edit the recipes, Calibre does not download anything. I first tried this with the Daily Mail to no success.

Is there something I must do besides eliminating the code for other sections I do not want?

This is what I reduced the code to:

from calibre.web.feeds.news import BasicNewsRecipe

class TheDailyMail(BasicNewsRecipe):
title = u'The Daily Mail'
oldest_article = 2
language = _('English')
author = 'RufusA'
simultaneous_downloads= 1
max_articles_per_feed = 50

extra_css = 'h1 {text-align: left;}'

remove_tags = [ dict(name='ul', attrs={'class':'article-icons-links'}) ]
remove_tags_after = dict(name='h3', attrs={'class':'social-links-title'})
remove_tags_before = dict(name='div', attrs={'id':'content'})
no_stylesheets = True

feeds = [
(u'Sport', u'http://www.dailymail.co.uk/sport/index.rss')]

def print_version(self, url):
main = url.partition('?')[0]
return main + '?printingPage=true'

However, nothing occurs when I try to download the information.

Thanks. Cheers.