View Single Post
Old 05-18-2012, 01:54 PM   #1
atordo
Connoisseur
atordo is a splendid one to beholdatordo is a splendid one to beholdatordo is a splendid one to beholdatordo is a splendid one to beholdatordo is a splendid one to beholdatordo is a splendid one to beholdatordo is a splendid one to beholdatordo is a splendid one to beholdatordo is a splendid one to beholdatordo is a splendid one to beholdatordo is a splendid one to behold
 
Posts: 89
Karma: 19669
Join Date: Apr 2012
Device: Kindle Touch
Something Awful (The Internet Makes You Stupid)

Spoiler:
Code:
from calibre.web.feeds.news import BasicNewsRecipe

class SomethingAwfulRecipe(BasicNewsRecipe):
    title          = u'Something Awful'
    description = 'The Internet Makes You Stupid'
    cover_url = 'http://i.somethingawful.com/core/head-logo-bluegren.png'
    oldest_article = 7
    max_articles_per_feed = 20
    auto_cleanup = True
    no_stylesheets = True
    language = 'en_US'
    remove_empty_feeds = True

    feeds = [
         ('Photoshop Phriday', 'http://www.somethingawful.com/rss/photoshop-phriday.rss.xml')
        ,('Comedy Goldmine', 'http://www.somethingawful.com/rss/comedy-goldmine.rss.xml')
        #,('The Flash Tub', 'http://www.somethingawful.com/rss/flash-tub.rss.xml')
        ,('Awful Link of the Day', 'http://www.somethingawful.com/rss/awful-links.rss.xml')
        ,('Fake Something Awfuls', 'http://www.somethingawful.com/rss/fake-something-awful.rss.xml')
        ,('The Barbarian\'s Dojo', 'http://www.somethingawful.com/rss/steve-sumner.rss.xml')
        ,('The Great Goon Database', 'http://www.somethingawful.com/rss/great-goon-database.rss.xml')
        ,('Livejournal Theater', 'http://www.somethingawful.com/rss/livejournal-theater.rss.xml')
        ,('Joystick Token Healthpack', 'http://www.somethingawful.com/rss/token-healthpack.rss.xml')
        #,('Webcam Ward', 'http://www.somethingawful.com/rss/webcam-ward.rss.xml')
        ,('Features / Articles', 'http://www.somethingawful.com/rss/feature-articles.rss.xml')
        ,('Guides', 'http://www.somethingawful.com/rss/guides.rss.xml')
        ,('Legal Threats', 'http://www.somethingawful.com/rss/legal-threats.rss.xml')
        ,('Pranks [ICQ]', 'http://www.somethingawful.com/rss/icq-pranks.rss.xml')
        ,('State Og', 'http://www.somethingawful.com/rss/state-og.rss.xml')
        ,('Everquest', 'http://www.somethingawful.com/rss/everquest.rss.xml')
        ,('Pranks [Email]', 'http://www.somethingawful.com/rss/email-pranks.rss.xml')
        ,('The Weekend Web', 'http://www.somethingawful.com/rss/weekend-web.rss.xml')
        ,('Daily Dirt', 'http://www.somethingawful.com/rss/daily-dirt.rss.xml')
        ,('The Art of Warcraft', 'http://www.somethingawful.com/rss/art-of-warcraft.rss.xml')
        ,('Video Game Article', 'http://www.somethingawful.com/rss/video-game-article.rss.xml')
        ,('The Awful Movie Database', 'http://www.somethingawful.com/rss/awful-movie-database.rss.xml')
        ,('Downloads', 'http://www.somethingawful.com/rss/downloads.rss.xml')
        ,('Pregame Wrapup', 'http://www.somethingawful.com/rss/pregame-wrapup.rss.xml')
        ,('Second Life Safari', 'http://www.somethingawful.com/rss/second-life-safari.rss.xml')
        ,('The Hogosphere', 'http://www.somethingawful.com/rss/hogosphere.rss.xml')
        ,('Front Page News', 'http://www.somethingawful.com/rss/news.rss.xml')
        ,('Forum Friday\'s Monday', 'http://www.somethingawful.com/rss/forum-fridays.rss.xml')
        ,('Cliff Yablonski Hates You', 'http://www.somethingawful.com/rss/cliff-yablonski.rss.xml')
        ,('Manifestos From the Internet', 'http://www.somethingawful.com/rss/manifestos-from-internet.rss.xml')
        ,('Johnston Checks In', 'http://www.somethingawful.com/rss/levi-johnston.rss.xml')
        ,('Twitter Tuesday', 'http://www.somethingawful.com/rss/twitter-tuesday.rss.xml')
        ,('Music Article', 'http://www.somethingawful.com/rss/music-article.rss.xml')
        ,('Reviews [Games]', 'http://www.somethingawful.com/rss/game-reviews.rss.xml')
        ,('Reviews [Movies]', 'http://www.somethingawful.com/rss/movie-reviews.rss.xml')
        ,('Rom Pit', 'http://www.somethingawful.com/rss/rom-pit.rss.xml')
        ,('Truth Media [Reviews]', 'http://www.somethingawful.com/rss/truth-media-reviews.rss.xml')
        ,('Truth Media [Flames]', 'http://www.somethingawful.com/rss/truth-media-flames.rss.xml')
        ,('Awful Anime', 'http://www.somethingawful.com/rss/hentai-game-reviews.rss.xml')
        ,('The Horrors of Pornography', 'http://www.somethingawful.com/rss/horrors-of-porn.rss.xml')
        ,('Your Band Sucks', 'http://www.somethingawful.com/rss/your-band-sucks.rss.xml')
        ,('Fashion SWAT', 'http://www.somethingawful.com/rss/fashion-swat.rss.xml')
        #,('AwfulVision', 'http://www.somethingawful.com/rss/awfulvision.rss.xml')
        ,('MMO Roulette', 'http://www.somethingawful.com/rss/mmo-roulette.rss.xml')
        ,('The Most Awful', 'http://www.somethingawful.com/rss/most-awful.rss.xml')
        ,('Garbage Day', 'http://www.somethingawful.com/rss/garbage-day.rss.xml')
        ,('WTF, D&D!?', 'http://www.somethingawful.com/rss/dungeons-and-dragons.rss.xml')
        ,('Current Releases', 'http://www.somethingawful.com/rss/current-movie-reviews.rss.xml')
    ]


Notes:
1. I've commented out sections related with video and flash, you can uncomment them if your device can handle the media.
2. Most sections haven't been updated for years, but remove_empty_feeds takes care of that.
3. I'm still struggling with the multi page articles, found an interesting snippet in the re-usable code thread but wasn't able to make it work. The site just uses something like:
Code:
		<p class="pagebar">
			Pages: <a href="?page=1" class="curpage">1</a> <a href="?page=2">2</a> <a href="?page=3">3</a> <a href="?page=4">4</a> <a href="?page=5">5</a> <a href="?page=6">6</a>  
			<a href="?page=2">Next page</a> &raquo;
		</p>
Any help would be appreciated.
atordo is offline   Reply With Quote