View Single Post
Old 12-28-2010, 01:36 PM   #2
jwiv
Junior Member
jwiv doesn't litterjwiv doesn't litter
 
Posts: 7
Karma: 110
Join Date: Dec 2010
Device: Kindle 3, Nexus 7 (2012)
If you don't mind some of the cruft, you can simply add the baltimoresun feeds straight from the RSS section (http://www.baltimoresun.com/services/rss/) That said, the Sun seems to add a LOT of extraneous crap in that feed, so I used the following to strip things out.

This is _really_ rough/ugly though. I'm mostly hacking it from examples and tial and error, but it seems to be working for my purposes.

If you want to read comments - simply remove the following line:

remove_tags_after = dict(name='div', attrs={'class':'shirttail-promo right clearfix'})

Code:
#!/usr/bin/env  python

__license__            = 'GPL v3'
__copyright__          = '2010, Josh Hall'

from calibre.web.feeds.news import BasicNewsRecipe
class BaltimoreSun(BasicNewsRecipe):
    title          = u'The Baltimore Sun'
    publication_type       = 'newspaper'
    category               = 'news, maryland, baltimore'
    oldest_article = 1
    max_articles_per_feed = 100
    masthead_url = 'http://www.baltimoresun.com/images/logo.png'

    remove_tags_before = dict(name='div', attrs={'class':['story', 'entry']})
    remove_tags_after = dict(name='div', attrs={'class':'shirttail-promo right clearfix'})
    keep_only_tags    = [dict(attrs={'class':['story',  'story-body-text',  'story-body',  'byline',  'articlebody',  'entry',  'entry-body',  'entry-more']})]
                           
    remove_tags      = [   dict(name='div', attrs={'class':['articlerail', 'sphereTools', 'tools', 'toppaginate', 'entry-footer-left', 'entry-footer-right',  'shirttail-promo right clearfix', 'clearfix']}),
                           dict(name='div', attrs={'id':["moduleArticleToolsContainer"]}),
                           dict(name='p', attrs={'class':"entry-footer"}),
                           dict(name='ul', attrs={'class':"article-nav clearfix"}),
                           dict(name=['iframe']),
	                   dict(name='div', attrs={'id':['footer', 'article-promo']}),
	       ]
    feeds          = [
         (u'Top Headlines', u'http://www.baltimoresun.com/rss2.0.xml'),
         (u'Baltimore City', u'http://www.baltimoresun.com/news/maryland/baltimore-city/rss2.0.xml'),
         (u'Baltimore Crime Beat', u'http://weblogs.baltimoresun.com/news/crime/blog/index.xml'),
         (u'Top Maryland', u'http://www.baltimoresun.com/news/maryland/rss2.0.xml'), 
         (u'Education', u'http://www.baltimoresun.com/news/education/rss2.0.xml'), 	
         (u'Local Politics', u'http://www.baltimoresun.com/news/maryland/politics/rss2.0.xml'), 
         (u'Nation/World', u'http://www.baltimoresun.com/news/nation-world/rss2.0.xml'),
         (u'Top Business', u'http://www.baltimoresun.com/business/rss2.0.xml'),
         (u'Top Sports', u'http://www.baltimoresun.com/sports/rss2.0.xml'),
         (u'Weird News', u'http://www.baltimoresun.com/news/offbeat/rss2.0.xml'),
         ]

Last edited by jwiv; 12-28-2010 at 04:46 PM. Reason: Corrected for multiple blog entry formats & hotlinks
jwiv is offline   Reply With Quote