Recipe request for security blogs

[rherson]

Hi,

I am loving RSS on the Kindle formatted by Calibre. However several of the feeds I would like to read are not working properly & I haven't the foggiest idea how to program in python. The feeds giving me the most problems are:

1) http://www.f-secure.com/weblog/weblog.rdf - only the headlines are coming out, even though the RSS contains the full text of each post

2) http://blog.richi.co.uk/feeds/posts/default - I'm not getting anything from this feed except the author's bio

I appreciate any assistance you can provide on these. Thank you in advance!

[rherson]

Hi,
Just wondering if anyone saw this request for a recipe to download the F-Secure labs blog (http://www.f-secure.com/weblog/weblog.rdf ). When I plug in the RSS URL the only thing that shows up are the headlines for each post. I have the same problem with a few other RSS's & would be happy to take this code & adapt it for the other feeds as well.
The strange thing is that this feed includes full content of the posts so I don't understand why they don't appear on the Calibre version.

Thank you very much in advance for any assistance you can provide.
Rebecca

[louhike]

Here's the code for the weblog of F-Secure :
Edit :

class AdvancedUserRecipe1301860159(BasicNewsRecipe):
    title          = u'F-Secure Weblog'
    description = u'All the news from the weblog of F-Secure'
    publisher = u'F-Secure'
    timefmt = ' [%a, %d %b, %Y]'
    encoding = 'ISO-8859-1'
    oldest_article = 7
    max_articles_per_feed = 100
    no_stylesheets = True
    use_embedded_content   = False
    language = 'en_EN'
    remove_javascript = True
    keep_only_tags = [dict(name='div', attrs={'class':'modSectionTd2'})]
    remove_tags = [dict(name='a'),dict(name='hr')]

    feeds          = [(u'Weblog', u'http://www.f-secure.com/weblog/weblog.rss')]
    def get_cover_url(self):
        return 'http://www.f-secure.com/weblog/archives/images/company_logo.png'

[rherson]

This is fabulous! Thank you so much!

I tried to adapt the recipe for the Sunbelt blog (http://feeds.feedburner.com/SunbeltBlog) and of course it didn't work. Any advice you can provide would be welcome.

Also, I read a lot of RSSs so I like to group them together if possible. Is there an easy way to group this F-Secure recipe that you so kindly created together with the rest of my security RSS's?

Right now the feed for my security RSS's looks like this:

class AdvancedUserRecipe1301899547(BasicNewsRecipe):
title = u'Security RSSs'
oldest_article = 7
max_articles_per_feed = 25

feeds = [(u'MailChannels', u'http://feeds.feedburner.com/MailChannelsBlog?format=xml'), (u'Marshal TRACE News', u'http://www.marshal.com/rss/trace.asp'), (u'Krebs on Security', u'http://feeds.feedburner.com/KrebsOnSecurity'), (u'Securosis', u'http://securosis.com/feeds/blog'), (u'Sunbelt Blog', u'http://feeds.feedburner.com/SunbeltBlog'), (u'TrendLabs', u'http://feeds.trendmicro.com/Anti-MalwareBlog'), (u'Sophos (Naked Security)', u'http://feeds.feedburner.com/nakedsecurity'), (u"Richi's Blog", u'http://blog.richi.co.uk/feeds/posts/default'), (u'F-Secure', u'http://www.f-secure.com/weblog/weblog.rdf'), (u'Matt Cutts Blog', u'http://feeds.mattcutts.com/mattcutts/uJBW?format=xml')]

Can I just add the F-Secure recipe to the end of that & it will all magically go together?

Ideally the Sunbelt recipe will be combined with these too....

Again - this is really really helpful - thank you so much!

[louhike]

No you can't, because of the line

keep_only_tags = [dict(name='div', attrs={'class':'modSectionTd2'})]

which is specific to F-Secure. It will broke the others RSS.
I'll give it a look this evening (I'm in France so it will be in 4 hours).
EDIT : You can add it, it's just it won't removed unnecessary tags.
I'm working on a code with all the RSS which keeps only the needed tags.

[louhike]

I've come to this code :

class AdvancedUserRecipe1301947892(BasicNewsRecipe):
    title          = u'Security RSSs'
    description = u'News from various websites about security'
    timefmt = ' [%a, %d %b, %Y]'
    encoding = 'ISO-8859-1'
    oldest_article = 7
    max_articles_per_feed = 100
    no_stylesheets = True
    use_embedded_content   = False
    language = 'en_EN'
    remove_javascript = True
    keep_only_tags = [dict(name='div', attrs={'class':['post',
                                                       'modSectionTd2',
                                                       'narrowcolumn',
                                                       'content',
                                                       'entry',
                                                       'post',
                                                       'hfeed',
                                                       'entry-content']}),
                        dict(name='div',attrs={'id':['container']})]
    remove_tags = [dict(name='p', attrs={'class':['small',
                                                  'metadata alt',
                                                  'postmetadata alt',
                                                  'insider_continue']}),
                   dict(name='div', attrs={'class':['nocomments',
                                                    'comment-navigation',
                                                    'topsy_widget_data topsy_theme_silver',
                                                    'socialtitlesingle',
                                                    'navigation',
                                                    'greeting',
                                                    'snap_nopreview sharing robots-nocontent',
                                                    'entry-utility',
                                                    'nav-below',
                                                    'related-posts',
                                                    'prev_next post_nav',
                                                    'action_fb',
                                                    'comments_link',
                                                    'title_bar',
                                                    'comment',
                                                    'comments',
                                                    'related-posts'
                                                    'widget-area']}),
                   dict(name='div',attrs={'id':['respond',
                                                'greet_block',
                                                'entry-author-info',
                                                'idc-container-parent',
                                                'idc-loading-comments',
                                                'idc-noscript',
                                                'comments']}),
                   dict(name='span', attrs={'class':['tags']}),
                   dict(name='h3', attrs={'id':['comments']}),
                   dict(name='ol',attrs={'class':['commentlist']}),
                   dict(name='ul',attrs={'class':['term_list']})]

    feeds          = [(u'MailsChannel', u'feeds.feedburner.com/MailChannelsBlog'),
                      (u'F-Secure', u'http://www.f-secure.com/weblog/weblog.rss'),
                      (u'Marshal TRACE News','labs.m86security.com/feed/'),
                      (u'Krebs on security','feeds.feedburner.com/KrebsOnSecurity'),
                      (u'Securosis','securosis.com/feeds/blog'),
                      (u'Sunblet Blog','http://feeds.feedburner.com/SunbeltBlog'),
                      (u'TrendLabs','http://feeds.trendmicro.com/Anti-MalwareBlog'),
                      (u'Sophos (Naked Security)','http://feeds.feedburner.com/nakedsecurity'),
                      (u'Richi\'s Blog','blog.richi.co.uk/feeds/posts/default'),
                      (u'Matt Cutts Blog','http://feeds.mattcutts.com/mattcutts/uJBW?format=xml')]

I might improve it in the future because it takes undesired tags for Sophos. Otherwise, it works quite well.