View Single Post
Old 09-11-2010, 10:24 PM   #2694
cynvision
Member
cynvision began at the beginning.
 
Posts: 14
Karma: 10
Join Date: Sep 2010
Device: nook
Okay. I've joined the club using .bat files to run the command line so all things are good there. But! I'm working on a little thing that grabs the update from the National Hurricane Center and I thought it was so simple I'm working on it in Calibre's window. I know from the thread here that there's the complicated way of using parse_index or get_links to filter a set of links to follow, but I really wanted to filter by the built-in methods and I must be doing them all wrong because they don't work on my computer.

I just want it to ignore the links ending in '.zip' and '.kmz' Where am I going wrong with raw reg expressions? I've tried with and without "\", with and without "r", and I'm going buggy. (Now I know the text link says .shp, but they're .zip packages that are linked. Right now they or the kmz are being read into the e-book as text.)

Spoiler:
Code:
import re
class NationalHurricaneCenter(BasicNewsRecipe):
    title          = u'National Hurricane Center (Atlantic)'
    oldest_article = 1
    max_articles_per_feed = 15

    feeds          = [(u'Atlantic Basin Tropical Advisories', u'http://www.nhc.noaa.gov/index-at.xml'),
                          (u'Flight Plan Of The Day', u'http://www.nhc.noaa.gov/xml/REPRPD.xml')]
    no_stylesheets = True
    remove_javascript     = True
    use_embedded_content  = False
    remove_attributes  = ['width','height']
    encoding = 'utf-8'
    masthead_url          = 'http://www.nhc.noaa.gov/gifs/xml_logo_nhc.gif'

    conversion_options = {
                   'linearize_tables' : True,
                    }
#this needs work, it's not avoiding the url's .zip or .kmz
#    filter_regexps =  [r'\.kmz'] 
#    preprocess_regexps     = [(re.compile(r'\.kmz', re.DOTALL), lambda m: '')]
    match_regexps = [r'\.shtml']


#    remove_tags_before  = dict(name='h2')
#    remove_tags_after  = dict(name='pre')

    keep_only_tags = [dict(name='h2'), dict(name='pre')]

#    remove_tags = [
#                     dict(name='div' , attrs={'class':'topbanner_780' }),
#                     dict(name='div' , attrs={'class':'navbkgrnd' }),
#                    ]
#http://www.nhc.noaa.gov/text/WTUS84-KBRO.shtml
#http://www.nhc.noaa.gov/text/WTUS84-KBRO.shtml?text

#http://www.nhc.noaa.gov/text/refresh/SJUTCPAT1+shtml/111458.shtml
#http://www.nhc.noaa.gov/text/refresh/SJUTCPAT1+shtml/111458.shtml?text
    def print_version(self, url):
                        return url + '?text'
cynvision is offline