Thread: web2lrf
View Single Post
Old 12-04-2007, 04:17 PM   #109
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,871
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Hmm another regression was preventing match_regexps from working. Fixed in svn. Note that in your case match regexps should be

match_regexps = ['http://online.barrons.com/.*?html\?mod=.*?|file://.*']

As for the cleanup hanging it seems to be following a long redirect chain

Use the following code to see the HTTP responses being sent by the server

Code:
def cleanup(self):
            try:
                self.browser.set_debug_responses(True)
                import sys, logging
                logger = logging.getLogger("mechanize")
                logger.addHandler(logging.StreamHandler(sys.stdout))
                logger.setLevel(logging.INFO)

                res = self.browser.open('http://online.barrons.com/logout')
            except:
                import traceback
                traceback.print_exc()
You may find the documentation at http://wwwsearch.sourceforge.net/mechanize/ useful for understanding how the browser object works.
kovidgoyal is offline   Reply With Quote