Quote:
Originally Posted by TonytheBookworm
I tried that with no luck....
|
I copied your two typos, but did it work without the typos? It's "top10Pagination" not "top10pagnation"
Quote:
Code:
pager = soup.find('div',attrs={'class':'articleFooter'}) # articleFooter contains the nextpage navigation
print 'the pager soup is: ', pager
if pager:
nexturl = pager.a['href']
|
With the code above, you're going for any link tag in the footer. IMHO, that's way too broad. "pager" should only be the tag that has a link to a next page, not something found on every article. It should never be found on the last page, or on a single page article. If you have to hunt for "Next page" (or whatever text appears on the Next page link) then do that, but unless you're certain that the articleFooter
never has any other links, your code won't work. I suspect your last page has a first page link or a previous page link in the articleFooter. Either would put your recipe into an endless loop.