tried to load this recently and it does't work anymore.. some small fixes.
cover method change
Code:
def get_cover_url(self):
soup = self.index_to_soup(
'https://frontline.thehindu.com/current-issue/')
tag = soup.find(attrs={'class': 'sptar-image'})
if tag:
self.cover_url = tag.find('img')['data-original']
return super().get_cover_url()
add '
article-container' to keep tags classes
change preprocess images
Code:
def preprocess_html(self, soup):
for source in soup.findAll('source', srcset=True, attrs={'media':'(min-width: 1600px)'}):
source.name = 'img'
source['src'] = source['srcset']
for img in soup.findAll('img', attrs={'data-original':True}):
img['src'] = img['data-original']
return soup