Update Indian express

[unkn0wn]

fixing some tags and removing unnecessary banners

https://github.com/kovidgoyal/calibr...00db42e010677d

remove_attributes = ['style','height','width']
    ignore_duplicate_articles = {'url'}
    
    keep_only_tags = [
        classes('heading-part full-details')
    ]
    remove_tags = [
        dict(name='nav', attrs={'class':'ie-breadcrumb'}),
        dict(name='div', attrs={'id':'ie_story_comments'}),
        dict(name='div', attrs={'class':['ie-int-campign-ad','custom_read_button','unitimg','copyright']}),
        dict(name='img', attrs={'src':'https://images.indianexpress.com/2021/06/explained-button-300-ie.jpeg'}),
        dict(name='a', attrs={'href':'https://indianexpress.com/section/explained/?utm_source=newbanner'}),
        dict(name='img', attrs={'src':'https://images.indianexpress.com/2021/06/opinion-button-300-ie.jpeg'}),
        dict(name='a', attrs={'href':'https://indianexpress.com/section/opinion/?utm_source=newbanner'}),
        classes('share-social appstext storytags pdsc-related-modify news-guard'),

[kovidgoyal]

https://github.com/kovidgoyal/calibr...1f2022d995c99f

[unkn0wn]

masthead_url = 'https://indianexpress.com/wp-content/themes/indianexpress/images/indian-express-logo-n.svg'

extra_css = '#storycenterbyline {font-size:small};'

def get_cover_url(self):
        soup = self.index_to_soup('https://www.magzter.com/IN/The-Indian-Express-Ltd./The-Indian-Express-Mumbai/Newspaper/')
        for citem in soup.findAll('meta', content=lambda s: s and s.endswith('view/3.jpg')):
            return citem['content']

remove_tags = [
classes('share-social appstext storytags ie-int-campign-ad ie-breadcrumb custom_read_button unitimg copyright pdsc-related-modify news-guard')

bold parts to be added. (story-tags to be replaced with storytags)

more feeds

('Research', 'https://indianexpress.com/section/research/feed/'),
('UPSC-CSE Key','https://indianexpress.com/section/upsc-current-affairs/feed/'),
('World','https://indianexpress.com/section/world/feed/'),
('Business', 'https://indianexpress.com/section/business/feed/'),

[unkn0wn]

also cover url for hindustan times

https://github.com/kovidgoyal/calibr...n_times.recipe

found that its much easier to get covers from magzter.

def get_cover_url(self):
        soup = self.index_to_soup('https://www.magzter.com/IN/HT-Digital-Streams-Ltd./Hindustan-Times-Delhi/Newspaper/')
        for citem in soup.findAll('meta', content=lambda s: s and s.endswith('view/3.jpg')):
            return citem['content']

We can get so many daily updated cover urls from here with same code and changing links.

[unkn0wn]

India today update https://github.com/kovidgoyal/calibr...a_today.recipe

extra_css = '[itemprop^="description"] {font-size: small; font-style: italic;}'
    
    def get_cover_url(self):
        soup = self.index_to_soup('https://www.magzter.com/IN/India-Today-Group/India-Today/News/')
        for citem in soup.findAll('meta', content=lambda s: s and s.endswith('view/3.jpg')):
            return citem['content']

we cant get this cover from default website


THE WEEK India

https://github.com/kovidgoyal/calibr...he_week.recipe

Cover url and other updates..

def get_cover_url(self):
        soup = self.index_to_soup('https://www.magzter.com/IN/Malayala_Manorama/THE_WEEK/Business/')
        for citem in soup.findAll('meta', content=lambda s: s and s.endswith('view/3.jpg')):
            return citem['content']

the quality of the cover url within the present recipe is very low.

remove all from line 36-57(end) ( present recipe won't load images within text of the article) (images are within src tag)
add below

keep_only_tags = [
        dict(name='h1'),
		dict(name='div', attrs={'class':['article-title','article-image','articlecontentbody section']}),
        ]
        
    remove_tags = [
        dict(name='div', attrs={'class':'highlights section'}),
        ]

Financial Express

cover url

def get_cover_url(self):
        soup = self.index_to_soup('https://www.magzter.com/IN/The-Indian-Express-Ltd./Financial-Express-Mumbai/Business/')
        for citem in soup.findAll('meta', content=lambda s: s and s.endswith('view/3.jpg')):
            return citem['content']

[unkn0wn]

remove_tags = [dict(name='div', attrs={'class': 'success_screen poll_withoutLogin hide'})]

def get_cover_url(self):
        soup = self.index_to_soup('https://www.magzter.com/IN/Bennett-Coleman-and-Company-Limited/The-Times-of-India-Delhi/Newspaper/')
        for citem in soup.findAll('meta', content=lambda s: s and s.endswith('view/3.jpg')):
            return citem['content']

i didnt know we had working recipe for ToI and it ranks 60 in usage https://calibre-ebook.com/dynamic/recipe-usage (Are these stats up-to-date?)

LiveMint

why not use same img everyday for livemint.

masthead_url = 'https://images.livemint.com/static/livemint-logo-v1.svg'

cover_url = 'https://epsfs.hindustantimes.com/MINT/2022/04/06/Delhi/Delhi/5_01/9376f23b_01_mr.jpg'

[kovidgoyal]

Yes, the stats are up-to-date

[unkn0wn]

def get_cover_url(self):
        soup = self.index_to_soup('https://www.magzter.com/IN/Bennett-Coleman-and-Company-Limited/The-Times-of-India-Delhi/Newspaper/')
        for citem in soup.findAll('meta', content=lambda s: s and s.endswith('view/3.jpg')):
            return citem['content']

uhh.. you missed to add this cover url part to TOI.

its just that fetching the daily front page as cover makes it much more interesting.. sorry i kept asking you to make so many changes..

[kovidgoyal]

If you send pull requests on github it will be easier to ensure I dont miss anything.

[unkn0wn]

(this new div is adding too much unnecessary stuff to all the articles)
add premium-story to remove_tags classes

new feed
('Political Pulse', 'https://indianexpress.com/section/india/political-pulse/feed/'),
('India', 'https://indianexpress.com/section/india/feed/'),

[unkn0wn]

similar problem with financial express
https://github.com/kovidgoyal/calibr...e_india.recipe

add remove_tags = [classes('parent_also_read')]

[bheeshmpita]

Originally Posted by unkn0wn

(this new div is adding too much unnecessary stuff to all the articles)
add premium-story to remove_tags classes
,

how to get rid of 'best of Express Premium' appearing repeatedly in indian express fetched news?

[unkn0wn]

the changes were already made.. load from default IE recipe.

[bheeshmpita]

Originally Posted by unkn0wn

the changes were already made.. load from default IE recipe.

oh ok, is there any way my custom recipe gets updated with changes with the change in the base recipe?

[unkn0wn]

no.. just use the default recipe