View Single Post
Old 06-08-2021, 12:56 AM   #7
hongho71
Connoisseur
hongho71 began at the beginning.
 
Posts: 55
Karma: 16
Join Date: Feb 2015
Device: Kindle Paperwhite
Hello, for those still interested in the FT Rss Feed. I couldn't get the entire FT newspaper to download but FT has a custom Rss Feed section in personal settings. There you can add your own articles (I added FT Opinions in my case) and you get the FT RSS link. Just copy the recipe below and replace it with your own FT link. You need valid username and password. Hope this helps.

Copy Recipe Below
---------------------------------------------

Code:
#!/usr/bin/env python2
# vim:fileencoding=utf-8
from __future__ import unicode_literals, division, absolute_import, print_function
from calibre.web.feeds.news import BasicNewsRecipe

class AdvancedUserRecipe1512304616(BasicNewsRecipe):
    title          = 'FT - Opinions'
    oldest_article = 60
    max_articles_per_feed = 100
    needs_subscription = True
    no_stylesheets = True
    remove_javascript = True
    use_embedded_content = False
    auto_cleanup   = True

       
    LOGIN = 'https://accounts.ft.com/login'
    LOGOUT = 'https://myaccount.ft.com/logout'
    INDEX = 'https://www.ft.com/todaysnewspaper/uk'
    PREFIX = 'https://www.ft.com'

    def get_browser(self):
        USER_AGENT = 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)'
        #USER_AGENT = 'Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0'
        br = BasicNewsRecipe.get_browser(self, user_agent=USER_AGENT)
        #br.set_debug_http(True)
        #br.set_debug_responses(True)
        br.open(self.INDEX)
        if self.username is not None and self.password is not None:
            br.open(self.LOGIN)
            br.select_form(id='email-form')
            br['email'] = self.username
            br.submit()
#            br.select_form(name='enter-password-form')
            br.select_form(id='login-form')
            br['password'] = self.password
            br.submit()
        br.addheaders += [('Referer', 'https://www.google.com')]
        br.addheaders += [('Cookie', 'ft-access-decision-policy=PRIVILEGED_REFERER_POLICY;')]
        br.addheaders += [("X-Forwarded-For", "66.249.66.1")]
        return br

#    remove_tags_before = dict(id='story')
 #   remove_tags_after = dict(id='story')

    remove_tags = [
        dict(attrs={'aria-label':'tools'.split()}),
        dict(attrs={'aria-label': lambda x: x and 'New York Times Logo' in x}),
        dict(href='#site-content #site-index'.split()),
        dict(attrs={'aria-hidden':'true'}),
        dict(attrs={'data-videoid':True}),
        dict(name='button meta link'.split()),
        dict(id=lambda x: x and x.startswith('story-ad-')),
        dict(name='head'),
        dict(role='toolbar'),
        dict(name='a', href=lambda x: x and '#story-continues-' in x),
        dict(name='a', href=lambda x: x and '#whats-next' in x),
        dict(id=lambda x: x and 'sharetools-' in x),
        dict(id='newsletter-promo supported-by-ad bottom-wrapper'.split()),
        dict(attrs={'class': lambda x: x and (
            'SectionBar' in x or 'recirculation' in x or 'ResponsiveAd' in x or 'accessibility-visuallyHidden' in x or 'RelatedCoverage' in x)}),
    ]    
    
    feeds          = [
        ('https://www.ft.com/myft/following/ced74d18-5498-4cca-9640-34bf65b1f431.rss'),
    ]

Last edited by PeterT; 06-08-2021 at 07:02 AM.
hongho71 is offline   Reply With Quote