Financial Times Recipe Not found in calibre anymore (www.ft.com)

[mixedbag]

Hello. I have come back to using Calibre again after a long time. Earlier there used to be a recipe for Financial times by default. Now there seems to be no recipe.

Is there a way i can find the recipe?

Thanks.

[kovidgoyal]

It was removed since that website was redesigned and no one was interested enough to adapt the recipe for the new design.

[hongho71]

I've asked kiklop74 to look at it again (gave credentials) since it appears the captcha login requirements maybe turned off based on another post and my own experience. There is a custom MyFT rss feed link in the settings and accounts, change preferences, rss feeds that you can copy and paste. If not able to grab the entire paper, it would be nice to at least grab a custom MyFT feed. For example, my custom MyFT feed is
https://www.ft.com/myft/following/ce...bf65b1f431.rss

Now, just have to have a recipe that logs into FT and be able to grab the feed if not the paper.

[bobbysteel]

I rewrote it at some point but it's still partly failing. I'll share when I'm at a pc next


Sent from my iPhone using Tapatalk

[bobbysteel]

this one is semi-working but haven't had time to sort it out
https://gist.github.com/bobbysteel/9...5ed5a05b112def

[MrDrDr]

Originally Posted by bobbysteel

this one is semi-working but haven't had time to sort it out
https://gist.github.com/bobbysteel/9...5ed5a05b112def

Thanks for posting this - am interested in getting this to work. I can't get any content to download - do you know if it's even possible with reCAPTCHA check?

[hongho71]

Hello, for those still interested in the FT Rss Feed. I couldn't get the entire FT newspaper to download but FT has a custom Rss Feed section in personal settings. There you can add your own articles (I added FT Opinions in my case) and you get the FT RSS link. Just copy the recipe below and replace it with your own FT link. You need valid username and password. Hope this helps.

Copy Recipe Below
---------------------------------------------

#!/usr/bin/env python2
# vim:fileencoding=utf-8
from __future__ import unicode_literals, division, absolute_import, print_function
from calibre.web.feeds.news import BasicNewsRecipe

class AdvancedUserRecipe1512304616(BasicNewsRecipe):
    title          = 'FT - Opinions'
    oldest_article = 60
    max_articles_per_feed = 100
    needs_subscription = True
    no_stylesheets = True
    remove_javascript = True
    use_embedded_content = False
    auto_cleanup   = True

       
    LOGIN = 'https://accounts.ft.com/login'
    LOGOUT = 'https://myaccount.ft.com/logout'
    INDEX = 'https://www.ft.com/todaysnewspaper/uk'
    PREFIX = 'https://www.ft.com'

    def get_browser(self):
        USER_AGENT = 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)'
        #USER_AGENT = 'Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0'
        br = BasicNewsRecipe.get_browser(self, user_agent=USER_AGENT)
        #br.set_debug_http(True)
        #br.set_debug_responses(True)
        br.open(self.INDEX)
        if self.username is not None and self.password is not None:
            br.open(self.LOGIN)
            br.select_form(id='email-form')
            br['email'] = self.username
            br.submit()
#            br.select_form(name='enter-password-form')
            br.select_form(id='login-form')
            br['password'] = self.password
            br.submit()
        br.addheaders += [('Referer', 'https://www.google.com')]
        br.addheaders += [('Cookie', 'ft-access-decision-policy=PRIVILEGED_REFERER_POLICY;')]
        br.addheaders += [("X-Forwarded-For", "66.249.66.1")]
        return br

#    remove_tags_before = dict(id='story')
 #   remove_tags_after = dict(id='story')

    remove_tags = [
        dict(attrs={'aria-label':'tools'.split()}),
        dict(attrs={'aria-label': lambda x: x and 'New York Times Logo' in x}),
        dict(href='#site-content #site-index'.split()),
        dict(attrs={'aria-hidden':'true'}),
        dict(attrs={'data-videoid':True}),
        dict(name='button meta link'.split()),
        dict(id=lambda x: x and x.startswith('story-ad-')),
        dict(name='head'),
        dict(role='toolbar'),
        dict(name='a', href=lambda x: x and '#story-continues-' in x),
        dict(name='a', href=lambda x: x and '#whats-next' in x),
        dict(id=lambda x: x and 'sharetools-' in x),
        dict(id='newsletter-promo supported-by-ad bottom-wrapper'.split()),
        dict(attrs={'class': lambda x: x and (
            'SectionBar' in x or 'recirculation' in x or 'ResponsiveAd' in x or 'accessibility-visuallyHidden' in x or 'RelatedCoverage' in x)}),
    ]    
    
    feeds          = [
        ('https://www.ft.com/myft/following/ced74d18-5498-4cca-9640-34bf65b1f431.rss'),
    ]

[eb00kreader]

Hi hongho71, can you please advise if this recipe works for you? I've tried chaning to my own RSS feed, but i'm not prompted for a username/password and it just fails. I understand that these need to be provided, but i'm not sure how.
As you've stated, the Calibre documentation says that if needs_subscription is set to True, which it is, then I'm supposed to be asked for a username and password. However, I am not.
Thanks

[hongho71]

I just checked the FT recipe and it works fine. You have to put the username and password in Calibre, Fetch News, FT Opinions (Name of my Recipe), under Account (required).

Make sure you use your own myFT RSS Feed in the recipe. Each user FT feed will be different.