improved The Economist recipe

[baron]

Thanks so much for this wonderful program.

While the Economist is great, I've been bothered by the fact that the titles don't follow the print or web edition in appearance and the fact that "related links" appeared in the middle of the article. Since I didn't see the need for those to interrupt the flow, I've removed those.

Without further ado, here are the changes I've made.

https://gist.github.com/971587

[warloser2000]

Hi baron. I was bothered by those "related links" as well. As a kindle user, there isn't a way, to my knowledge, of using them. I've read that other e-readers are able to make use of them. That said, thank very much for your recipe. I tried it, I loved it. It make it much more enjoyable to read the Economist. =)

[kaniff]

Hi Baron, can you make a .recipe file for this? All I see is a bunch of code and I have no idea what to do with that, thanks.

[Starson17]

Originally Posted by kaniff

Hi Baron, can you make a .recipe file for this? All I see is a bunch of code and I have no idea what to do with that, thanks.

Start at the "Add a custom news source" dialog screen reached by clicking the black triangle to the right of "Fetch News" on Calibre's main icon toolbar.

Switch to the Advanced Mode.

Copy the code you saw and paste it into the panel on the right (replace everything in that panel with the new code).

Click the button marked "Add/Update recipe" to save it.

The recipe is saved under the title of the recipe that appears on the line "title=" found in each recipe.

You can change the title line to change the name of your recipe. If a custom recipe with that name already exists, you will get an overwrite warning. Don't worry about overwriting builtin recipes - you can't and you can have a custom recipe with the same name as a builtin.

Schedule your new custom recipe normally. It's in the custom area (which won't appear until you have at least one custom recipe).

[dereferenced]

Bump. I followed the directions. When I try to download, I get the "'Could not find any articles. Has your subscription expired?'" exception.

Does this recipie need to be modified? I have a login, but I noticed the browser w/ login info was commented out. I went to the index URL when logged out in my browser
INDEX = 'http://www.economist.com/printedition'
And noticed I got redirected to
http://www.economist.com/printedition/2011-07-30

Could this be breaking the recipie?

[baron]

that's strange. I've gotten the latest issue. I have my login details filled in on the Calibre side and my subscription's actually expired.

the INDEX thing shouldn't be a problem, it just ensures that the latest edition is processed. You might want to double check the credentials you're using within Calibre. HTH

[kovidgoyal]

The economist does not require login to access its articles. The format of the printedition page changed this week. I've already updated the recipe accordingly.

[QueenPepper]

Originally Posted by kovidgoyal

The economist does not require login to access its articles. The format of the printedition page changed this week. I've already updated the recipe accordingly.

That's great. Could you share the updated recipe with us, please?

[QueenPepper]

Oh, never mind, I see that it's in the latest Calibre update! Thanks!

[TOWiz]

[Edit: never mind, I posted before reading today's replies. It seems that there's a fixed version of this recipe built into the latest Calibre. I deleted the custom recipe and am now using the built-in one. Seems to work.]

Thanks, baron, I really like the formatting this recipe gets.

I've run into a bit of difficulty, though: after the first two uses, the download now fails with the message "Could not find any articles. Has your subscription expired?" My Economist subscription is current, and I have full access to the site via my browser after logging in.

I've changed the "needs_subscription" value to true in the recipe, but then the failure message says "The 'The Economist' recipe needs a username and password." Reading the doc, I thought that setting needs_subscription to true means Calibre will ask me for my username and password, but that doesn't happen. It just proceeds with running the recipe and failing because it doesn't have the login info.

I've also tried uncommenting the get_browser method in the recipe and hard-coding my username and password strings in there. But that still failed with the same "needs username/password" message.

I'm not really sure how to solve the problem. Does anyone know why Calibre isn't asking me for my username and password?

Many thanks...

[technicaltitch]

Excuse the noob here, but I can't get either the 0.8.11 version or the baron's version working - I get "Has your subscription expired?" with both since the Economist changed format this week. (Perhaps they have different formats or rules for different countries? I assume an upgrade install will replace the built in recipes with the new versions.)

In case anyone else is getting this issue, and they are as clueless as me with python, soup, etc, I have found that replacing parse_index (and economist_parse_index) with the following works. It is horrendous code and should be deleted asap, and is the stuff of nightmares for Kovid - I've cut every corner I can and barely understand it, but it gets the article text from the new website into my PRS until someone cleverer looks at it.

PHP Code:

    def parse_index(self):
        articles = []
        soup = self.index_to_soup(self.INDEX)
        cover = None
        feeds = []
        for section in soup.findAll('div', attrs={'class':'article'}):
            section_title = 'section title'
            articles = []
            for post in section.findAll('a', href=True):
                url = post['href']
                if url.startswith('/'):
                  url = 'http://www.economist.com'+url
                  title = self.tag_to_string(post)
                  articles.append({'title':title, 'url':url})
            if articles:
                feeds.append((section_title, articles))
        return feeds 


If this doesn't work for some people, perhaps the Economist needs two recipes.

HUGE thanks for oneillpt for this brilliant post:
https://www.mobileread.com/forums/sho...71&postcount=3

And this thread:
https://www.mobileread.com/forums/arc.../t-121643.html

[msmart]

thanks technicaltitch your quick fix worked for the pages but my table of content not displayed correctly (a lot of empty section titles...)

I have the same error message as you... (germany)

[Mlieberman55]

Originally Posted by kovidgoyal

The economist does not require login to access its articles. The format of the printedition page changed this week. I've already updated the recipe accordingly.

Thank you for updating The Economist so quickly. FYI, I am still missing the front "Briefing" section.

[kovidgoyal]

Originally Posted by Mlieberman55

Thank you for updating The Economist so quickly. FYI, I am still missing the front "Briefing" section.

That should be included now. Try it again.

[satisficer]

I updated to 0.8.12 and get the following error -- this is from archlinux pkg updated today -- but I received similar message using command-line on ubuntu using today's binary:

calibre, version 0.8.12
ERROR: Conversion Error: <b>Failed</b>: Fetch news from The Economist (RSS)

Fetch news from The Economist (RSS)
Resolved conversion options
calibre version: 0.8.12

...{OPTIONS HERE}...

InputFormatPlugin: Recipe Input running
Traceback (most recent call last):
  File "/usr/bin/calibre-parallel", line 19, in <module>
    sys.exit(main())
  File "/usr/lib/calibre/calibre/utils/ipc/worker.py", line 181, in main
    result = func(*args, **kwargs)
  File "/usr/lib/calibre/calibre/gui2/convert/gui_conversion.py", line 25, in gui_convert
    plumber.run()
  File "/usr/lib/calibre/calibre/ebooks/conversion/plumber.py", line 937, in run
    accelerators, tdir)
  File "/usr/lib/calibre/calibre/customize/conversion.py", line 204, in __call__
    log, accelerators)
  File "/usr/lib/calibre/calibre/web/feeds/input.py", line 105, in convert
    ro.download()
  File "/usr/lib/calibre/calibre/web/feeds/news.py", line 737, in download
    res = self.build_index()
  File "/usr/lib/calibre/calibre/web/feeds/news.py", line 874, in build_index
    feeds = feeds_from_index(self.parse_index(), oldest_article=self.oldest_article,
  File "/tmp/calibre_0.8.12_tmp_KXHovR/calibre_0.8.12_BkAxYX_recipes/recipe0.py", line 69, in parse_index
    self.feed_dict.items()])
  File "/tmp/calibre_0.8.12_tmp_KXHovR/calibre_0.8.12_BkAxYX_recipes/recipe0.py", line 73, in eco_sort_sections
    raise ValueError('No new articles found')
ValueError: No new articles found

Is anyone seeing this in the updated version?