MobileRead Forums
Register Guidelines E-Books Today's Posts Search

Go Back   MobileRead Forums > E-Book Software > Calibre > Recipes
Reload this Page Discover Magazine recipe no longer working

Notices

Reply
 
Thread Tools Search this Thread
Old 02-10-2013, 05:33 PM   #1
jumafl
Enthusiast
jumafl began at the beginning.
 
Posts: 32
Karma: 10
Join Date: Apr 2012
Device: Amazon Kindle Paperwhite
Discover Magazine recipe no longer working
The standard Discover Magazine recipe that's delivered with Calibre no longer works. The output mobi file contains a bunch of "failed feed" messages. I believe the problem started when the site's RSS feeds moved to FeedBurner.

Fortunately I found that I'm able get most of what I want by creating a basic news recipe. The basic recipe was working fine until today when I noticed one news article was incomplete. When I checked the website I saw the following: "This article is only available to registered users of DISCOVERMagazine.com. Register now for FREE and get instant access!"

The FeedBurner page is: http://feeds.feedburner.com/20ThingsYouDidntKnowAbout

The story on the page is called: 20 Things You Didn't Know About... Hibernation

The direct link to the story is: http://discovermagazine.com/2013/mar...r#.URgbcWdgWMc
jumafl is offline   Reply With Quote
Old 02-11-2013, 06:13 AM   #2
Krittika Goyal
Vox calibre
Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.
 
Krittika Goyal's Avatar
 
Posts: 412
Karma: 1175230
Join Date: Jan 2009
Device: Sony reader prs700, kobo
we will need login info to fix this. If you give me your login info I will try. Right now put sonme dummy password and when I have fixed it you can switch to some secure password.
Krittika Goyal is offline   Reply With Quote
Old 02-11-2013, 08:08 PM   #3
jumafl
Enthusiast
jumafl began at the beginning.
 
Posts: 32
Karma: 10
Join Date: Apr 2012
Device: Amazon Kindle Paperwhite
Thank you Krittika. I appreciate the help. Registration on the site is free so I set up a test account for you to use. I'll send you the information in a private message.
jumafl is offline   Reply With Quote
Old 02-14-2013, 05:40 AM   #4
Krittika Goyal
Vox calibre
Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.Krittika Goyal ought to be getting tired of karma fortunes by now.
 
Krittika Goyal's Avatar
 
Posts: 412
Karma: 1175230
Join Date: Jan 2009
Device: Sony reader prs700, kobo
I am working on it. may take a couple of days.
Krittika Goyal is offline   Reply With Quote
Old 04-02-2015, 11:20 PM   #5
truth1ness
Zealot
truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!
 
Posts: 126
Karma: 50000
Join Date: Mar 2015
Device: none
Discover doesn't seem to be working for me. Each section just says Failed Feed Error 404. Is this recipe supposed to be able to pull down the print version if I have a login? http://discovermagazine.com/2015/may
truth1ness is offline   Reply With Quote
Old 04-17-2015, 09:56 PM   #6
truth1ness
Zealot
truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!
 
Posts: 126
Karma: 50000
Join Date: Mar 2015
Device: none
I'm working on this script and making another variation for the monthly magazine version but I think the login portion of the script is not working and I'm not sure how to proceed because it seems to have the correct field names.

I put in a working feed and stripped down the default DiscoverMagazine recipe so it would just [try to] log in and do no filtering. The raw output of the ebook it produces shows Sign In/Register so I know the log in portion is not working (there is probably an easier way to check this but I'm not sure how). Below is the stripped down version I used.

Registration on the site is free so you can make one if you don't already have a login.

Spoiler:

#!/usr/bin/env python2
__license__ = 'GPL v3'
__copyright__ = '2008, Kovid Goyal kovid@kovidgoyal.net'
__docformat__ = 'restructuredtext en'

'''
discovermagazine.com
'''

import re
from calibre.web.feeds.news import BasicNewsRecipe

class DiscoverMagazine(BasicNewsRecipe):

title = u'Discover Magazine'
description = u'Science, Technology and the Future'
__author__ = 'Starson17'
language = 'en'

oldest_article = 33
max_articles_per_feed = 20
no_stylesheets = True
remove_javascript = True
use_embedded_content = False
encoding = 'utf-8'
extra_css = '.headline {font-size: x-large;} \n .fact {padding-top: 10pt}'

# Login stuff
needs_subscription = True
use_javascript_to_login = True
requires_version = (0, 9, 20)

def javascript_login(self, br, username, password):
br.visit('http://discovermagazine.com', timeout=120)
f = br.select_form('div.login.section div.form')
f['username'] = username
f['password'] = password
br.submit('input[id="signInButton"]', timeout=120)
br.run_for_a_time(20)
# End login stuff


def append_page(self, soup, appendtag, position):
pager = soup.find('span',attrs={'class':'next'})
if pager:
nexturl = pager.a['href']
soup2 = self.index_to_soup(nexturl)
texttag = soup2.find('div', attrs={'class':'articlebody'})
newpos = len(texttag.contents)
self.append_page(soup2,texttag,newpos)
texttag.extract()
appendtag.insert(position,texttag)

feeds = [
(u'Top Stories', u'http://feeds.feedburner.com/DiscoverTopStories'),
]
truth1ness is offline   Reply With Quote
Old 04-17-2015, 11:09 PM   #7
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,856
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Simply put

br.show_browser()

at the end of the javascript_login() method. This will open the browser window. Note that this will not work on linux (where the browser is headless).
kovidgoyal is offline   Reply With Quote
Old 04-21-2015, 12:39 AM   #8
truth1ness
Zealot
truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!
 
Posts: 126
Karma: 50000
Join Date: Mar 2015
Device: none
I've added a new Discover Magazine recipe for the Monthly Print version of the magazine to the repository. The monthly print version does require a subscription.
truth1ness is offline   Reply With Quote
Old 05-08-2015, 05:09 PM   #9
NSILMike
Guru
NSILMike turned on, tuned in, and dropped out.NSILMike turned on, tuned in, and dropped out.NSILMike turned on, tuned in, and dropped out.NSILMike turned on, tuned in, and dropped out.NSILMike turned on, tuned in, and dropped out.NSILMike turned on, tuned in, and dropped out.NSILMike turned on, tuned in, and dropped out.NSILMike turned on, tuned in, and dropped out.NSILMike turned on, tuned in, and dropped out.NSILMike turned on, tuned in, and dropped out.NSILMike turned on, tuned in, and dropped out.
 
Posts: 735
Karma: 35936
Join Date: Apr 2011
Location: Shrewsury, MA
Device: Lenovo Android Tablet
Quote:
Originally Posted by truth1ness View Post
I've added a new Discover Magazine recipe for the Monthly Print version of the magazine to the repository. The monthly print version does require a subscription.
Works well! Many thanks. It does require a login (free) but not a 'paid' subscription... or maybe I'm not getting the entire issue...? Seems like I am though.
Last edited by NSILMike; 05-08-2015 at 05:12 PM.
NSILMike is offline   Reply With Quote
Old 05-08-2015, 07:30 PM   #10
truth1ness
Zealot
truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!truth1ness is faster than a rolling 'o,' stronger than silent 'e,' and leaps capital 'T' in a single bound!
 
Posts: 126
Karma: 50000
Join Date: Mar 2015
Device: none
I don't think it will work without a paid subscription, but can't argue if you are getting the whole thing. You may be getting all the articles but they'll probably be truncated to only the first paragraph since that's what they show on subscription only pages like http://discovermagazine.com/2015/jun...rrow-never-was
truth1ness is offline   Reply With Quote
Reply

« Previous Thread | Next Thread »

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Science AAAS recipe no longer seems to be working dmkaplan Recipes 5 11-01-2012 02:14 AM
El Pais recipe no longer working correctly phisalis Recipes 5 09-26-2012 03:57 AM
Recipe for Sueddeutsche no longer working George's Kindle Recipes 2 12-15-2011 08:39 PM
gocomics recipe not longer working in full Andreas Recipes 8 05-20-2011 10:25 AM
Irish Times recipe - no longer working patrickpc Recipes 1 11-17-2010 12:16 PM


All times are GMT -4. The time now is 08:42 PM.

Contact Us - FAQ - Guidelines - Privacy - Top

MobileRead.com is a privately owned, operated and funded community.