View Single Post
Old 07-11-2010, 09:28 AM   #26
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by rollercoaster View Post
@Starson17 Your work helped me a lot and I think I know how to fix this but I still need your help to write python code. You are on the spot with fetching the AUTH token.
Yes, until the auth token was sent correctly, we couldn't get the list of feeds from http://www.google.com/reader/api/0/tag/list

Obviously, that link doesn't contain the user info. As soon as I fixed the auth header, that link worked, so I thought I was sending the header correctly for all links. Stupid me, I didn't check. (funny thing - I left in the print headers code so you could see I was sending the auth code and you used it to see that I wasn't sending it - at least not for the article fetch. )

Quote:
The issue is after the " def get_feeds(self): " method. We get the various feeds here and then each feed is queried to get articles/items under them. The request to those feeds DOES NOT pass the AUTH token as a header. And so, if you look at the log in jobs list then you will see that, google redirects these requests with a "302 Moved Temporarily" and sends to the login page. So we never get to the list of items in the feed.
Yes, I saw I was getting redirects, but didn't realize I wasn't sending the header. I used the same method I've used before, and it was working on the links page. It didn't work on the links page until I got the header format corrected, so I assumed it was working elsewhere to send that header.

Quote:
Anyway, have a look at this. I tried to get it working but could not. Language barrier I guess
It this doesnt/cant work for some reason then, conceivably, we can fill the login form with the username / pass each time and then submit and read the list of items. but that is the long way around.
I'd seen this link, but since I thought I was sending the header correctly, I never tried this method of adding headers. As soon as I did, it worked to pull all the articles. Here's the fixed recipe:

Spoiler:
Code:
import urllib, re, mechanize
from calibre.web.feeds.recipes import BasicNewsRecipe
from calibre import __appname__

class GoogleReader(BasicNewsRecipe):
    title   = 'Google Reader'
    description = 'Feeds from your Google Reader account.'
    needs_subscription = True
    __author__ = 'davec, rollercoaster, Starson17'
    base_url = 'http://www.google.com/reader/atom/'
    max_articles_per_feed = 50
    get_options = '?n=%d&xt=user/-/state/com.google/read' % max_articles_per_feed
    use_embedded_content = True

    def get_browser(self):
        br = BasicNewsRecipe.get_browser(self)
        if self.username is not None and self.password is not None:
            request = urllib.urlencode([('Email', self.username), ('Passwd', self.password),
                                        ('service', 'reader'), ('accountType', 'HOSTED_OR_GOOGLE'), ('source', __appname__)])
            response = br.open('https://www.google.com/accounts/ClientLogin', request)
            auth = re.search('Auth=(\S*)', response.read()).group(1)
            print 'Auth is: ', auth
            cookies = mechanize.CookieJar()
            br = mechanize.build_opener(mechanize.HTTPCookieProcessor(cookies))
            br.addheaders = [('Authorization', 'GoogleLogin auth='+auth)]
        return br

    def get_feeds(self):
        feeds = []
        soup = self.index_to_soup('http://www.google.com/reader/api/0/tag/list')
        for id in soup.findAll(True, attrs={'name':['id']}):
            url = id.contents[0]
            feeds.append((re.search('/([^/]*)$', url).group(1),
                          self.base_url + urllib.quote(url.encode('utf-8')) + self.get_options))
        return feeds

Test it, and if it also works for you, pass it to Kovid. You may also want to modify and test the Uber version. You shouldn't have any trouble, but if you do, let me know.

Last edited by Starson17; 07-12-2010 at 10:45 AM.
Starson17 is offline   Reply With Quote