View Single Post
Old 07-11-2010, 05:11 AM   #25
rollercoaster
Zealot
rollercoaster once ate a cherry pie in a record 7 seconds.rollercoaster once ate a cherry pie in a record 7 seconds.rollercoaster once ate a cherry pie in a record 7 seconds.rollercoaster once ate a cherry pie in a record 7 seconds.rollercoaster once ate a cherry pie in a record 7 seconds.rollercoaster once ate a cherry pie in a record 7 seconds.rollercoaster once ate a cherry pie in a record 7 seconds.rollercoaster once ate a cherry pie in a record 7 seconds.rollercoaster once ate a cherry pie in a record 7 seconds.rollercoaster once ate a cherry pie in a record 7 seconds.rollercoaster once ate a cherry pie in a record 7 seconds.
 
rollercoaster's Avatar
 
Posts: 126
Karma: 1826
Join Date: Jan 2010
Device: Kindle 2
@Starson17 Your work helped me a lot and I think I know how to fix this but I still need your help to write python code. You are on the spot with fetching the AUTH token.

The issue is after the " def get_feeds(self): " method. We get the various feeds here and then each feed is queried to get articles/items under them. The request to those feeds DOES NOT pass the AUTH token as a header. And so, if you look at the log in jobs list then you will see that, google redirects these requests with a "302 Moved Temporarily" and sends to the login page. So we never get to the list of items in the feed.

Code:
send: 'POST /accounts/ClientLogin HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 97\r\nHost: www.google.com\r\nContent-Type: application/x-www-form-urlencoded\r\nConnection: close\r\nUser-Agent: Mozilla/5.0 (X11; U; i686 Linux; en_US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4\r\n\r\n'
send: 'Email=someone&Passwd=someone&service=reader&accountType=HOSTED_OR_GOOGLE&source=calibre'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: text/plain
header: Cache-control: no-cache, no-store
header: Pragma: no-cache
header: Expires: Mon, 01-Jan-1990 00:00:00 GMT
header: Date: Sun, 11 Jul 2010 08:50:30 GMT
header: X-Content-Type-Options: nosniff
header: X-XSS-Protection: 1; mode=block
header: Content-Length: 733
header: Server: GSE
header: Connection: close
send: 'GET /reader/api/0/tag/list HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: www.google.com\r\nConnection: close\r\nAuthorization: GoogleLogin auth=DQAAAKIAAABiWf2ZNf7xjpkr97aUJusKEEnI9ClihCCQMtpu3NrqbJRAOzU-ix4K42nnKcAjsQaREfV0XrS-ISsZRs6YrMoH_gBB3M0rUD2thscZHmkvBxZP0S07gvwthEaruqD-j2hAwIKTXWh4csgdmhzjjvgcP6dSlJPqnzMVuxgC2J-t-EyFNDY4W8Ihi-ybimjip_pITYMP7vSuPCyJWOFCrQzM1ES8Ewmp_nn1LU4-_uihxw\r\nUser-Agent: Mozilla/5.0 (X11; U; i686 Linux; en_US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: text/xml; charset=UTF-8
header: X-Reader-Google-Version: 506-000
header: X-Reader-User: 10347379608328268864
header: Set-Cookie: GRLD=UNSET:10347379608328268864;Path=/reader/;HttpOnly
header: Date: Sun, 11 Jul 2010 08:50:30 GMT
header: Expires: Sun, 11 Jul 2010 08:50:30 GMT
header: Cache-Control: private, max-age=0
header: X-Content-Type-Options: nosniff
header: X-Frame-Options: SAMEORIGIN
header: X-XSS-Protection: 1; mode=block
header: Server: GSE
header: Connection: close
send: 'GET /reader/atom/user/10347379608328268864/state/com.google/starred?n=250&xt=user/-/state/com.google/read HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: www.google.com\r\nCookie: GRLD=UNSET:10347379608328268864\r\nConnection: close\r\nUser-Agent: Mozilla/5.0 (X11; U; i686 Linux; en_US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4\r\n\r\n'
reply: 'HTTP/1.1 302 Moved Temporarily\r\n'
header: Content-Type: text/html; charset=UTF-8
header: X-Reader-Google-Version: 506-000
header: Location: https://www.google.com/accounts/ServiceLogin?service=reader&passive=1209600&continue=http%3A%2F%2Fwww.google.com%2Freader%2Fatom%2Fuser%2F10347379608328268864%2Fstate%2Fcom.google%2Fstarred%3Fn%3D250%26xt%3Duser%252F-%252Fstate%252Fcom.google%252Fread&followup=http%3A%2F%2Fwww.google.com%2Freader%2Fatom%2Fuser%2F10347379608328268864%2Fstate%2Fcom.google%2Fstarred%3Fn%3D250%26xt%3Duser%252F-%252Fstate%252Fcom.google%252Fread
header: Date: Sun, 11 Jul 2010 08:50:31 GMT
header: Expires: Sun, 11 Jul 2010 08:50:31 GMT
header: Cache-Control: private, max-age=0
header: X-Content-Type-Options: nosniff
header: X-Frame-Options: SAMEORIGIN
header: X-XSS-Protection: 1; mode=block
header: Content-Length: 1381
header: Server: GSE
header: Connection: close
send: 'GET /accounts/ServiceLogin?service=reader&passive=1209600&continue=http%3A%2F%2Fwww.google.com%2Freader%2Fatom%2Fuser%2F10347379608328268864%2Fstate%2Fcom.google%2Fstarred%3Fn%3D250%26xt%3Duser%252F-%252Fstate%252Fcom.google%252Fread&followup=http%3A%2F%2Fwww.google.com%2Freader%2Fatom%2Fuser%2F10347379608328268864%2Fstate%2Fcom.google%2Fstarred%3Fn%3D250%26xt%3Duser%252F-%252Fstate%252Fcom.google%252Fread HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: www.google.com\r\nConnection: close\r\nUser-Agent: Mozilla/5.0 (X11; U; i686 Linux; en_US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: text/html; charset=UTF-8
header: Cache-control: no-cache, no-store
header: Pragma: no-cache
header: Expires: Mon, 01-Jan-1990 00:00:00 GMT
header: Set-Cookie: GALX=T4Ka0MjTUcE;Path=/accounts;Secure
header: Date: Sun, 11 Jul 2010 08:50:32 GMT
header: X-Content-Type-Options: nosniff
header: X-XSS-Protection: 1; mode=block
header: Content-Length: 13863
header: Server: GSE
header: Connection: close
send: 'GET /reader/atom/user/10347379608328268864/state/com.google/reading-list?n=250&xt=user/-/state/com.google/read HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: www.google.com\r\nCookie: GRLD=UNSET:10347379608328268864\r\nConnection: close\r\nUser-Agent: Mozilla/5.0 (X11; U; i686 Linux; en_US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4\r\n\r\n'
reply: 'HTTP/1.1 302 Moved Temporarily\r\n'
header: Content-Type: text/html; charset=UTF-8
header: X-Reader-Google-Version: 506-000
header: Location: https://www.google.com/accounts/ServiceLogin?service=reader&passive=1209600&continue=http%3A%2F%2Fwww.google.com%2Freader%2Fatom%2Fuser%2F10347379608328268864%2Fstate%2Fcom.google%2Freading-list%3Fn%3D250%26xt%3Duser%252F-%252Fstate%252Fcom.google%252Fread&followup=http%3A%2F%2Fwww.google.com%2Freader%2Fatom%2Fuser%2F10347379608328268864%2Fstate%2Fcom.google%2Freading-list%3Fn%3D250%26xt%3Duser%252F-%252Fstate%252Fcom.google%252Fread
header: Date: Sun, 11 Jul 2010 08:50:33 GMT
header: Expires: Sun, 11 Jul 2010 08:50:33 GMT
header: Cache-Control: private, max-age=0
header: X-Content-Type-Options: nosniff
header: X-Frame-Options: SAMEORIGIN
header: X-XSS-Protection: 1; mode=block
header: Content-Length: 1381
header: Server: GSE
header: Connection: close
send: 'GET /accounts/ServiceLogin?service=reader&passive=1209600&continue=http%3A%2F%2Fwww.google.com%2Freader%2Fatom%2Fuser%2F10347379608328268864%2Fstate%2Fcom.google%2Freading-list%3Fn%3D250%26xt%3Duser%252F-%252Fstate%252Fcom.google%252Fread&followup=http%3A%2F%2Fwww.google.com%2Freader%2Fatom%2Fuser%2F10347379608328268864%2Fstate%2Fcom.google%2Freading-list%3Fn%3D250%26xt%3Duser%252F-%252Fstate%252Fcom.google%252Fread HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: www.google.com\r\nCookie: GALX=T4Ka0MjTUcE\r\nConnection: close\r\nUser-Agent: Mozilla/5.0 (X11; U; i686 Linux; en_US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: text/html; charset=UTF-8
header: Cache-control: no-cache, no-store
header: Pragma: no-cache
header: Expires: Mon, 01-Jan-1990 00:00:00 GMT
header: Set-Cookie: GALX=T4Ka0MjTUcE;Path=/accounts;Secure
header: Date: Sun, 11 Jul 2010 08:50:34 GMT
header: X-Content-Type-Options: nosniff
header: X-XSS-Protection: 1; mode=block
header: Content-Length: 13901
header: Server: GSE
header: Connection: close
I was using trial n error to write the code but there is too little documentation for both mechanize and calibre recipes. It took me an hour to find out that I can write to the log by using " PRINT <value> "

Anyway, have a look at this. I tried to get it working but could not. Language barrier I guess
It this doesnt/cant work for some reason then, conceivably, we can fill the login form with the username / pass each time and then submit and read the list of items. but that is the long way around.

So in summary, we need get the individual feed requests after "get_feeds(self)" to include the AUTH token and the recipe should hum along beautifully.

Last edited by rollercoaster; 07-11-2010 at 05:26 AM.
rollercoaster is offline   Reply With Quote