02-20-2018, 07:12 AM | #1 |
Big Poppa
Posts: 110
Karma: 10
Join Date: Jul 2010
Device: Nook
|
Setting referer problems
Hi Kovid (and others)
I'm looking at customizing a recipe now which requires a particular referer. I've found old code here suggesting something like the following would work. Code:
def get_browser(self): USER_AGENT = 'Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0' br = BasicNewsRecipe.get_browser(self, user_agent=USER_AGENT) orig_open_novisit = br.open_novisit br.set_debug_http(True) br.set_debug_responses(True) br.set_debug_redirects(True) def my_open_no_visit(url, **kwargs): req = mechanize.Request(url, headers = {'Referer':'https://www.facebook.com/'}) return orig_open_novisit(req) br.open_novisit = my_open_no_visit return br Code:
1% Fetching feeds... send: 'GET /XXXXXXXXXX HTTP/1.1\r\nAccept-Encoding: identity\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0\r\nHost: www.XXXXXXXX.com\r\nAccept: */*\r\nConnection: close\r\n\r\n' reply: 'HTTP/1.1 200 OK\r\n' header: X-Content-Type-Options: nosniff header: X-Download-Options: noopen header: X-Xss-Protection: 1; mode=block header: Cache-Control: max-age=0, no-cache, no-store, must-revalidate header: Link: <//www.XXXXXXXXXX.com/assets/main.css>; as="style"; rel="preload"; nopush; as="script"; rel="preload"; nopush, as="script"; rel="preload"; nopush header: Content-Type: text/html; charset=utf-8 header: Etag: W/"6f1ba-VqmRNsUPEdGWrd9DOPy5aS7TMf8" header: Content-Length: 455098 header: Accept-Ranges: bytes header: Date: Tue, 20 Feb 2018 12:04:26 GMT header: Age: 95 header: Connection: close header: Vary: flags, anonymous-user, edition, Accept-Encoding header: Set-Cookie: Allocation=2d4c; Expires=Tue, 09 Mar 2021 18:00:00 GMT; Domain=.XXXXXXXX.com; Path=/ header: Set-Cookie: x-policy=-; Max-Age=60; Domain=.XXXX.com; Path=/ header: Set-Cookie: x-privileged-referer-model=(null); Max-Age=60; Domain=.XXXXX.com; Path=/ header: Set-Cookie: x-privileged-referer-phase=(null); Max-Age=60; Domain=.XXXX.com; Path=/ header: Strict-Transport-Security: max-age=2592000 in section: Front Page & Second Front title: XXXXXXXXXXXXXXXX url: https://www.XXXXXXXXXXXXX.com/XXXXXXXXXXXXXXXXXXXX |
02-20-2018, 09:57 AM | #2 |
creator of calibre
Posts: 43,775
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Just use
Code:
br.addheaders += [('Referer', 'whatever')] |
02-21-2018, 08:41 AM | #3 |
Big Poppa
Posts: 110
Karma: 10
Join Date: Jul 2010
Device: Nook
|
Awesome, thanks that worked
|
Thread Tools | Search this Thread |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
having problems setting up cc on my phone | Sydney's Mom | Calibre Companion | 12 | 08-15-2016 02:40 PM |
Setting up email server problems | npyachtsman | Devices | 0 | 02-13-2012 02:56 AM |
902 Problems setting up wireless connection | gshrtldgMR | PocketBook | 3 | 01-03-2012 12:02 PM |
Problems setting date and time | Katie H | Kobo Reader | 2 | 05-15-2010 09:00 AM |
Having Problems Setting My E-Mail Accounts Up On The eDGe | Batman Jr. | enTourage eDGe | 3 | 03-22-2010 07:42 PM |