Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Recipes

Notices

Reply
 
Thread Tools Search this Thread
Old 02-20-2018, 07:12 AM   #1
bobbysteel
Big Poppa
bobbysteel began at the beginning.
 
Posts: 110
Karma: 10
Join Date: Jul 2010
Device: Nook
Setting referer problems

Hi Kovid (and others)
I'm looking at customizing a recipe now which requires a particular referer. I've found old code here suggesting something like the following would work.

Code:
    def get_browser(self):
        USER_AGENT = 'Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0'
        br = BasicNewsRecipe.get_browser(self, user_agent=USER_AGENT)
        orig_open_novisit = br.open_novisit
        br.set_debug_http(True)
        br.set_debug_responses(True)
        br.set_debug_redirects(True)
        def my_open_no_visit(url, **kwargs):
                req = mechanize.Request(url, headers = {'Referer':'https://www.facebook.com/'})
                return orig_open_novisit(req)
        br.open_novisit = my_open_no_visit
        return br
However when running this the logs show every other header but referer set. Is there some other change that's happened that would invalidate this?
Code:
1% Fetching feeds...
send: 'GET /XXXXXXXXXX HTTP/1.1\r\nAccept-Encoding: identity\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0\r\nHost: www.XXXXXXXX.com\r\nAccept: */*\r\nConnection: close\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: X-Content-Type-Options: nosniff
header: X-Download-Options: noopen
header: X-Xss-Protection: 1; mode=block
header: Cache-Control: max-age=0, no-cache, no-store, must-revalidate
header: Link: <//www.XXXXXXXXXX.com/assets/main.css>; as="style"; rel="preload"; nopush; as="script"; rel="preload"; nopush, as="script"; rel="preload"; nopush
header: Content-Type: text/html; charset=utf-8
header: Etag: W/"6f1ba-VqmRNsUPEdGWrd9DOPy5aS7TMf8"
header: Content-Length: 455098
header: Accept-Ranges: bytes
header: Date: Tue, 20 Feb 2018 12:04:26 GMT
header: Age: 95
header: Connection: close
header: Vary: flags, anonymous-user, edition, Accept-Encoding
header: Set-Cookie: Allocation=2d4c; Expires=Tue, 09 Mar 2021 18:00:00 GMT; Domain=.XXXXXXXX.com; Path=/
header: Set-Cookie: x-policy=-; Max-Age=60; Domain=.XXXX.com; Path=/
header: Set-Cookie: x-privileged-referer-model=(null); Max-Age=60; Domain=.XXXXX.com; Path=/
header: Set-Cookie: x-privileged-referer-phase=(null); Max-Age=60; Domain=.XXXX.com; Path=/
header: Strict-Transport-Security: max-age=2592000
in section:   Front Page & Second Front
title:   XXXXXXXXXXXXXXXX   url:  https://www.XXXXXXXXXXXXX.com/XXXXXXXXXXXXXXXXXXXX
bobbysteel is offline   Reply With Quote
Old 02-20-2018, 09:57 AM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,775
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Just use

Code:
br.addheaders += [('Referer', 'whatever')]
kovidgoyal is online now   Reply With Quote
Old 02-21-2018, 08:41 AM   #3
bobbysteel
Big Poppa
bobbysteel began at the beginning.
 
Posts: 110
Karma: 10
Join Date: Jul 2010
Device: Nook
Awesome, thanks that worked
bobbysteel is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
having problems setting up cc on my phone Sydney's Mom Calibre Companion 12 08-15-2016 02:40 PM
Setting up email server problems npyachtsman Devices 0 02-13-2012 02:56 AM
902 Problems setting up wireless connection gshrtldgMR PocketBook 3 01-03-2012 12:02 PM
Problems setting date and time Katie H Kobo Reader 2 05-15-2010 09:00 AM
Having Problems Setting My E-Mail Accounts Up On The eDGe Batman Jr. enTourage eDGe 3 03-22-2010 07:42 PM


All times are GMT -4. The time now is 11:30 AM.


MobileRead.com is a privately owned, operated and funded community.