Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Recipes

Notices

Reply
 
Thread Tools Search this Thread
Old 07-23-2012, 10:05 AM   #1
yoss15
Enthusiast
yoss15 began at the beginning.
 
Posts: 37
Karma: 10
Join Date: Jul 2011
Device: Kindle
Referrer Help

I'm writing a recipe for a site which requires referers to access the articles properly. I set up the referer and the majority of the articles download fine but there are a lot which come out blank. I can't seem to find any pattern or reason why some aren't working.

Can anyone recommend some troubleshooting tips to figure out what the cause is?
yoss15 is offline   Reply With Quote
Old 07-23-2012, 11:51 AM   #2
Steven630
Zealot
Steven630 began at the beginning.
 
Posts: 142
Karma: 10
Join Date: May 2012
Device: Kindle Paperwhite2
I guess there's a limit to the referer trick. Use it too much, and it doesn't work. If that is the case, the same problem will occur even if you access the articles through a browser.How do you add referer to a recipe though?Sort of curious.
Steven630 is offline   Reply With Quote
Old 07-23-2012, 01:33 PM   #3
yoss15
Enthusiast
yoss15 began at the beginning.
 
Posts: 37
Karma: 10
Join Date: Jul 2011
Device: Kindle
Yeah but it doesn't seem to go in any order. Its not like the first so many work and the rest stop working, so I'm wondering if there is someway to do it in two sessions or more, allowing it work.

Here is the code for the referer:

Code:
    def get_browser(self):
        br = BasicNewsRecipe.get_browser(self)
        cookies = mechanize.CookieJar()
        br = mechanize.build_opener(mechanize.HTTPCookieProcessor(cookies))
        br.addheaders = [('Referer','http://www."referralsite".com')]
        return br
Just replace "referralsite" with what you want the referer to be.
yoss15 is offline   Reply With Quote
Old 07-24-2012, 12:04 AM   #4
Steven630
Zealot
Steven630 began at the beginning.
 
Posts: 142
Karma: 10
Join Date: May 2012
Device: Kindle Paperwhite2
I tried the code. But it says global name 'mechanize' is not defined. Do I need to import something at the beginning of the recipe? I imported mechanize, but got a "NotImplementedError".

Perhaps you can try adding a delay between downloads?

Last edited by Steven630; 07-24-2012 at 12:12 AM.
Steven630 is offline   Reply With Quote
Old 07-25-2012, 07:22 PM   #5
yoss15
Enthusiast
yoss15 began at the beginning.
 
Posts: 37
Karma: 10
Join Date: Jul 2011
Device: Kindle
You need an import mechanize statement at the beginning of the code.

Also oddly enough the issue only seems to come up when using the command line interface. I downloaded through the GUI with no issues, the only difference I know of being that the command line was .epub output and the GUI was .mobi.

Anyone have any idea why that might be?
yoss15 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Help With Referrer rth26 Recipes 3 04-10-2012 09:58 AM
Referrer problem with recipe Starson17 Calibre 7 06-26-2010 04:47 PM


All times are GMT -4. The time now is 10:48 AM.


MobileRead.com is a privately owned, operated and funded community.