[Starson17]

Originally Posted by nickredding

Set referer to http://news.google.com/news/search?q=wsj and it works unless you have done something to your environment that overrides referer.

I'm forging most referers on my system. I don't like tracking. I wasn't implying it won't work, just that it won't work on my system until I allow it, and I didn't want to go to that effort, just for testing. Since it works for you, I don't need to test it further.

As to how to do it in a recipe - Mechanize will forge the referer header. I've done it for other recipes. If you want to look at how it's done, I recall that I did it in the Google Reader authentication recipe, in one of the comic recipes (probably gocomics) and in one of the skeptic recipes I wrote. (Various headers - not always referer.)