HTTP Request - Mimic Different Browsers?

[EnergyLens]

I am trying to write a recipe for a site that doesn't seem to respond to the calibre HTTP requests with anything valid.

When I open the RSS url in Safari it tailors the XML to the browser
feed://www.frontporchrepublic.com/feed/rss/

When I open the web site directly in a recipe:
http://www.frontporchrepublic.com
I get
HTTP Error 403: Bad Behavior

If I open the RSS url:
feed://www.frontporchrepublic.com/feed/rss/
I get
<urlopen error unknown url type: feed>

I'm guessing this is because the HTTP Request coming from Calibre doesn't specify a browser type that the web site recognizes.

Is there a way to populate the HTTP Request with information to make the website think that Calibre is Firefox or Safari?

Thanks!

[kovidgoyal]

calibre mimics browsers anyway by default. IIRC it pretends to be Firefox 3.6.13 on Linux by default.

You can change this by implementing get_browser in your recipe. See the browser() function in the calibre source source

[Starson17]

Originally Posted by EnergyLens

I get
HTTP Error 403: Bad Behavior

Search for "Bad Behavior" here, or look at my Skeptic recipe.
The Bad Behavior module can be configured to require headers that the recipe browser may not supply by default.
See this post for a possible fix:
https://www.mobileread.com/forums/sho...93&postcount=2

[EnergyLens]

@Starson17 - your code snippet was just what I needed
@kovidgoyal - thanks for inspiring me to check the source

[Starson17]

Originally Posted by EnergyLens

@Starson17 - your code snippet was just what I needed

I'm glad it helped.