Using Internet Explorer to download in Calibre

[TacitGibbon]

Calibre's recipe for a few newspapers ( like WSJ ) allows for user to provide login for full version access. My institution provides access to some of these sites via cookies ( You visit a custom Url that sets cookie in IE for x days for which one will have full access to the sites on that machine using IE)

Is there a way to configure calibre to use IE ( or IE cookies ) to download news pages ?

[Starson17]

Originally Posted by TacitGibbon

Calibre's recipe for a few newspapers ( like WSJ ) allows for user to provide login for full version access. My institution provides access to some of these sites via cookies ( You visit a custom Url that sets cookie in IE for x days for which one will have full access to the sites on that machine using IE)

Is there a way to configure calibre to use IE ( or IE cookies ) to download news pages ?

Are you saying that only IE works (FireFox and Chrome, etc. do not) or just that cookie support is required. I suspect it's just a cookie issue, and yes, the recipe system supports cookies. The recipe has to be written to go to the login page, provide the user/pass, which sets the cookies and then retrieve articles.

I wrote some code like this for Google Reader authentication that you can look at.

[khromov]

A cookie is just a text string though, so Tacit can read the content of the cookie (possible to do in all browsers) and then set the cookie in the recipe. You could also make the cookie last forever unless they specifically blocked that on the server end.

As for setting a cookie it is done by using get_browser() in calibre to fetch the browser object and then set the cookie by creating a "cookie jar" and adding a cookie to it - Calibre uses the "mechanize" python browser. Please see this for further documentation and help:
http://wwwsearch.sourceforge.net/mechanize/

[Starson17]

Originally Posted by zapt0

A cookie is just a text string though, so Tacit can read the content of the cookie (possible to do in all browsers) and then set the cookie in the recipe. You could also make the cookie last forever unless they specifically blocked that on the server end.

As for setting a cookie it is done by using get_browser() in calibre to fetch the browser object and then set the cookie by creating a "cookie jar" and adding a cookie to it - Calibre uses the "mechanize" python browser. Please see this for further documentation and help:
http://wwwsearch.sourceforge.net/mechanize/

Yes, that's how it's done (there's no need to explicitly set the cookie jar or use mechanize unless you need to modify the cookies or want to track them for debugging, as get_browser does it all for you), but he seemed to be saying that only IE worked (I suspect it's not true, but one can't be sure).

[khromov]

Originally Posted by Starson17

Yes, that's how it's done (there's no need to explicitly set the cookie jar or use mechanize unless you need to modify the cookies or want to track them for debugging, as get_browser does it all for you), but he seemed to be saying that only IE worked (I suspect it's not true, but one can't be sure).

Yeah it sounds kind of fishy that it'd only work in IE, worst-case one can always spoof the user-agent.

[Starson17]

Originally Posted by zapt0

Yeah it sounds kind of fishy that it'd only work in IE, worst-case one can always spoof the user-agent.

Yes, or modify some other header, as needed. I was working on getting a baseline on what he needed for authentication before going into details. Your post saved me the trouble

[TacitGibbon]

Thanks folks. I did not say (or know if ) it works Only on IE. All I was trying to say was that the access to sites are provided using cookie method. I visit site http://www.internalsite.net/provider=newspaper and it redirects and sets a cookie after which I am able to access the newspaper using the browser.

You guys have suggested using the get_browser() method, but a bit more details would help given I'm not very technical, but interested in getting this working.

Thanks a ton !

Tacit