Help With Referrer

rth26 · 04-08-2012, 09:33 AM

I have been playing around with a few recipes trying to set the referrers.

Essential I am trying to get around the login on news sites. I understand this is controversial, however I'm a college student and pay for access to the news sources in my mandatory student fees.

Unfortunately the access is only available via the paper sources on campus or through journals online which are extremely frustrating the navigate and tend to take a while to be updated with the latest articles.

I have tried setting the referrer a couple of different ways including using mechanize and it works partially for some recipes and not at all for others.

I would be willing to talk about it through PM or email since I don't want to get in any trouble or upset anyone posting the code.

nickredding · 04-08-2012, 11:34 AM

Quote:

Originally Posted by rth26

I have tried setting the referrer a couple of different ways including using mechanize and it works partially for some recipes and not at all for others.

If that is the case then setting the referrer is not the problem, it is what you are setting it to versus what the websites want to see.

rth26 · 04-08-2012, 11:04 PM

Well I'm trying to get it to work with the Financial Times.

The referrer I'm attempting to use definitely works, I've tested it with RefController in firefox with no problems.

It works with the one FT recipe but not for all the articles, about 50 or so articles in it appears to stop working. However it doesn't seem to wrk at all for the UK edition, which is actually the one I would prefer because it the layout it better for the Kindle.

Any idea?

rth26 · 04-10-2012, 09:58 AM

I've been playing around with it and I'm still not sure what the problem is.

I ran this command:

ebook-convert ftuk.recipe _Test_1 --test -vv > _Test.txt

I'm not exactly sure what is supposed to do besides being used for testing purposes but the index it gives me has a small sample of articles (about 4) and they are all there, downloaded fine.

But running:
ebook-convert ftuk.recipe ftuk.epub

Give me an epub with the text of the articles missing.

Any ideas?

04-08-2012, 09:33 AM	#1
rth26 Junior Member Posts: 3 Karma: 10 Join Date: Apr 2012 Device: Kindle	Help With Referrer I have been playing around with a few recipes trying to set the referrers. Essential I am trying to get around the login on news sites. I understand this is controversial, however I'm a college student and pay for access to the news sources in my mandatory student fees. Unfortunately the access is only available via the paper sources on campus or through journals online which are extremely frustrating the navigate and tend to take a while to be updated with the latest articles. I have tried setting the referrer a couple of different ways including using mechanize and it works partially for some recipes and not at all for others. I would be willing to talk about it through PM or email since I don't want to get in any trouble or upset anyone posting the code.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Referrer problem with recipe	Starson17	Calibre	7	06-26-2010 04:47 PM

04-08-2012, 11:04 PM	#3
rth26 Junior Member Posts: 3 Karma: 10 Join Date: Apr 2012 Device: Kindle	Well I'm trying to get it to work with the Financial Times. The referrer I'm attempting to use definitely works, I've tested it with RefController in firefox with no problems. It works with the one FT recipe but not for all the articles, about 50 or so articles in it appears to stop working. However it doesn't seem to wrk at all for the UK edition, which is actually the one I would prefer because it the layout it better for the Kindle. Any idea?

04-10-2012, 09:58 AM	#4
rth26 Junior Member Posts: 3 Karma: 10 Join Date: Apr 2012 Device: Kindle	I've been playing around with it and I'm still not sure what the problem is. I ran this command: ebook-convert ftuk.recipe _Test_1 --test -vv > _Test.txt I'm not exactly sure what is supposed to do besides being used for testing purposes but the index it gives me has a small sample of articles (about 4) and they are all there, downloaded fine. But running: ebook-convert ftuk.recipe ftuk.epub Give me an epub with the text of the articles missing. Any ideas?

Advert