11-25-2009, 03:09 AM | #1 |
Grand Sorcerer
Posts: 10,270
Karma: 1126878541
Join Date: Oct 2009
Device: Astak Pocket PRO, iPod Touch, PRS-650
|
Fetching The Australian
It seems The Australian (under English AU) is unable to be downloaded at the moment from the recipes in Calibre. As it tries to download each article it says they are failing. At the end I finish up with a 0.0MB newspaper with just the initial hyperlinks page.
Are others that try to get The Australian having the same problem at the moment? |
11-25-2009, 03:49 AM | #2 | |
Wizard
Posts: 2,981
Karma: 11862367
Join Date: Apr 2008
Device: Sony Reader PRS-T2
|
Quote:
A little offtopic: I wonder if The Oz will make it harder to access their content in the future? It is a News Corporation newspaper after all, and Rupert Murdoch has been threatening to make people pay for all content.. |
|
11-25-2009, 09:30 AM | #3 |
creator of calibre
Posts: 43,835
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
The WSJ for example is password protected, but calibre can handle that easily.
|
11-25-2009, 10:08 AM | #4 |
Connoisseur
Posts: 78
Karma: 192
Join Date: Nov 2009
Device: Sony PRS-600
|
I had a quick look, and it seems their website changed. But more important, the articlelinks from their feed are redirected twice; the last one points to a local path. It looks like Calibre chokes on that, which is weird because when I add the following to the recipe
Code:
def get_article_url(self, article) : br = self.get_browser() br.open(article.link).read() #print br.geturl() return br.geturl() Also the articles are now empty, but that is because the CSS classes changed |
11-25-2009, 01:49 PM | #5 |
creator of calibre
Posts: 43,835
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Hmm, the download code should deal with redirects automatically, you shouldn't have to load the article first and then return the url.
|
11-25-2009, 06:48 PM | #6 |
Grand Sorcerer
Posts: 10,270
Karma: 1126878541
Join Date: Oct 2009
Device: Astak Pocket PRO, iPod Touch, PRS-650
|
Thanks for the replies so far. So I guess it isn't just me then.
|
11-29-2009, 07:00 AM | #7 | |
Connoisseur
Posts: 78
Karma: 192
Join Date: Nov 2009
Device: Sony PRS-600
|
Quote:
The following also works: Code:
def get_article_url(self, article): return article.id Anyway, while waiting for the men in white coats to arrive, I cooked up the following theoz.zip With all feeds enabled it downloads in just under 7 minutes, but YMMV |
|
11-30-2009, 04:15 AM | #8 |
Grand Sorcerer
Posts: 10,270
Karma: 1126878541
Join Date: Oct 2009
Device: Astak Pocket PRO, iPod Touch, PRS-650
|
Evanmaastrigt
Thanks for the recipe! Works really well. Download was about 8 minutes for me. |
Thread Tools | Search this Thread |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Trouble fetching news | jpustin | Calibre | 6 | 06-28-2010 01:57 PM |
Australian Reader | blackmorehouse | Introduce Yourself | 8 | 06-25-2010 10:52 PM |
No Fetching | Lector77 | Calibre | 13 | 06-21-2010 07:19 PM |
Recipe for The Australian | Valhalla | Calibre | 5 | 06-13-2010 04:38 PM |
Another Australian | pikelet | Introduce Yourself | 4 | 10-30-2009 04:41 PM |