Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 11-25-2009, 03:09 AM   #1
Javed
Grand Sorcerer
Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.
 
Javed's Avatar
 
Posts: 10,271
Karma: 1126878541
Join Date: Oct 2009
Device: Astak Pocket PRO, iPod Touch, PRS-650
Fetching The Australian

It seems The Australian (under English AU) is unable to be downloaded at the moment from the recipes in Calibre. As it tries to download each article it says they are failing. At the end I finish up with a 0.0MB newspaper with just the initial hyperlinks page.

Are others that try to get The Australian having the same problem at the moment?
Javed is offline   Reply With Quote
Old 11-25-2009, 03:49 AM   #2
AprilHare
Wizard
AprilHare ought to be getting tired of karma fortunes by now.AprilHare ought to be getting tired of karma fortunes by now.AprilHare ought to be getting tired of karma fortunes by now.AprilHare ought to be getting tired of karma fortunes by now.AprilHare ought to be getting tired of karma fortunes by now.AprilHare ought to be getting tired of karma fortunes by now.AprilHare ought to be getting tired of karma fortunes by now.AprilHare ought to be getting tired of karma fortunes by now.AprilHare ought to be getting tired of karma fortunes by now.AprilHare ought to be getting tired of karma fortunes by now.AprilHare ought to be getting tired of karma fortunes by now.
 
AprilHare's Avatar
 
Posts: 2,827
Karma: 11253505
Join Date: Apr 2008
Device: Sony Reader PRS-T2
Quote:
Originally Posted by Javed View Post
It seems The Australian (under English AU) is unable to be downloaded at the moment from the recipes in Calibre. As it tries to download each article it says they are failing. At the end I finish up with a 0.0MB newspaper with just the initial hyperlinks page.

Are others that try to get The Australian having the same problem at the moment?
Will test later.
A little offtopic: I wonder if The Oz will make it harder to access their content in the future? It is a News Corporation newspaper after all, and Rupert Murdoch has been threatening to make people pay for all content..
AprilHare is offline   Reply With Quote
Old 11-25-2009, 09:30 AM   #3
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,756
Karma: 4998511
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Quote:
Originally Posted by AprilHare View Post
Will test later.
A little offtopic: I wonder if The Oz will make it harder to access their content in the future? It is a News Corporation newspaper after all, and Rupert Murdoch has been threatening to make people pay for all content..
The WSJ for example is password protected, but calibre can handle that easily.
kovidgoyal is offline   Reply With Quote
Old 11-25-2009, 10:08 AM   #4
evanmaastrigt
Connoisseur
evanmaastrigt doesn't litterevanmaastrigt doesn't litter
 
Posts: 78
Karma: 192
Join Date: Nov 2009
Device: Sony PRS-600
I had a quick look, and it seems their website changed. But more important, the articlelinks from their feed are redirected twice; the last one points to a local path. It looks like Calibre chokes on that, which is weird because when I add the following to the recipe
Code:
    def get_article_url(self, article) :
        br = self.get_browser()
        br.open(article.link).read()
        #print br.geturl()

        return br.geturl()
articles are downloaded all right. But this makes it freaking slow. It there another workaround, anyone?

Also the articles are now empty, but that is because the CSS classes changed
evanmaastrigt is offline   Reply With Quote
Old 11-25-2009, 01:49 PM   #5
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,756
Karma: 4998511
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Hmm, the download code should deal with redirects automatically, you shouldn't have to load the article first and then return the url.
kovidgoyal is offline   Reply With Quote
Old 11-25-2009, 06:48 PM   #6
Javed
Grand Sorcerer
Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.
 
Javed's Avatar
 
Posts: 10,271
Karma: 1126878541
Join Date: Oct 2009
Device: Astak Pocket PRO, iPod Touch, PRS-650
Thanks for the replies so far. So I guess it isn't just me then.
Javed is offline   Reply With Quote
Old 11-29-2009, 07:00 AM   #7
evanmaastrigt
Connoisseur
evanmaastrigt doesn't litterevanmaastrigt doesn't litter
 
Posts: 78
Karma: 192
Join Date: Nov 2009
Device: Sony PRS-600
Quote:
Originally Posted by kovidgoyal View Post
Hmm, the download code should deal with redirects automatically, you shouldn't have to load the article first and then return the url.
That's what I thought too. But the download fails without this trick. Try it for yourself.
The following also works:
Code:
    
    def get_article_url(self, article):
        return article.id
which is just as weird.

Anyway, while waiting for the men in white coats to arrive, I cooked up the following

theoz.zip

With all feeds enabled it downloads in just under 7 minutes, but YMMV
evanmaastrigt is offline   Reply With Quote
Old 11-30-2009, 04:15 AM   #8
Javed
Grand Sorcerer
Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.Javed ought to be getting tired of karma fortunes by now.
 
Javed's Avatar
 
Posts: 10,271
Karma: 1126878541
Join Date: Oct 2009
Device: Astak Pocket PRO, iPod Touch, PRS-650
Evanmaastrigt

Thanks for the recipe! Works really well. Download was about 8 minutes for me.
Javed is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Trouble fetching news jpustin Calibre 6 06-28-2010 01:57 PM
Australian Reader blackmorehouse Introduce Yourself 8 06-25-2010 10:52 PM
No Fetching Lector77 Calibre 13 06-21-2010 07:19 PM
Recipe for The Australian Valhalla Calibre 5 06-13-2010 04:38 PM
Another Australian pikelet Introduce Yourself 4 10-30-2009 04:41 PM


All times are GMT -4. The time now is 03:14 PM.


MobileRead.com is a privately owned, operated and funded community.