View Single Post
Old 03-14-2018, 11:37 AM   #9
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 33,213
Karma: 10205090
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Quote:
Originally Posted by bobbysteel View Post
Same here. It seems to be a case of the page being denied on the server from viewing via paywall.
If so, there is not much we can do about it, since the NYT requires a captcha to login, so we cannot log in in the recipe. You could try using delay = 1 which might avoid it (though it will make downloads very slow). Or if you want to get more sophisticated you can detect the paywall markup in preprocess_raw_html() and re-request the article.

Last edited by kovidgoyal; 03-14-2018 at 11:41 AM.
kovidgoyal is offline   Reply With Quote