NY Times fails - Page 2

kovidgoyal · 09-18-2022, 09:57 PM

Quote:

Originally Posted by BillD

Under English news recipes ... there are now 2 recipes called "The New York Times", and the one called "...(Web)" is gone. Not sure if this intentional or not, or maybe just something I managed to mess up in my installation (Windows).

No that was a bug, the two recipes differ in only one line and I forgot to preserve that difference when updating them.

hongho71 · 10-03-2022, 11:39 AM

The New York Times recipe (both print and web versions) only downloads a handful of articles and not the entire thing. Any fix for this? Thanks!

hongho71 · 10-03-2022, 11:26 PM

Nevermind, I had VPN on and it restricted news fetch. When I turned off VPN, everything downloads as normal. I guess NY Times doesn't like VPN.

NSILMike · 10-10-2022, 08:13 AM

NY Times is failing again. Now it gets stuck at 22% and never completes.

davidboy · 11-07-2022, 07:41 AM

Hi, I have found out the reason why the New YorK Times(web) failed to be fetched: the NYT system will block your IP if you fetch news too fast! So the solution to the problem should be: to add a delay between fetching each HTML page. I set the delay time to be 15 seconds and it works. You can set the delay time to be shorter, but the longer the delay time is, the better. I paste my recipe setting below for your reference. Pay attention to the setting of "delay = 15" :
class NewYorkTimes(BasicNewsRecipe):

if is_web_edition:
title = 'The New York Times (Web)'
description = 'New York Times (Web). You can edit the recipe to remove sections you are not interested in.'
delay = 15
else:
title = 'The New York Times'
description = 'Today\'s New York Times'
encoding = 'utf-8'
__author__ = 'Kovid Goyal'
language = 'en'
ignore_duplicate_articles = {'title', 'url'}
no_stylesheets = True
compress_news_images = True
compress_news_images_auto_size = 5
conversion_options = {'flow_size': 0}
delay = 15

kovidgoyal · 11-07-2022, 07:53 AM

I doubt you need 15 1 or 2 is usually enough for most bot detection.

fengli · 11-10-2022, 09:00 PM

The same problem, extraction is blank, delayed 15 seconds or failed

kovidgoyal · 11-11-2022, 12:11 AM

Are you in China? Because it works fine without even a delay here.

fengli · 11-11-2022, 07:59 AM

yes china is fail ，How to solve this？

kovidgoyal · 11-11-2022, 09:45 AM

Use a custom recipe with whatever delay works for you. or use a vpn

fengli · 11-11-2022, 11:43 PM

vpn still fail

fengli · 11-11-2022, 11:55 PM

Quote:

Originally Posted by kovidgoyal

Are you running the builtin recipe? Post the log from the download.

Where to Download the Revised New York Times Cookbook

DNSB · 11-12-2022, 01:51 AM

Since you mentioned download, you can purchase the Essential New Times Cookbook 10th Anniversary Edition edited by Amanda Hesser from a variety of ebook vendors. If you are looking for an ebook edition of The New York Times Cookbook edited by Craig Claiborne, I was unable to find an ebook edition of it so no downloads.

Here in Canada, I could order the Essential New York Times Cookbook from Amazon.ca, Kobo, Google, etc.. If you are asking where to download it for free, that is ebook piracy which is not a condoned activity on Mobileread and is very likely to have you banned.

kck · 11-14-2022, 01:55 PM

A delay of 20 works for me, 15 doesn‘t. This means that it takes 2 hours to fetch the NYT. I guess that is as it is.

Best regards,
KCK

mkgtu · 11-15-2022, 01:59 PM

Beware. After a failed attempt to download the NYT this morning, my access to my paid subscription to NYT has been blocked on all platforms and web browsers - if I use the same IP address I was using this morning, my regular home wifi.
See screenshot.

The NYT mobile apps still work. But can't open any web articles in any browser, even after logout and login again, without using VPN, mobile data connection, or different WiFi connection.

And BTW I finally got a full download this morning using 15 second delay through a VPN. Took 41 minutes!

Sent from my SM-G975U using Tapatalk

11-12-2022, 01:51 AM	#28
DNSB Bibliophagist Posts: 35,498 Karma: 145557716 Join Date: Jul 2010 Location: Vancouver Device: Kobo Sage, Forma, Clara HD, Lenovo M8 FHD, Paperwhite 4, Tolino epos	Since you mentioned download, you can purchase the Essential New Times Cookbook 10th Anniversary Edition edited by Amanda Hesser from a variety of ebook vendors. If you are looking for an ebook edition of The New York Times Cookbook edited by Craig Claiborne, I was unable to find an ebook edition of it so no downloads. Here in Canada, I could order the Essential New York Times Cookbook from Amazon.ca, Kobo, Google, etc.. If you are asking where to download it for free, that is ebook piracy which is not a condoned activity on Mobileread and is very likely to have you banned. Last edited by DNSB; 11-12-2022 at 01:59 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
New York Times fails repeatedly	NSILMike	Recipes	7	02-19-2022 07:23 AM
The New York Times fails	NSILMike	Recipes	13	12-03-2021 09:31 AM
NY Times recipe fails	jazzbox	Recipes	3	04-21-2019 10:53 PM
NY Times fails repeatedly	NSILMike	Recipes	5	03-02-2017 02:46 PM
Emailing from Calibre fails at times	Klippy	Calibre	10	05-25-2015 05:16 PM

10-03-2022, 11:39 AM	#17
hongho71 Enthusiast Posts: 39 Karma: 10 Join Date: Feb 2015 Device: Kindle Paperwhite	The New York Times recipe (both print and web versions) only downloads a handful of articles and not the entire thing. Any fix for this? Thanks!

10-03-2022, 11:26 PM	#18
hongho71 Enthusiast Posts: 39 Karma: 10 Join Date: Feb 2015 Device: Kindle Paperwhite	Nevermind, I had VPN on and it restricted news fetch. When I turned off VPN, everything downloads as normal. I guess NY Times doesn't like VPN.

10-10-2022, 08:13 AM	#19
NSILMike Guru Posts: 735 Karma: 35936 Join Date: Apr 2011 Location: Shrewsury, MA Device: Lenovo Android Tablet	NY Times is failing again. Now it gets stuck at 22% and never completes.

11-07-2022, 07:41 AM	#20
davidboy Junior Member Posts: 1 Karma: 10 Join Date: Nov 2022 Device: kindle 10	Hi, I have found out the reason why the New YorK Times(web) failed to be fetched: the NYT system will block your IP if you fetch news too fast! So the solution to the problem should be: to add a delay between fetching each HTML page. I set the delay time to be 15 seconds and it works. You can set the delay time to be shorter, but the longer the delay time is, the better. I paste my recipe setting below for your reference. Pay attention to the setting of "delay = 15" : class NewYorkTimes(BasicNewsRecipe): if is_web_edition: title = 'The New York Times (Web)' description = 'New York Times (Web). You can edit the recipe to remove sections you are not interested in.' delay = 15 else: title = 'The New York Times' description = 'Today\'s New York Times' encoding = 'utf-8' __author__ = 'Kovid Goyal' language = 'en' ignore_duplicate_articles = {'title', 'url'} no_stylesheets = True compress_news_images = True compress_news_images_auto_size = 5 conversion_options = {'flow_size': 0} delay = 15

11-07-2022, 07:53 AM	#21
kovidgoyal creator of calibre Posts: 43,866 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I doubt you need 15 1 or 2 is usually enough for most bot detection.

11-10-2022, 09:00 PM	#22
fengli Connoisseur Posts: 80 Karma: 10 Join Date: Aug 2022 Device: PC	The same problem, extraction is blank, delayed 15 seconds or failed

11-11-2022, 12:11 AM	#23
kovidgoyal creator of calibre Posts: 43,866 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Are you in China? Because it works fine without even a delay here.

11-11-2022, 07:59 AM	#24
fengli Connoisseur Posts: 80 Karma: 10 Join Date: Aug 2022 Device: PC	yes china is fail ，How to solve this？

11-11-2022, 09:45 AM	#25
kovidgoyal creator of calibre Posts: 43,866 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Use a custom recipe with whatever delay works for you. or use a vpn

11-11-2022, 11:43 PM	#26
fengli Connoisseur Posts: 80 Karma: 10 Join Date: Aug 2022 Device: PC	vpn still fail

Advert

Advert

11-14-2022, 01:55 PM	#29
kck Member Posts: 10 Karma: 10 Join Date: May 2014 Device: none	A delay of 20 works for me, 15 doesn‘t. This means that it takes 2 hours to fetch the NYT. I guess that is as it is. Best regards, KCK

11-15-2022, 01:59 PM	#30
mkgtu Zealot Posts: 129 Karma: 33000 Join Date: Feb 2010 Device: Currently:Voyage, Oasis 3, Kindle mobile apps, andKindle Fire	Beware. After a failed attempt to download the NYT this morning, my access to my paid subscription to NYT has been blocked on all platforms and web browsers - if I use the same IP address I was using this morning, my regular home wifi. See screenshot. The NYT mobile apps still work. But can't open any web articles in any browser, even after logout and login again, without using VPN, mobile data connection, or different WiFi connection. And BTW I finally got a full download this morning using 15 second delay through a VPN. Took 41 minutes! Sent from my SM-G975U using Tapatalk