How to add delay in fetching the articles

Ramana · 05-20-2019, 07:44 AM

I have the requirement to download the aritcles those span over multiple pages. In the url, the page number changes. When I added all the articles to feeds, getting the error while fetching. May because of issue of not to respond that many requests by the server. I want to delay in between fetch of each article. Is there a way we can add delay during fetch ?

kovidgoyal · 05-20-2019, 10:25 AM

Use

delay

in your recipe

Ramana · 05-20-2019, 12:03 PM

Quote:

Originally Posted by kovidgoyal

Use

delay

in your recipe

Hi Kovid,
If you use delay in recipe, only one download happens at a time. But the fetching of article is happening continuously without any delay. I want delay to be added for each article download even in case of one download at a time.
Is there any api for this ? If not, can this be added as enhancement in BasicNewsRecipe ?

kovidgoyal · 05-20-2019, 09:54 PM

No delay is inserted between every request made to the server, for pages, images, etc.

Ramana · 06-03-2019, 08:10 AM

Quote:

Originally Posted by kovidgoyal

No delay is inserted between every request made to the server, for pages, images, etc.

Hi Kovid,
Could you add optional delay property in sending request to the server ?

kovidgoyal · 06-03-2019, 09:25 AM

there already is one

Ramana · 06-04-2019, 12:36 AM

Quote:

Originally Posted by kovidgoyal

there already is one

I meant not about the delay in parallel requests. I am looking for the delay to submit the request to the server for each article. It if it is not implemented, could you add the support in coming calibre releases ? If it is already implemented how can we add delay submitting the request to server ?

kovidgoyal · 06-04-2019, 01:07 AM

One last time, delay is a delay that is implemented per request. Which means there is a delay between *every* request sent to the server.

Ramana · 06-04-2019, 01:17 AM

Quote:

Originally Posted by kovidgoyal

One last time, delay is a delay that is implemented per request. Which means there is a delay between *every* request sent to the server.

Thank you.

05-20-2019, 07:44 AM	#1
Ramana Connoisseur Posts: 51 Karma: 20728 Join Date: May 2015 Device: Onyx Boox i86 Hdml plus	How to add delay in fetching the articles I have the requirement to download the aritcles those span over multiple pages. In the url, the page number changes. When I added all the articles to feeds, getting the error while fetching. May because of issue of not to respond that many requests by the server. I want to delay in between fetch of each article. Is there a way we can add delay during fetch ?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Financial Times (UK) recipe no longer fetching all articles	piet8stevens	Recipes	1	02-23-2013 04:15 AM
[Tool] Erudite - Pull articles from Instapaper and add to Calbire.	ecognito	Plugins	0	09-14-2012 08:25 AM
Fetching multi-page articles	Steven630	Recipes	27	08-21-2012 10:04 PM
Problem: Recipe for Foreign Affairs not fetching premium articles	besianm	Recipes	1	03-07-2012 04:41 AM

05-20-2019, 10:25 AM	#2
kovidgoyal creator of calibre Posts: 45,345 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Use delay in your recipe

05-20-2019, 09:54 PM	#4
kovidgoyal creator of calibre Posts: 45,345 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	No delay is inserted between every request made to the server, for pages, images, etc.

06-03-2019, 09:25 AM	#6
kovidgoyal creator of calibre Posts: 45,345 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	there already is one

06-04-2019, 01:07 AM	#8
kovidgoyal creator of calibre Posts: 45,345 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	One last time, delay is a delay that is implemented per request. Which means there is a delay between every request sent to the server.

Advert

Advert