View Single Post
Old 11-02-2023, 01:36 AM   #1
etmurphy
Emmet Murphy
etmurphy began at the beginning.
 
Posts: 5
Karma: 10
Join Date: Apr 2022
Location: San Francisco, CA, USA
Device: Kindle Oasis
WSJ recipe gets "Too Many Requests" error for some articles

The WSJ recipe will download around 70% of the first articles for an issue, but the remaining articles will get only the text "Too Many Requests". Could the explanation be that archive.is rate limits requests, for example allowing only so many requests per minute? (PR 2075 in release 6.29 notes the switch to archive.is: https://github.com/kovidgoyal/calibre/pull/2075)

Assuming that rate limiting is the root cause, is there a way to configure the number of download threads or otherwise slow down Calibre downloads so that it can get all articles for each WSJ issue?
etmurphy is offline   Reply With Quote