Originally Posted by nickredding
I have concluded that this occurs when the NYTimes website stubbornly serves the ad page again even after a 5 second delay and a page reload.
Interestingly, this only seems to happen when there are multiple downloads taking place simultaneously.
The recipe I submitted has simultaneous_downloads=1 and this seems to prevent the problem from arising.
I have resolved this and attached an updated recipe with simultaneous_downloads=1 commented out (leaving caliber to apply the default number of concurrent downloads). It turns out that the "skip this ad" link contains some URL magic that prevents the ad from being re-served, so I have left this link intact (the previous recipe was removing it).