Wall Street Journal recipe is no longer downloading all articles.

nick.vaughn · 04-13-2015, 11:33 AM

PeterT · 04-13-2015, 12:07 PM

Could you maybe elaborate on what seems to be missing?

nick.vaughn · 04-14-2015, 08:05 AM

It seems to be somewhat random but usually the first article in a section is dropped (often more). This started about a week ago. I suspect they've changed something on the website and I'm not the only one experiencing this issue but I don't have the technical knowledge to figure it out.

Today I have all 4 articles in the Front Section.

In the U.S. section (http://www.wsj.com/itp/20150414/us/us) I have only 5 of the 10 articles and the first one is missing.

In the World section (http://www.wsj.com/itp/20150414/us/world) I have 9 of the 12 articles with the first one among the missing.

The Greater New York section has all 16 articles downloaded.

The Business & Tech section (http://www.wsj.com/itp/20150414/us/businessandtech) is missing 4-5 articles included the first one out of about 20.

In the Money & Investing section (http://www.wsj.com/itp/20150414/us/moneyandinvesting) I'm missing 4 articles including the first out of 15.

kovidgoyal · 04-14-2015, 10:57 PM

That sounds like something on your computer is interfering with the download process, or the WSJ server has implemented some anti-scraping measures. Look at the log of the download it will tell you what failures are occurring. You get the log by clicking the jobs spinner in the bottom right corner.

mfitz · 04-21-2015, 08:04 PM

There will, perhaps, be other challenges due to the newly launched WSJ redesign. I haven't been able to use The Wall Street Journal (account required) recipe successfully for at least a month. I tried fixing it myself with no luck.

Heres' the error I'm getting:
https://gist.github.com/anonymous/2341d784e5c28b610a5f

Any suggestions or fixes would be deeply appreciated.

Thanks!

kovidgoyal · 04-23-2015, 08:21 AM

I can see no errors in that log. Is it incomplete?

PeterT · 04-23-2015, 08:24 AM

Line 409 I see

Code:

Found article: France’s Move to Gain Clout at Renault Roils Key Alliance 
		 The French government’s bid to expand its voting control at Renault is roiling tQEventDispatcherUNIXPrivate(): Unable to create thread pipe: Too many open files
QEventDispatcherUNIXPrivate(): Can not continue without a thread pipe

kovidgoyal · 04-23-2015, 08:36 AM

http://docs.basho.com/riak/latest/op...imit/#Mac-OS-X

PeterT · 04-23-2015, 08:47 AM

Ah; so the infamous out of handles strikes again!

kovidgoyal · 04-23-2015, 08:50 AM

Yeah I dont know what it is with OS X systems and their very low default open file handle limits.

04-13-2015, 11:33 AM	#1
nick.vaughn Junior Member Posts: 2 Karma: 10 Join Date: Apr 2015 Device: Galaxy S 10.5	Wall Street Journal recipe is no longer downloading all articles. Wall Street Journal recipe is no longer downloading all articles.

04-21-2015, 08:04 PM	#5
mfitz Junior Member Posts: 1 Karma: 10 Join Date: Apr 2015 Device: Kindle Voyage	There will, perhaps, be other challenges due to the newly launched WSJ redesign. I haven't been able to use The Wall Street Journal (account required) recipe successfully for at least a month. I tried fixing it myself with no luck. Heres' the error I'm getting: https://gist.github.com/anonymous/2341d784e5c28b610a5f Any suggestions or fixes would be deeply appreciated. Thanks! Last edited by mfitz; 04-21-2015 at 10:26 PM. Reason: Fixed doubled words

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Fetch News for The Wall Street Journal (En) is not downloading it's articles	Brookings	Recipes	0	09-04-2014 04:36 AM
Wall Street Journal recipe broken?	nisew	Recipes	2	09-28-2011 05:08 PM
Wall Street Journal recipe - How good is it?	SpiderMatt	Recipes	3	08-28-2011 10:24 PM
Wall Street Journal no longer downloads	jetliu	Calibre	11	03-25-2009 10:00 AM
Wall Street Journal Recipe Quality?	rhsanborn	Calibre	6	12-04-2008 11:42 PM

04-13-2015, 12:07 PM	#2
PeterT Grand Sorcerer Posts: 13,511 Karma: 78910202 Join Date: Nov 2007 Location: Toronto Device: Libra H2O, Libra Colour	Could you maybe elaborate on what seems to be missing?

04-14-2015, 08:05 AM	#3
nick.vaughn Junior Member Posts: 2 Karma: 10 Join Date: Apr 2015 Device: Galaxy S 10.5	It seems to be somewhat random but usually the first article in a section is dropped (often more). This started about a week ago. I suspect they've changed something on the website and I'm not the only one experiencing this issue but I don't have the technical knowledge to figure it out. Today I have all 4 articles in the Front Section. In the U.S. section (http://www.wsj.com/itp/20150414/us/us) I have only 5 of the 10 articles and the first one is missing. In the World section (http://www.wsj.com/itp/20150414/us/world) I have 9 of the 12 articles with the first one among the missing. The Greater New York section has all 16 articles downloaded. The Business & Tech section (http://www.wsj.com/itp/20150414/us/businessandtech) is missing 4-5 articles included the first one out of about 20. In the Money & Investing section (http://www.wsj.com/itp/20150414/us/moneyandinvesting) I'm missing 4 articles including the first out of 15.

04-14-2015, 10:57 PM	#4
kovidgoyal creator of calibre Posts: 45,345 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	That sounds like something on your computer is interfering with the download process, or the WSJ server has implemented some anti-scraping measures. Look at the log of the download it will tell you what failures are occurring. You get the log by clicking the jobs spinner in the bottom right corner.

04-23-2015, 08:21 AM	#6
kovidgoyal creator of calibre Posts: 45,345 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I can see no errors in that log. Is it incomplete?

04-23-2015, 08:36 AM	#8
kovidgoyal creator of calibre Posts: 45,345 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	http://docs.basho.com/riak/latest/op...imit/#Mac-OS-X

04-23-2015, 08:47 AM	#9
PeterT Grand Sorcerer Posts: 13,511 Karma: 78910202 Join Date: Nov 2007 Location: Toronto Device: Libra H2O, Libra Colour	Ah; so the infamous out of handles strikes again!

04-23-2015, 08:50 AM	#10
kovidgoyal creator of calibre Posts: 45,345 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Yeah I dont know what it is with OS X systems and their very low default open file handle limits.

Advert

Advert