Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Recipes

Notices

Reply
 
Thread Tools Search this Thread
Old 10-09-2011, 03:49 PM   #1
jzall
Junior Member
jzall began at the beginning.
 
Posts: 2
Karma: 10
Join Date: May 2011
Device: nookColor
NYTimes recipe skipping articles

The NY Times recipe seems to be skipping articles lately. In today's Front Page section (9 Oct), for example, there are 6 articles in that section on the website, but only 4 show up in the Calibre download. Does anyone have any suggestions for fixing this? Thanks.
jzall is offline   Reply With Quote
Old 12-01-2012, 09:49 AM   #2
BobbyVan
Member
BobbyVan began at the beginning.
 
Posts: 20
Karma: 20
Join Date: Jan 2012
Device: Kindle Paperwhite
Quote:
Originally Posted by jzall View Post
The NY Times recipe seems to be skipping articles lately. In today's Front Page section (9 Oct), for example, there are 6 articles in that section on the website, but only 4 show up in the Calibre download. Does anyone have any suggestions for fixing this? Thanks.
This has been a periodic problem for me for quite a while now. Today only one article is showing up in the recipe, whereas the website (http://www.nytimes.com/pages/todayspaper/index.html) is showing 6 front page articles. It appears that this happens because the "Front Page" section is formatted differently than the other sections (including each "lede" as well as links to comment sections).

Here's a snippet from my log file for a few of the Front Page articles from today that failed to download:

Quote:
http://www.nytimes.com/2012/12/01/bu...pagewanted=all
Downloading
Fetching http://www.nytimes.com/2012/12/01/us...pagewanted=all
Fetching http://www.nytimes.com/2012/12/01/bu...pagewanted=all
Fetching http://www.nytimes.com/2012/12/01/wo...pagewanted=all
Found forwarding link: /2012/12/01/business/a-hospital-war-reflects-a-tightening-bind-for-doctors-nationwide.html?adxnnl=1&pagewanted=all&adxnnlx=13 54374508-I536HSB8AoTB7E+3dlGiDg
Skipping ad to article at 'http://www.nytimes.com/2012/12/01/business/a-hospital-war-reflects-a-tightening-bind-for-doctors-nationwide.html?pagewanted=all'
Found forwarding link: /2012/12/01/us/dream-act-gives-young-immigrants-a-political-voice.html?adxnnl=1&pagewanted=all&adxnnlx=1354374 508-I536HSB8AoTB7E+3dlGiDg
Skipping ad to article at 'http://www.nytimes.com/2012/12/01/us/dream-act-gives-young-immigrants-a-political-voice.html?pagewanted=all'
Found forwarding link: /2012/12/01/world/middleeast/israel-moves-to-expand-settlements-in-east-jerusalem.html?adxnnl=1&pagewanted=all&adxnnlx=135 4374427-+mg8fvhpducBZ7vP0d08XA
Found forwarding link: /2012/12/01/world/africa/south-africa-corruption-fuels-battle-for-political-spoils.html?adxnnl=1&pagewanted=all&adxnnlx=135437 4507-TM0xVHzZl0ftY2tWlwhdlA
Found forwarding link: /2012/12/01/business/online-retailers-rush-to-adjust-prices-in-real-time.html?adxnnl=1&pagewanted=all&adxnnlx=13543745 08-I536HSB8AoTB7E+3dlGiDg
Skipping ad to article at 'http://www.nytimes.com/2012/12/01/world/middleeast/israel-moves-to-expand-settlements-in-east-jerusalem.html?pagewanted=all'
Skipping ad to article at 'http://www.nytimes.com/2012/12/01/business/online-retailers-rush-to-adjust-prices-in-real-time.html?pagewanted=all'Skipping ad to article at 'http://www.nytimes.com/2012/12/01/world/africa/south-africa-corruption-fuels-battle-for-political-spoils.html?pagewanted=all'

Could not fetch link http://www.nytimes.com/2012/12/01/bu...pagewanted=all
Traceback (most recent call last):
File "site-packages\calibre\web\fetch\simple.py", line 474, in process_links
File "site-packages\calibre\web\fetch\simple.py", line 163, in get_soup
File "site-packages\calibre\ebooks\chardet.py", line 109, in xml_to_unicode
File "site-packages\calibre\ebooks\chardet.py", line 73, in detect_xml_encoding
TypeError: 'NoneType' object is not callable

http://www.nytimes.com/2012/12/01/bu...pagewanted=all saved to
Downloading
Fetching http://www.nytimes.com/2012/12/01/sp...pagewanted=all
Could not fetch link http://www.nytimes.com/2012/12/01/wo...pagewanted=all
Traceback (most recent call last):
File "site-packages\calibre\web\fetch\simple.py", line 474, in process_links
File "site-packages\calibre\web\fetch\simple.py", line 163, in get_soup
File "site-packages\calibre\ebooks\chardet.py", line 109, in xml_to_unicode
File "site-packages\calibre\ebooks\chardet.py", line 73, in detect_xml_encoding
TypeError: 'NoneType' object is not callable

http://www.nytimes.com/2012/12/01/wo...pagewanted=all saved to
Failed to download article: Retail Frenzy: Prices on the Web Change Hourly from http://www.nytimes.com/2012/12/01/bu...pagewanted=all
Traceback (most recent call last):
File "site-packages\calibre\utils\threadpool.py", line 95, in run
File "site-packages\calibre\web\feeds\news.py", line 1017, in fetch_article
File "site-packages\calibre\web\feeds\news.py", line 1012, in _fetch_article
Exception: Could not fetch article. The debug traceback is available earlier in this log
Thanks to everyone who keeps Calibre running and the recipes working!

Last edited by BobbyVan; 12-01-2012 at 10:19 AM. Reason: added log info
BobbyVan is offline   Reply With Quote
 
Enthusiast
Old 12-03-2012, 07:34 PM   #3
nickredding
onlinenewsreader.net
nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'nickredding knows the difference between 'who' and 'whom'
 
Posts: 318
Karma: 10143
Join Date: Dec 2009
Location: Phoenix, AZ & Victoria, BC
Device: Kindle 3, Kindle Fire, IPad3, iPhone4, Playbook, HTC Inspire
There is an update coming that will fix this
nickredding is offline   Reply With Quote
Old 12-03-2012, 11:06 PM   #4
BobbyVan
Member
BobbyVan began at the beginning.
 
Posts: 20
Karma: 20
Join Date: Jan 2012
Device: Kindle Paperwhite
THANK YOU!!!

BobbyVan is offline   Reply With Quote
Reply

Tags
nytimes

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
MacWorld recipe - only headlines - no articles simonz Recipes 4 06-04-2011 09:02 AM
Recipe gets headings but not full articles stanleti Recipes 0 04-19-2011 03:24 PM
Truncation of the NYTimes Headlines recipe Nanoox Recipes 7 03-05-2011 10:49 PM
ReadItLater recipe only downloads 10 saved articles? usuario74 Recipes 1 02-20-2011 04:24 PM
Reversing articles order in a custom news recipe? mairabc Calibre 5 12-12-2009 05:24 PM


All times are GMT -4. The time now is 02:45 AM.


MobileRead.com is a privately owned, operated and funded community.