Fetch News failing (All strings must be XML compatible

[nuveen]

Hi all,

I've been using Calibre to download the RSS feeds from several blogs onto my Kindle into three different groups: Economics, Sociology, and Politics. Today my Economics feed stopped working, while the other two still work. It keeps saying "Python function terminated unexpectedly
All strings must be XML compatible: Unicode or ASCII, no NULL bytes (Error Code: 1)." I've searched for these terms and the answers on other threads seem to deal with non-English characters in usernames--this seems unrelated to my issue here. Here is the end of the text:

Recursion limit reached. Skipping links in file:c:\users\nuveen\appdata\local\temp\calibre_0. 7.44_tmp_rilikb\calibre_0.7.44_zcnjwv_feeds2disk.h tml
file:c:\users\nuveen\appdata\local\temp\calibre_0. 7.44_tmp_rilikb\calibre_0.7.44_zcnjwv_feeds2disk.h tml saved to c:\users\nuveen\appdata\local\temp\calibre_0.7.44_ tmp_rilikb\calibre_0.7.44_hyyb40_plumber\feed_15\a rticle_6\calibre_0.7.44_zcnjwv_feeds2disk.xhtml
Downloaded article: Are the Owners of Capital Doing Better? from http://modeledbehavior.com/2011/02/0...-doing-better/
Python function terminated unexpectedly
All strings must be XML compatible: Unicode or ASCII, no NULL bytes (Error Code: 1)
Traceback (most recent call last):
File "site.py", line 103, in main
File "site.py", line 85, in run_entry_point
File "site-packages\calibre\utils\ipc\worker.py", line 110, in main
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 25, in gui_convert
File "site-packages\calibre\ebooks\conversion\plumber.py", line 904, in run
File "site-packages\calibre\customize\conversion.py", line 204, in __call__
File "site-packages\calibre\web\feeds\input.py", line 105, in convert
File "site-packages\calibre\web\feeds\news.py", line 734, in download
File "site-packages\calibre\web\feeds\news.py", line 976, in build_index
File "site-packages\calibre\web\feeds\news.py", line 820, in feed2index
File "site-packages\calibre\web\feeds\templates.py", line 40, in generate
File "site-packages\calibre\web\feeds\templates.py", line 172, in _generate
File "site-packages\lxml\builder.py", line 222, in __call__
File "site-packages\lxml\builder.py", line 185, in add_text
File "lxml.etree.pyx", line 821, in lxml.etree._Element.text.__set__ (src/lxml/lxml.etree.c:33308)
File "apihelpers.pxi", line 646, in lxml.etree._setNodeText (src/lxml/lxml.etree.c:15287)
File "apihelpers.pxi", line 1295, in lxml.etree._utf8 (src/lxml/lxml.etree.c:20212)
ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes

Any help would be greatly appreciated.

[kovidgoyal]

You need to post your recipe if you want help.

[Starson17]

You might want to ask in the Recipes forum when you post your recipe.

[nuveen]

Thanks for the responses. I made the recipe in basic mode, but when I switch to advanced mode, here is what comes up:

class AdvancedUserRecipe1297377993(BasicNewsRecipe):
title = u'Economics'
oldest_article = 10
max_articles_per_feed = 100

feeds = [(u'Marginal Revolution', u'http://www.marginalrevolution.com/marginalrevolution/index.rdf'), (u'Felix Salmon', u'http://blogs.reuters.com/felix-salmon/feed/'), (u"Economist's View", u'http://feeds.feedburner.com/EconomistsView'), (u'Rortybomb', u'http://rortybomb.wordpress.com/feed/'), (u'Brad DeLong', u'http://feeds.feedburner.com/BradDelongsSemi-dailyJournal'), (u'Baseline Scenario', u'http://feeds.feedburner.com/BaselineScenario?format=xml'), (u'Chris Blattman', u'http://feeds.feedburner.com/chrisblattman?format=xml'), (u'The Money Illusion', u'http://feeds.feedburner.com/Themoneyillusion?format=xml'), (u'Macroeconomic Resilience', u'http://feeds.feedburner.com/MacroeconomicResilience?format=xml'), (u'Naked Capitalism', u'http://feeds.feedburner.com/NakedCapitalism'), (u'worthwhile canadian initiative', u'http://worthwhile.typepad.com/worthwhile_canadian_initi/atom.xml'), (u'Econbrowser', u'http://feeds.feedburner.com/Econbrowser'), (u'interfluidity', u'http://www.interfluidity.com/feed'), (u'Economics of Contempt', u'http://feeds.feedburner.com/economicsofcontempt'), (u'Paul Krugman', u'http://www.wizardrss.com/feed/krugman.blogs.nytimes.com/feed/'), (u'Modeled Behavior', u'http://modeledbehavior.com/feed/')]

[kovidgoyal]

One of your feeds has a form feed character in its description which is blowing up the XML parser. I'll add a workaround in the next release.

[nuveen]

Thanks for the reply. I'll try eliminating feeds one at a time and see what happens. Thanks for making a great product!

[jmaciek]

Originally Posted by kovidgoyal

I'll add a workaround in the next release.

Has this happened already? I'm getting the same issue with the following simple recipe

class AdvancedUserRecipe1311785189(BasicNewsRecipe):
    title          = u'Blogi'
    oldest_article = 1
    max_articles_per_feed = 100

    feeds          = [(u'Ziemkiewicz', u'http://blog.rp.pl/ziemkiewicz/feed')]

[kovidgoyal]

The workaround was for that specific character, your feed presumably has another invalid character.

[fluzao]

Great collection of Econ blogs, by the way!

Good luck.

[jmaciek]

Thanks, Kovid, that's what I thought.

One more check before I surrender: I trimmed the HTML using the preprocess_regexps leaving only the content <div> section. Does this mean the faulty character is in this section? In other words, is there a way to strip the document from all the risky stuff before any XML parsing starts? I though preprocess_regexps should do the job but apparently it doesn't.

BTW, what are the kinds of faulty characters I should be looking for?

[kovidgoyal]

No there's no way to strip the document of all risky characters reliably. If there were, calibre would do it automatically. Incidentally, running your recipe works for me.

[kcmtoken]

Can someone help me to download the econ blogs that Nuveen has mentioned. It would be a great help for me.