Problem with preprocess_regexps and Unicode

[mccande]

I am preparing a recipe for a Belgian newspaper where I have to replace a styled apostrophe with a simple one (Unicode characters 0x92 and 0x27)

The formula I use is

preprocess_regexps = [
(re.compile(ru'\0092'), lambda match: ru'\u0027')
]

but I cannot get the epub2disk to start. I always receive the standard error message
C:\Documents and Settings\Denis\test>feeds2disk --debug --test libe.py
Traceback (most recent call last):
File "main.py", line 167, in <module>
File "main.py", line 162, in main
File "main.py", line 133, in run_recipe
File "calibre\web\feeds\recipes\__init__.pyo", line 80, in compile_recipe
File "c:\docume~1\denis\locals~1\temp\calibre_0.4.115_s _e8f1_recipes\recipe1.p
y", line 4, in <module>
libe.py
NameError: name 'libe' is not defined

What is wrong with the use of regexp?

[kovidgoyal]

post the full recipe

[mccande]

Here is the recipe which works without the regex part.

[kovidgoyal]

The first thing I see wrong is

(re.compile(ru'\0092'), lambda match: ru'\u0027')
should be

(re.compile(ru'\u0092'), lambda match: ru'\u0027')

Note the missing u

[mccande]

Thanks but it still does not work

[kiklop74]

Originally Posted by mccande

Thanks but it still does not work

You are missing this at the start of your script:

import string, re

class AdvancedUserRecipe1229426345(BasicNewsRecipe):
....

[mccande]

This still does not start

import string, re
class AdvancedUserRecipe1229426345(BasicNewsRecipe):
title = u'La Libre Belgique'
__author__ = 'Denis McCann'
oldest_article = 1
max_articles_per_feed = 100
use_embedded_content = False
no_stylesheets = True
simultaneous_downloads = 1

remove_tags_after = [dict(id='articleText')]

preprocess_regexps = [
(re.compile(ru'\u0092'), lambda match: ru'\u0027')
]


keep_only_tags = [
dict(name='p', attrs={'id':'avantTitre'}),
dict(name='p', attrs={'id':'writer'}),
dict(name='p', attrs={'id':'publicationDate'}),
dict(name='div', attrs={'id':'articleHat'}),
dict(name='div', attrs={'id':'c'}),
dict(name='div', attrs={'id':'articleText'})
]

feeds = [
(u'A la Une', u'http://www.lalibre.be/rss/?section=10'),
(u'Belgique', u'http://www.lalibre.be/rss/?section=10&subsection=90'),
(u'Europe', u'http://www.lalibre.be/rss/?section=10&subsection=91'),
(u'Bruxelles', u'http://www.lalibre.be/rss/?section=10&subsection=1083'),
(u'Brabant', u'http://www.lalibre.be/rss/?section=10&subsection=1106'),
(u'Economie', u'http://www.lalibre.be/rss/?section=3'),
(u'Opinion', u'http://www.lalibre.be/rss/?section=11&subsection=118')
]

[kiklop74]

Change the regular expression to look like this and it will work:

    
preprocess_regexps = [(re.compile(u'\u0092'), lambda match: u'\u0027')]

Note the absence of r. String can be unicode or raw but not both.

[mccande]

Thanks a lot. That works and will be useful for other feeds.

The syntax of this function is far from obvious.