Custom recipes (archive, read-only)

[nickredding]

Mobipocket reader, also Kindle. I could try EPUB, but I doubt that's where the issue is. I did a debug pipeline and the emdash has been replaced in the input directory, before any of the output-specific processing is performed. I think the substitution must be happening in BeautifulSoup.

[kovidgoyal]

The news download system replaces entities with their UTF-8 equivalents. That's expected. Are you saying they're being saved as cp1252 in the input sub directory?

[nickredding]

The original news source looks like

this — that

and it shows up in the input dirctory of the debug pipeline as

this — that

where those three characters are the three UTF-8 byte codes that represent emdash. I'm looking at the input directory with MS Expression Web. I guess I'm not understanding how these three UTF-8 byte codes are supposed to get back to an emdash for display on a device via Mobipocket reader, Kindle, MS Expression Web, or anything else.

[kovidgoyal]

Basically the UTF-8 byte sequence for an emdash is rendered as an emdash by viewers that understand UTF-8 and have the necessary fonts to render the character.

Does the resultant MOBI display correctly in the calibre viewer?

[kiklop74]

Originally Posted by nickredding

I'm having a problem with a news feed that has emdash's included literally (instead of using — ) and they are being handled as follows: the emdash is recognized as such, and translated into a unicode emdash (u2014) which then turns up in the output as the UTF-8 equivalent (0xE2 0x80 0x94) and is displayed as — which is the CP1252 interpretation of those three character codes. I can't figure out how to fix this -- preprocess_regexps doesn't work. Can anyone help?

Would you mind sending your code?

[nickredding]

Basically the UTF-8 byte sequence for an emdash is rendered as an emdash by viewers that understand UTF-8 and have the necessary fonts to render the character.

Does the resultant MOBI display correctly in the calibre viewer?

No -- it shows — just like Mobipocket reader, Kindle and Internet Explorer (from the input directory of the debug pipeline)

[nickredding]

Would you mind sending your code?

I was running the standard recipe for the Vancouver Province. The article in question is off the index page now, but if you run the recipe on
http://www.theprovince.com/sports/20...576/story.html you'll see the problem--two emdashes in the body of the article

[kovidgoyal]

That webpage incorrectly declares its encoding to be is8859-1, when it is actually utf-8. Set encoding='utf-8' in your recipe.

[nickredding]

That webpage incorrectly declares its encoding to be is8859-1, when it is actually utf-8. Set encoding='utf-8' in your recipe.

Bingo -- thank you very much!

[nickredding]

How did you deduce the encoding is utf-8?

[kovidgoyal]

Simple, I tried decoding it using utf-8, and the emdash was correctly decoded. I use a program called iconv to do this conveniently, but you can use calibre-debug as well

[mrgrossm]

I created recipes for both the Detroit News and Free Press, but I can't get it right! The biggest problem is that both have a background, the News one is light enough, but the Free Press is really dark. Also both have lots of junk after the article that I don't know how to get rid of.

Can anybody help?

[macrogeek]

I used the editor to make a quick and dirty recipe for Kukuburi.com.
I'm pretty happy w/ the result, but can't seem to export the recipe from Calibre.

Would anyone like to clean it up and save it as a file?
I didn't know how to trim the bottom buttons out of the feed.

class AdvancedUserRecipe1267141443(BasicNewsRecipe):

title = u'Kukuburi'

oldest_article = 30

max_articles_per_feed = 100

feeds = [(u'http://feeds.feedburner.com/kukuburi?format=xml', u'http://feeds.feedburner.com/kukuburi?format=xml')]

[aoitenshi]

Can I request for receipe for TODAY online (SG) ?

http://www.todayonline.com/RSS

[moriakaice]

Originally Posted by macrogeek

I used the editor to make a quick and dirty recipe for Kukuburi.com.
I'm pretty happy w/ the result, but can't seem to export the recipe from Calibre.

Would anyone like to clean it up and save it as a file?
I didn't know how to trim the bottom buttons out of the feed.

Here, this should do the job:

#!/usr/bin/env  python

__license__ = 'GPL v3'
__author__ = 'Mori'
__version__ = 'v. 0.1'
'''
Kukuburi.com
'''

from calibre.web.feeds.news import BasicNewsRecipe
import re

class KukuburiRecipe(BasicNewsRecipe):
	__author__ = 'Mori'
	language = 'en'

	title = u'Kukuburi'
	publisher = u'Ramón Pérez'
	description = u'KUKUBURI by Ramón Pérez'
	
	no_stylesheets = True
	remove_javascript = True
	
	oldest_article = 100
	max_articles_per_feed = 100
	
	feeds = [
		(u'Kukuburi', u'http://feeds2.feedburner.com/Kukuburi')
	]
	
	preprocess_regexps = [
		(re.compile(i[0], re.IGNORECASE | re.DOTALL), i[1]) for i in 
		[
			(r'<!--.*?-->', lambda match: ''),
			(r'<div class="feedflare".*?</div>', lambda match: '')
		]
	]