Custom recipes (archive, read-only)

[rty]

Originally Posted by elsuave

Did a recipe for Foreign Policy (http://www.foreignpolicy.com/) ever come out? It's been mentioned a couple of times in this thread, with an unsuccessful attempt here: https://www.mobileread.com/forums/sho...&postcount=616.

If not, would anybody like to try their hand at it? RSS feed is available here: http://www.foreignpolicy.com/node/feed

Here it is: a recipe for FOREIGN POLICY.

[bobbysteel]

The Economist seems to have added an annoying "related items" in their print view. This is annoying since it is usually right in the center of an article. I've amended the recipe with the following tag, which removes the related items div. I suggest this gets added into the main distribution recipe, as I can't imagine it's useful to have those links there.

remove_tags = [dict(name=['script', 'noscript', 'title', 'iframe', 'cf_floatingcontent']),
dict(attrs={'class':['dblClkTrk', 'ec-article-info']}),
dict(name='div', attrs={'class':'related-items'})]

[elsuave]

Originally Posted by rty

Here it is: a recipe for FOREIGN POLICY.

Thank you, rty! It works like a charm.

[rty]

Originally Posted by ehsahog

Hi all,


Would it be possible for someone with knowledge to write a recipe for akihabara news (http://en.akihabaranews.com/feed)

Lots of thanks in advance!
/Anders

This site doesn't seem to have long enough articles to spend time on.

[kiklop74]

New (updated) recipe for Foreign Policy:

[elsuave]

Originally Posted by kiklop74

New recipe for Foreign Policy:

Kiklop: You appear to have included an .epub example, and not the recipe itself.

[kovidgoyal]

@bobbysteel: The related links are useful on readers that support web browsing.

[kiklop74]

I updated the post sorry for the mistake.

[bobbysteel]

Originally Posted by kovidgoyal

@bobbysteel: The related links are useful on readers that support web browsing.

Fair enough, but they're extremely annoying and flow poorly on non-browsing readers. I use a Nook mostly in the subway and can't stand seeing these links. It seems like you should offer both options really. I'm happy to use my custom recipe of course

Thanks again for all the work you've put in to this Kovid!

[schnortz]

I have not had luck finding a solution within my searches (probably not using the correct terminology.)

I have successfully modified the Cincinnati Enquirer recipe as the basis of my Appleton Post Crescent recipe as they both used similar web templates. However, I am not having luck with the following...

1. Remove the Additional Information box that comes up after a couple of paragraphs of each article. I have tried

preprocess_regexps = [
(re.compile(r'<p></p><div*.</div>', re.IGNORECASE | re.DOTALL), lambda match : r''),
]

without success.

2. Remove any RSS feeds that start with the word "Photo" or "Photos:"

Any guidance that you can give would be very helpful.

[Starson17]

Originally Posted by schnortz

I am not having luck with the following...

I enjoy answering these little puzzles, but it's a lot easier if you provide a link to the page that you are having trouble with, and a copy of the recipe you're using.

Here you are asking why this doesn't match something. Usually, that would be impossible without a link to the "something," but I do see an error in this.

1. Remove the Additional Information box that comes up after a couple of paragraphs of each article. I have tried

preprocess_regexps = [
(re.compile(r'<p></p><div*.</div>', re.IGNORECASE | re.DOTALL), lambda match : r''),
]

without success.

I assume you wanted to delete everything in the <div> tag, but you reversed the "everything." it should be ".*" not "*."

2. Remove any RSS feeds that start with the word "Photo" or "Photos:"

Any guidance that you can give would be very helpful.

I suspect you want to remove any articles that start with those words, not "feeds" - correct? You control the list of feeds.
For articles, I used to think that filter_regexps would do that job, but I never got it to work. Maybe it only works on recursed links, not the main article link.

[schnortz]

Starson17... thanks for responding.

The recipe I am using is the following (modified with your suggested change, even if it was unsuccessful). Hope I'm not violating etiquette by posting the code.

import string, re

#!/usr/bin/env python
__license__ = 'GPL v3'
__copyright__ = '2009 Kovid Goyal <kovid at kovidgoyal.net>'

from calibre.web.feeds.news import BasicNewsRecipe

class AppletonPostCrescent(BasicNewsRecipe):
title = u'Appleton Post Crescent'
oldest_article = 2
language = 'en'

__author__ = 'Joseph Kitzmiller and Sujata Raman'
max_articles_per_feed = 25
no_stylesheets = True
use_embedded_content = False
remove_javascript = True
encoding = 'cp1252'
cover_url = u'http://www.postcrescent.com/ic/assets/frontpage.pdf'
publisher = 'Appleton Post Crescent, Gannett'
category = 'news, Appleton, Fox Cities, Wisconsin'

extra_css = '''
h1{font-family:Arial,Helvetica,sans-serif; font-size:large; color:#0E5398; }
h2{color:#666666;}
.blog_title{color:#4E0000; font-family:Georgia,"Times New Roman",Times,serif; font-size:large;}
.sidebar-photo{font-family:Arial,Helvetica,sans-serif; color:#333333; font-size:30%;}
.blog_post{font-family:Arial,Helvetica,sans-serif; color:#222222; font-size:xx-small;}
.article-bodytext{font-family:Arial,Helvetica,sans-serif; font-size:xx-small; color:#222222;font-weight:normal;}
.ratingbyline{font-family:Arial,Helvetica,sans-serif; color:#333333; font-size:50%;}
.author{font-family:Arial,Helvetica,sans-serif; color:#777777; font-size:50%;}
.date{font-family:Arial,Helvetica,sans-serif; color:#777777; font-size:50%;}
.padding{font-family:Arial,Helvetica,sans-serif; font-size:70%; color:#222222; font-weight:normal;}
'''

preprocess_regexps = [
(re.compile(r'<p></p><div.*</div>', re.IGNORECASE | re.DOTALL), lambda match : r''),
]

keep_only_tags = [dict(name='div', attrs={'class':['padding','sidebar-photo']})]

remove_tags = [ dict(name=['object','link','table','embed','script', 'noscript'])
,dict(name='div',attrs={'id':["pluckcomments","StoryChat"]})
,dict(name='div',attrs={'class':['article-tools',"padding article-sidebar",'articleflex-container','poster-container','newslist','footer-container','sidebar-related','sub']})
,dict(name='p',attrs={'class':['posted','tags']})]

feeds = [(u'Breaking News', u'http://www.postcrescent.com/apps/pbcs.dll/misc?URL=/templates/RSSbreaking.pbs&mime=xml'),
(u'Latest Headlines', u'http://www.postcrescent.com/apps/pbcs.dll/misc?URL=/templates/RSSlatest.pbs&mime=xml'),
(u'Local News', u'http://www.postcrescent.com/apps/pbcs.dll/misc?URL=/templates/RSSlocal.pbs&mime=xml'),
(u'Sports', u'http://www.postcrescent.com/apps/pbcs.dll/misc?URL=/templates/RSSsports.pbs&mime=xml'),
(u'Buzz Blog', u'http://sitelife.postcrescent.com/ver1.0/Blog/BlogRss?plckBlogId=Blog:9a8980f0-f726-439c-8c4e-1dc0f788941e'),
(u'Weekend Blog', u'http://sitelife.postcrescent.com/ver1.0/Blog/BlogRss?plckBlogId=Blog:9dbf4deb-0468-41b7-a0c7-3a777c03d64c')]


def preprocess_html(self, soup):
for item in soup.findAll(style=True):
del item['style']
for item in soup.findAll(face=True):
del item['face']
return soup

And as requested... here is a link to an artichttps://www.mobileread.com/forums/newreply.php?do=newreply&noquote=1&p=989970le that has the "Additional Information box"... http://www.postcrescent.com/article/...AA&located=rss

And yes, I meant articles. Here is their Local News RSS Feed... http://www.postcrescent.com/apps/pbc...l.pbs&mime=xml As of now, there were a couple of "Photos: ..."

Thanks in advance.
[/LIST]

[rty]

Recipe for BBC Chinese

Language: Chinese (Simplified)
PS. Tested OK on B&N Nook.

[rty]

Originally Posted by sibermage

I did a search but couldn't find anything regarding the Sing Tao news site.
Is it possible to obtain the news articles for http://news.singtao.ca/toronto/

Thanks.

Here it is: Recipe for SINGTAO DAILY CANADA

Language: Chinese (Traditional)
Tested OK on B&N Nook e-reader.

Updated: Recipe updated to remove the hidden/bogus tab character that prevented the recipe to be imported into Calibre.

[rty]

Recipe for China Economic Net Magazine

Tested OK B&N Nook
Language: Chinese (Simplified)