|
Member
Posts: 14
Karma: 10
Join Date: Jun 2010
Device: kindle 3
|
New recipe for Kathimerini (Greek newspaper)
Here's a first shot at a recipe for the revised Kathimerini.
It only downloads today's news. Kathimerini usually updates at around 12:00 from Tuesday till Saturday and on 18:00 on Sundays (Athens time).
Without photos (2,5MB for this Sunday's edition):
Spoiler:
Code:
from calibre.web.feeds.recipes import BasicNewsRecipe
class Kathimerini(BasicNewsRecipe):
title = 'Kathimerini'
__author__ = 'jenniepet'
description = 'News from Greece'
max_articles_per_feed = 100
oldest_article = 1
publisher = 'Kathimerini'
category = 'news, GR'
language = 'el'
encoding = 'utf-8'
conversion_options = { 'linearize_tables': True}
no_stylesheets = True
remove_tags_before = dict(id='site-body')
remove_tags_after = [dict(id='social')]
remove_tags = [dict(attrs={'class':['post-tools', 'edition edition_PRINT', 'clearing-featured-img']})]
#Categories In order of Appearance: Politics-1-2-3, Greece-1-2, World-1-2, People-Specials
#Greek Economy-1-2, Business, International Economy, Real Estate
#Environment, Science, Technology, Culture-1-2, Travel, Sport
feeds = [(u'\u03A0\u03BF\u03BB\u03B9\u03C4\u03B9\u03BA\u03AE','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=2&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u03A0\u03BF\u03BB\u03B9\u03C4\u03B9\u03BA\u03AE 2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=2&type=&edition=PRINT&author=0&fromDate=&toDate=&page=1'),
(u'\u03A0\u03BF\u03BB\u03B9\u03C4\u03B9\u03BA\u03AE 3','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=2&type=&edition=PRINT&author=0&fromDate=&toDate=&page=2'),
(u'\u0395\u03BB\u03BB\u03AC\u03B4\u03B1','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=4&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u0395\u03BB\u03BB\u03AC\u03B4\u03B1 2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=4&type=&edition=PRINT&author=0&fromDate=&toDate=&page=1'),
(u'\u039A\u03CC\u03C3\u03BC\u03BF\u03C2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=5&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u039A\u03CC\u03C3\u03BC\u03BF\u03C2 2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=5&type=&edition=PRINT&author=0&fromDate=&toDate=&page=1'),
(u'\u03A0\u03C1\u03CC\u03C3\u03C9\u03C0\u03B1','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=41&cat=42&cat=43&cat=24&cat=25&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u0395\u03BB\u03BB\u03B7\u03BD\u03B9\u03BA\u03AE \u039F\u03B9\u03BA\u03BF\u03BD\u03BF\u03BC\u03AF\u03B1','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=17&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u0395\u03BB\u03BB\u03B7\u03BD\u03B9\u03BA\u03AE \u039F\u03B9\u03BA\u03BF\u03BD\u03BF\u03BC\u03AF\u03B1 2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=17&type=&edition=PRINT&author=0&fromDate=&toDate=&page=1'),
(u'\u0395\u03C0\u03B9\u03C7\u03B5\u03B9\u03C1\u03AE\u03C3\u03B5\u03B9\u03C2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=18&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u0394\u03B9\u03B5\u03B8\u03BD\u03AE\u03C2 \u039F\u03B9\u03BA\u03BF\u03BD\u03BF\u03BC\u03AF\u03B1','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=19&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'Real Estate','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=21&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u03A0\u03B5\u03C1\u03B9\u03B2\u03AC\u03BB\u03BB\u03BF\u03BD','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=6&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u0395\u03C0\u03B9\u03C3\u03C4\u03AE\u03BC\u03B7','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=7&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u03A4\u03B5\u03C7\u03BD\u03BF\u03BB\u03BF\u03B3\u03AF\u03B1','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=47&cat=48&cat=49&cat=50&cat=51&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u03A0\u03BF\u03BB\u03B9\u03C4\u03B9\u03C3\u03BC\u03CC\u03C2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=31&cat=32&cat=33&cat=34&cat=35&cat=36&cat=37&cat=38&cat=39&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u03A0\u03BF\u03BB\u03B9\u03C4\u03B9\u03C3\u03BC\u03CC\u03C2 2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=31&cat=32&cat=33&cat=34&cat=35&cat=36&cat=37&cat=38&cat=39&type=&edition=PRINT&author=0&fromDate=&toDate=&page=1'),
(u'\u03A4\u03B1\u03BE\u03AF\u03B4\u03B9\u03B1','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=11&cat=10&cat=12&cat=14&cat=15&cat=13&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u0391\u03B8\u03BB\u03B7\u03C4\u03B9\u03C3\u03BC\u03CC\u03C2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=3&type=&edition=PRINT&author=0&fromDate=&toDate=')
]
def get_cover_url(self):
import time
return 'http://s.kathimerini.gr/resources/issue-cover/%s.jpg' %time.strftime('%d-%m-%Y')
With photos (10MB for this Sunday's edition):
Spoiler:
Code:
from calibre.web.feeds.recipes import BasicNewsRecipe
class Kathimerini(BasicNewsRecipe):
title = 'Kathimerini'
__author__ = 'jenniepet'
description = 'News from Greece'
max_articles_per_feed = 100
oldest_article = 1
publisher = 'Kathimerini'
category = 'news, GR'
language = 'el'
encoding = 'utf-8'
conversion_options = { 'linearize_tables': True}
no_stylesheets = True
remove_tags_before = dict(id='site-body')
remove_tags_after = [dict(id='social')]
remove_tags = [dict(attrs={'class':['post-tools', 'edition edition_PRINT']})]
#to remove images comment the line above and uncomment the line below
# remove_tags = [dict(attrs={'class':['post-tools', 'edition edition_PRINT', 'clearing-featured-img']})]
#Categories In order of Appearance: Politics-1-2-3, Greece-1-2, World-1-2, People-Specials
#Greek Economy-1-2, Business, International Economy, Real Estate
#Environment, Science, Technology, Culture-1-2, Travel, Sport
feeds = [(u'\u03A0\u03BF\u03BB\u03B9\u03C4\u03B9\u03BA\u03AE','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=2&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u03A0\u03BF\u03BB\u03B9\u03C4\u03B9\u03BA\u03AE 2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=2&type=&edition=PRINT&author=0&fromDate=&toDate=&page=1'),
(u'\u03A0\u03BF\u03BB\u03B9\u03C4\u03B9\u03BA\u03AE 3','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=2&type=&edition=PRINT&author=0&fromDate=&toDate=&page=2'),
(u'\u0395\u03BB\u03BB\u03AC\u03B4\u03B1','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=4&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u0395\u03BB\u03BB\u03AC\u03B4\u03B1 2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=4&type=&edition=PRINT&author=0&fromDate=&toDate=&page=1'),
(u'\u039A\u03CC\u03C3\u03BC\u03BF\u03C2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=5&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u039A\u03CC\u03C3\u03BC\u03BF\u03C2 2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=5&type=&edition=PRINT&author=0&fromDate=&toDate=&page=1'),
(u'\u03A0\u03C1\u03CC\u03C3\u03C9\u03C0\u03B1','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=41&cat=42&cat=43&cat=24&cat=25&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u0395\u03BB\u03BB\u03B7\u03BD\u03B9\u03BA\u03AE \u039F\u03B9\u03BA\u03BF\u03BD\u03BF\u03BC\u03AF\u03B1','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=17&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u0395\u03BB\u03BB\u03B7\u03BD\u03B9\u03BA\u03AE \u039F\u03B9\u03BA\u03BF\u03BD\u03BF\u03BC\u03AF\u03B1 2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=17&type=&edition=PRINT&author=0&fromDate=&toDate=&page=1'),
(u'\u0395\u03C0\u03B9\u03C7\u03B5\u03B9\u03C1\u03AE\u03C3\u03B5\u03B9\u03C2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=18&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u0394\u03B9\u03B5\u03B8\u03BD\u03AE\u03C2 \u039F\u03B9\u03BA\u03BF\u03BD\u03BF\u03BC\u03AF\u03B1','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=19&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'Real Estate','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=21&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u03A0\u03B5\u03C1\u03B9\u03B2\u03AC\u03BB\u03BB\u03BF\u03BD','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=6&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u0395\u03C0\u03B9\u03C3\u03C4\u03AE\u03BC\u03B7','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=7&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u03A4\u03B5\u03C7\u03BD\u03BF\u03BB\u03BF\u03B3\u03AF\u03B1','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=47&cat=48&cat=49&cat=50&cat=51&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u03A0\u03BF\u03BB\u03B9\u03C4\u03B9\u03C3\u03BC\u03CC\u03C2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=31&cat=32&cat=33&cat=34&cat=35&cat=36&cat=37&cat=38&cat=39&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u03A0\u03BF\u03BB\u03B9\u03C4\u03B9\u03C3\u03BC\u03CC\u03C2 2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=31&cat=32&cat=33&cat=34&cat=35&cat=36&cat=37&cat=38&cat=39&type=&edition=PRINT&author=0&fromDate=&toDate=&page=1'),
(u'\u03A4\u03B1\u03BE\u03AF\u03B4\u03B9\u03B1','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=11&cat=10&cat=12&cat=14&cat=15&cat=13&type=&edition=PRINT&author=0&fromDate=&toDate='),
(u'\u0391\u03B8\u03BB\u03B7\u03C4\u03B9\u03C3\u03BC\u03CC\u03C2','http://www.kathimerini.gr/rss?i=news.el.search&q=&t=0&w=&c=&s=p&cat=3&type=&edition=PRINT&author=0&fromDate=&toDate=')
]
def get_cover_url(self):
import time
return 'http://s.kathimerini.gr/resources/issue-cover/%s.jpg' %time.strftime('%d-%m-%Y')
A few questions:
I don't know how to add the cover image. Can anyone help? The cover picture for yesterday, February 1st, 2014, was "http://s.kathimerini.gr/resources/issue-cover/01-02-2014.jpg"
I tried using only one RSS feed, but it wouldn't download more than 30 articles. So, I added &page=1 etc. to get the older articles. Is there a better way to do that?
Is there a way to include cartoons but not other images? Apparently, the only thing that differentiates them from other images is that cartoon pages include the class "article_SKETCH" in the <body> tag. I removed all images with:
Code:
remove_tags = [dict(attrs={'class':['clearing-featured-img']})]
Last edited by jennie; 02-06-2014 at 07:54 AM.
Reason: Updated recipe code
|