Custom recipes (archive, read-only)

[kovidgoyal]

@Spankpoo: Just override get_article_url and return None for those articles you want to exclude.

[evanmaastrigt]

Here is a new recipe for nrcnext, a newsblog from Dutch daily newspaper nrcnext. It does not use their RSS feeds because these are sometimes out of date (specially in the morning).

nrcnext.zip

[Spankypoo]

Originally Posted by kovidgoyal

@Spankpoo: Just override get_article_url and return None for those articles you want to exclude.

Thanks! And if I want to include anything containing a string?

[evanmaastrigt]

Originally Posted by fortunados

If anyone could cook a recipe of just give me any hints I would apprecciate.

Well, it turns out to be very simple after all. All the RSS feeds of this paper have a typo: they have a double slash in them. Most browsers ignore that, but Calibre's browser does not. If you correct it, it becomes a very simple recipe.

farodevigo.zip

About that flash movie: you don't get to see it because you are a regular visitor; it is only shown to new visitors. But what happens is that in fact the whole page is loaded, and some javascript overlayers the page with that movie if some cookie is not set. But that's a guess, their code is in Spanish, and my Spanish is not what it used to be.

Feel free to post back your final recipe if might interest other people.

[fortunados]

Thanks a lot for you effort !!!!!!!!

I am going to try right away and come back...

I did it. I checked it , It was so easy, but what is unabelieveble for me is
How did you realize ???

I was even checking the page with telnet with my limited knoledges.

I will coock a recipe with th rest of the sections, not very nive formatted as I dont have idea about but for sure withh all the sections.

Once again thank a lot for solving this problem that drove me crazy for days.!!!!

Originally Posted by evanmaastrigt

Well, it turns out to be very simple after all. All the RSS feeds of this paper have a typo: they have a double slash in them. Most browsers ignore that, but Calibre's browser does not. If you correct it, it becomes a very simple recipe.

Attachment 39876

About that flash movie: you don't get to see it because you are a regular visitor; it is only shown to new visitors. But what happens is that in fact the whole page is loaded, and some javascript overlayers the page with that movie if some cookie is not set. But that's a guess, their code is in Spanish, and my Spanish is not what it used to be.

Feel free to post back your final recipe if might interest other people.

[Swoard]

Hi,

may be someone can help with this one:
http://frank-meyer.eu/blog/
and here the rss feed: http://frank-meyer.eu/blog/index.php?tempskin=_rss2

I get the book made with the heading of each topic. :-)
But when I click the link, then it is empty. :-(
Calibre does not follow the link to extract the text.
May be someone can help.

Thanks
Swoard

[evanmaastrigt]

Here is a new recipe for the Spanish newspaper Levante. Just comment out the feeds you don't want. By 'comment out' I mean putting a single '#' in front of the appropriate 'feeds.append(...) line.

There is one issue: feed titles are without accented characters. Once you have the recipe in calibre you will have to correct it there. This has something to do with my editor. I will look into it later.

levante.zip

[kovidgoyal]

Safest way to handle special characters in python scripts is to just use the encoded for like this u'a\u0062cd' where \u0062 is the code for b

[evanmaastrigt]

Originally Posted by kovidgoyal

Safest way to handle special characters in python scripts is to just use the encoded for like this u'a\u0062cd' where \u0062 is the code for b

Muchas Gracias for the tip

[dogncow]

First of all, I want to give a big THANK YOU to Kovid Goyal and everybody who help making calibre.

Calibre includles the recipe for 'the Economist' but this recipe is only available for Economist subscribers who can access 'printed edidtion' pages.

Because my subscribtion just expired few days ago, I tried to access other pages and I found this RSS page. Everybody can access this RSS page.

http://feeds.feedburner.com/economis..._print_edition

It has descriptions of latest news and links to 'full articles.' It would great if Calibre recognizes the list of latest news and downloads its full articles. This RSS page might be worse than the 'printed edition' page, but it is still good news source for poor people who dosen't have subscibtion. :-)
If anyone has time to take a look at this one. I'd appreciate it. Thanks!

[kovidgoyal]

@dogncow: Sure will be in next release.

[dogncow]

Originally Posted by kovidgoyal

@dogncow: Sure will be in next release.

Thank you very much. I'll look forward to next version.

[fortunados]

Well I have tryed the recipe and is working ok with all the sections. It is not perfect formatted and I don´t know what most of the sections does but is working, and even if it is a local newspaper I put ut here just in case Someone is interested.

It is normal that takes so long to get the recipe ?

It is there any way to get the recipe cooked in EPUB even if the default settings for calibre is LRF ??



Las section of def preprocess_html I don't know what is doing but I realized the many of the recipes have the section so I used it.

Even if remove_tags is empty I left it there just in case that in the future something not whised appears on the result.

Thanks

Recipe

#!/usr/bin/env  python

__license__   = 'GPL v3'
__copyright__ = '2009, Jos <nomedeslabrasa at gmail.com>'
'''
elmundo.es
'''

from calibre.web.feeds.news import BasicNewsRecipe

class FarodeVigo(BasicNewsRecipe):
    title                 = 'Faro de Vigo'
    __author__            = 'Jos'
    description           = 'Noticias de Vigo'
    publisher             = 'Faro de Vigo'
    category              = 'Noticias'
    oldest_article        = 7
    max_articles_per_feed = 100
    no_stylesheets        = True
    use_embedded_content  = False
    encoding              = 'latin1'
    cover_url             = 'http://www.farodevigo.es/elementosWeb/mediaweb/images/iconos/logo2.jpg'
    remove_javascript     = True

    html2lrf_options = [
                          '--comment', description
                        , '--category', category
                        , '--publisher', publisher
                        ]

    html2epub_options = 'publisher="' + publisher + '"\ncomments="' + description + '"\ntags="' + category + '"'


    keep_only_tags = [dict(name='div', attrs={'class' : 'noticia_titular'}),
                              dict(name='div', attrs={'class' : 'subtitulo'}),
                              dict(name='div', attrs={'class' : 'cuadro_multimedia'}),
                              dict(name='div', attrs={'id' : 'noticia_texto', 'class' : 'noticia_texto'})]
    remove_tags = [
                  ]

    feeds = [ (u'Vigo', u'http://www.farodevigo.es/elementosInt/rss/1'					)
				,(u'Gran Vigo', u'http://www.farodevigo.es/elementosInt/rss/2'				)
				,(u'Al minuto', u'http://www.farodevigo.es/elementosInt/rss/AlMinuto'		)
				,(u'Galicia', u'http://www.farodevigo.es/elementosInt/rss/4'				)
				,(u'Comarcas', u'http://www.farodevigo.es/elementosInt/rss/3'				)
				,(u'Pontevedra', u'http://www.farodevigo.es/elementosInt/rss/15'			)
				,(u'Ourense', u'http://www.farodevigo.es/elementosInt/rss/16'				)
				,(u'Arosa', u'http://www.farodevigo.es/elementosInt/rss/17'				)
				,(u'Morrazo', u'http://www.farodevigo.es/elementosInt/rss/18'				)
				,(u'Deza-Tabeirós-Montes', u'http://www.farodevigo.es/elementosInt/rss/19'	)
				,(u'España', u'http://www.farodevigo.es/elementosInt/rss/6'				)
				,(u'Mundo', u'http://www.farodevigo.es/elementosInt/rss/7'					)
				,(u'Opinión', u'http://www.farodevigo.es/elementosInt/rss/5'				)
				,(u'Economía', u'http://www.farodevigo.es/elementosInt/rss/10'				)
				,(u'Sociedad y Cultura', u'http://www.farodevigo.es/elementosInt/rss/8'	)
				,(u'Sucesos', u'http://www.farodevigo.es/elementosInt/rss/9'				)
				,(u'Deportes', u'http://www.farodevigo.es/elementosInt/rss/11'				)
				,(u'Agenda', u'http://www.farodevigo.es/elementosInt/rss/21'				)
				,(u'Gente', u'http://www.farodevigo.es/elementosInt/rss/24'				)
				,(u'Televisión', u'http://www.farodevigo.es/elementosInt/rss/25'			)
				,(u'Ciencia y tecnología', u'http://www.farodevigo.es/elementosInt/rss/26'	)
				,(u'Humor', u'http://www.farodevigo.es/elementosInt/rss/12'				)
				,(u'Última', u'http://www.farodevigo.es/elementosInt/rss/13'				)
				,(u'Cartas', u'http://www.farodevigo.es/elementosInt/rss/20'				)
            ]

    def preprocess_html(self, soup):
        for item in soup.findAll(style=True):
            del item['style']
        return soup

    language = 'es'

[olaf]

Question - is there a way to directly copy a calibre-created newspaper to the Kindle, and have previous versions go to the archive? If I email to my Kindle address, it will do that (for a fee each day), but other than naming the issue differently (like appending the date), I don't know if there's a way to do it with a copy via cable, without simply overlaying the issue that was there.

[kovidgoyal]

each new download should have a different filename.