the feed only contains a link ...

[alexxxm]

hi Kovid,
I'm writing a brief recipe for an italian newspaper, http://www.repubblica.it .

Using the simplified interface, I see that the feed just contains links to the articles (http://www.repubblica.it/rss/homepage/rss2.0.xml).

In the manual, you mention this as a typical problem (Now we will look at a news source that does not provide full content feeds. In such feeds, the full article is a webpage and the feed only contains a link to the webpage with a short summary of the article.) - but then you forgot to explain how to solve it, explaining instead how to obtain the prionted version...

maybe it's a dumb question, but python is really a stranger to me so maybe if you could explain the basic steps...
thanks!

alessandro

[kiklop74]

On this page you have all the feeds from that newspaper

http://www.repubblica.it/servizi/rss...tml?ref=hpfoot

[alexxxm]

yes, in fact I took them exactly there.

Then, when you insert any of them in the simplified calibre feed interface, and download the newspaper, what you get is just a brief list of headlines, each pointing to the corresponding article.
What I need is the hint on how to modify the recipe - in the advanced interface - to substitute those with their links.

alessandro

[kiklop74]

Show us your code.

[alexxxm]

the code? Nothing to see there, in fact it's just the default produced by calibre:

class AdvancedUserRecipe1234959710(BasicNewsRecipe):
    title          = u'la Repubblica'
    oldest_article = 1
    max_articles_per_feed = 100
    
    feeds          = [(u'Repubblica homepage', u'http://www.repubblica.it/rss/homepage/rss2.0.xml'), (u'Repubblica Scienze', u'http://www.repubblica.it/rss/scienze/rss2.0.xml'), (u'Repubblica Tecnologia', u'http://www.repubblica.it/rss/tecnologia/rss2.0.xml'), (u'Repubblica Esteri', u'http://www.repubblica.it/rss/esteri/rss2.0.xml')]

alessandro

[kiklop74]

I tried this recipe and it gets complete articles. You just need to filter out the trash.

[kiklop74]

Here is extended recipe that filters out the extra trash:

class AdvancedUserRecipe1234959710(BasicNewsRecipe):
    title          = u'la Repubblica'
    oldest_article = 1
    max_articles_per_feed = 100
    remove_javascript = True
    no_stylesheets = True
    
    keep_only_tags     = [dict(name='div', attrs={'class':'articolo'})]


    remove_tags        = [
                            dict(name=['object','link'])
                           ,dict(name='span',attrs={'class':'linkindice'})
                           ,dict(name='div',attrs={'class':'bottom-mobile'})
                           ,dict(name='div',attrs={'id':['rssdiv','blocco']})
                         ]
    
    feeds          = [(u'Repubblica homepage', u'http://www.repubblica.it/rss/homepage/rss2.0.xml'), (u'Repubblica Scienze', u'http://www.repubblica.it/rss/scienze/rss2.0.xml'), (u'Repubblica Tecnologia', u'http://www.repubblica.it/rss/tecnologia/rss2.0.xml'), (u'Repubblica Esteri', u'http://www.repubblica.it/rss/esteri/rss2.0.xml')]

[alexxxm]

thanks a lot, it works fine now!


alessandro