Philadelphia Inquirer Recipe

[lgwapnitsky]

I'm new to Calibre and have started exploring the recipes. I'd like to modify the Philadelphia Inquirer recipe, as it is currently only downloading the basic RSS feed information and no link to (or info from) the full article. I've not programmed in Python before and would appreciate some guidance on how I can tackle this issue.

Thanks,
Larry

[Starson17]

Originally Posted by lgwapnitsky

I'm new to Calibre and have started exploring the recipes. I'd like to modify the Philadelphia Inquirer recipe, as it is currently only downloading the basic RSS feed information and no link to (or info from) the full article. I've not programmed in Python before and would appreciate some guidance on how I can tackle this issue.

Thanks,
Larry

http://manual.calibre-ebook.com/news.html#news

[sexymax15]

Here's the recipe for Philadelphia Inquirrer(Philly.com).It works without any problems.

Spoiler:

import re

from calibre.web.feeds.news import BasicNewsRecipe
from calibre.ebooks.chardet import xml_to_unicode
class AdvancedUserRecipe1308312288(BasicNewsRecipe):
    title          = u'Philadelphia Inquirer(Philly.com)'
    oldest_article = 15
    max_articles_per_feed = 20
    use_embedded_content = False
    remove_empty_feeds = True
    no_stylesheets = True
    remove_javascript = True

   # remove_tags_before = {'class':'article_timestamp'}
    #remove_tags_after = {'class':'graylabel'}
    keep_only_tags= [dict(name=['h1','p'])]
    remove_tags = [dict(name=['hr','dl','dt','img','meta','iframe','link','script','form','input','label']),
dict(id=['toggleConfirmEmailDiv','toggleTOS','toggleUsernameMsgDiv','toggleConfirmYear','navT1_philly','secondaryNav','navPlacement','globalPrimaryNav'
,'ugc-footer-philly','bv_footer_include','footer','header',
'container_rag_bottom','section_rectangle','contentrightside'])
,{'class':['megamenu3 megamenu','container misc','container_inner misc_inner'
,'misccontainer_left_32','headlineonly','misccontainer_middle_32'
,'misccontainer_right_32','headline formBegin',
'post_balloon','relatedlist','linkssubhead','b_sq','dotted-rule-above'
,'container','headlines-digest','graylabel','container_inner'
,'rlinks_colorbar1','rlinks_colorbar2','supercontainer','container_5col_left','container_image_left',
'digest-headline2','digest-lead','container_5col_leftmiddle',
'container_5col_middlemiddle','container_5col_rightmiddle'
,'container_5col_right','divclear','supercontainer_outer force-width',
'supercontainer','containertitle  kicker-title',
'pollquestion','pollchoice','photomore','pollbutton','container rssbox','containertitle video ',
'containertitle_image ','container_tabtwo','selected'
,'shadetabs','selected','tabcontentstyle','tabcontent','inner_container'
,'arrow','container_ad','containertitlespacer','adUnit','tracking','sitemsg_911 clearfix']}]

    extra_css             = """ 
                               h1{font-family: Georgia,serif; font-size: xx-large} 
                               
                            """


    feeds          = [(u'News', u'http://www.philly.com/philly_news.rss')]

Screenshot:

Moderator Notice
Edited to add tags.

[lgwapnitsky]

I saw that, but some of what's in the documentation isn't clearly detailed, e.g. What is "soup"? I'm not too familiar with Python and its inner workings (I'm currently programming heavily in C#).

Thanks,
Larry


Sexymax - didn't see your response.

Thanks!

[lgwapnitsky]

Originally Posted by sexymax15

Here's the recipe for Philadelphia Inquirrer(Philly.com).It works without any problems.



Screenshot:

Beautiful! Perfect! I'll try that when I get home later!

[Starson17]

Originally Posted by lgwapnitsky

I saw that, but some of what's in the documentation isn't clearly detailed, e.g. What is "soup"? I'm not too familiar with Python and its inner workings (I'm currently programming heavily in C#).

FYI, "soup" is html of the article page currently being processed which has been dropped into a database by BeautifulSoup so that you can find, remove, replace and manipulate all the html tags on that page by where they are relative to other tags (Next, Previous) were they are relative to other tags at the same level (Next Sibling, parent), etc. It's at the heart of how Calibre's recipe system scrapes web pages.

[lgwapnitsky]

[QUOTE=sexymax15;1613187]Here's the recipe for Philadelphia Inquirrer(Philly.com).It works without any problems.


question - can I add more of the Philly.com under the feeds list? I think this is how it was set up in the one provided by Calibre.

Thanks,
Larry

[sexymax15]

[QUOTE=lgwapnitsky;1614260]

Originally Posted by sexymax15

Here's the recipe for Philadelphia Inquirrer(Philly.com).It works without any problems.


question - can I add more of the Philly.com under the feeds list? I think this is how it was set up in the one provided by Calibre.

Thanks,
Larry

Just replace the feeds with your feeds.

feeds = [(u'News', u'http://www.philly.com/philly_news.rss')]

[Jetkey]

Hi folks, just joined the forum. I can't seem to get anything from the Inquirer (Philly.com) recipe. I have a bunch of other recipes that work just fine, but all I get from this one is a "News" link but no articles. I'm converting everything to mobi for my Kindle 3. Any ideas why it seems to work for some but not in my case?