Combined Feeds

[mean_gene]

So I've been playing with yahoo pipes to merge multiple feeds into one but can't seem to get that to work consistently with calibre recipes.

I'd like to pull data from multiple sites and have it all under one section on the section list. For example, grab the latest news from fox, cnn, etc. and put it under the "latest news" section (as opposed to 2 different sections). And then do the same thing with a few technology sites and put that under a technology section. Etc, etc...

I've tried adding commas and semicolons between the urls like this:

Spoiler:

feeds = [
('Latest Headlines', 'http://feeds.foxnews.com/foxnews/latest',
'http://rss.cnn.com/rss/cnn_topstories.rss')
]

-----------------

feeds = [
('Latest Headlines', 'http://feeds.foxnews.com/foxnews/latest';
'http://rss.cnn.com/rss/cnn_topstories.rss')
]

Didn't expect that to work. I also dug through the code for the NY Times Headlines because it appears to do what I want but couldn't connect the dots.

If it's possible, throw some code my way please.

[mean_gene]

So I'm thinking I could use parse_feeds to loop through all articles looking for the feed title and move them (append and delete) to the other title.

Something like this (modified form the reusable code section)

Spoiler:

movedArticles = []
for curfeed in feeds:
delList = []
searchfeed = Feed()
for a,curarticle in enumerate(curfeed.articles):
if searchfeed.title = 'national':
movedArticles.append(curarticle)
delList.append(curarticle)
if len(delList)>0:
for d in delList:
index = curfeed.articles.index(d)
curfeed.articles[index:index+1] = []

if len(movedArticles) > 0:
pfeed = Feed()
pfeed.title = 'Recipes'
pfeed.id_counter = len(movedArticles)
pfeed.articles = movedArticles[:]
feeds.append(pfeed)

I'll try this when I get home today and tweak it a bit more.

Any thoughts?

[Starson17]

Originally Posted by mean_gene

So I'm thinking I could use parse_feeds to loop through all articles looking for the feed title and move them (append and delete) to the other title.

Something like this (modified form the reusable code section)...
Any thoughts?

The problem with trying to get the "latest news from fox, cnn, etc" all into one recipe usually has nothing to do with sorting the articles (you've already found the tools in the reusable code section that can do that.) It has to do with the fact that 1) different sites organize the article pages differently, so you need to process them with different code to extract the content you want and remove the unwanted junk, and 2) the latest news from multiple sites tends to be the same "latest news"

[mean_gene]

the latest news from multiple sites tends to be the same "latest news"

Agree completely on the redundant news issue; might not have been the best example. I'd also like to pull feeds from multiple technology sites as well.

Is there a way to process the feeds individually (within a loop maybe) or will the redundant code goof things up?

[Starson17]

Originally Posted by mean_gene

Is there a way to process the feeds individually (within a loop maybe) or will the redundant code goof things up?

What do you mean by "process the feeds individually "? The code samples describe how feed parsing is done. There are two distinct ways that feeds are parsed, automatically for RSS feeds and parse_feeds. The code samples show how to mix, match and sort the articles from the parsed feed list if you need to do that.

[mean_gene]

Ok, that tells me what I need to know; thanks. Just wanted to confirm that what I wanted to do was feasible before I spent half the day coding. I've somehow avoided python during my undergrad and grad work but the API documentation looks pretty good. Here's goes nothing...