Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Recipes

Notices

Reply
 
Thread Tools Search this Thread
Old 12-09-2010, 05:46 PM   #1
mean_gene
Enthusiast
mean_gene began at the beginning.
 
Posts: 41
Karma: 10
Join Date: Dec 2010
Device: Kindle 3G/Wifi
Combined Feeds

So I've been playing with yahoo pipes to merge multiple feeds into one but can't seem to get that to work consistently with calibre recipes.

I'd like to pull data from multiple sites and have it all under one section on the section list. For example, grab the latest news from fox, cnn, etc. and put it under the "latest news" section (as opposed to 2 different sections). And then do the same thing with a few technology sites and put that under a technology section. Etc, etc...

I've tried adding commas and semicolons between the urls like this:

Spoiler:
feeds = [
('Latest Headlines', 'http://feeds.foxnews.com/foxnews/latest',
'http://rss.cnn.com/rss/cnn_topstories.rss')
]

-----------------

feeds = [
('Latest Headlines', 'http://feeds.foxnews.com/foxnews/latest';
'http://rss.cnn.com/rss/cnn_topstories.rss')
]


Didn't expect that to work. I also dug through the code for the NY Times Headlines because it appears to do what I want but couldn't connect the dots.

If it's possible, throw some code my way please.
mean_gene is offline   Reply With Quote
Old 12-10-2010, 09:52 AM   #2
mean_gene
Enthusiast
mean_gene began at the beginning.
 
Posts: 41
Karma: 10
Join Date: Dec 2010
Device: Kindle 3G/Wifi
So I'm thinking I could use parse_feeds to loop through all articles looking for the feed title and move them (append and delete) to the other title.

Something like this (modified form the reusable code section)

Spoiler:
movedArticles = []
for curfeed in feeds:
delList = []
searchfeed = Feed()
for a,curarticle in enumerate(curfeed.articles):
if searchfeed.title = 'national':
movedArticles.append(curarticle)
delList.append(curarticle)
if len(delList)>0:
for d in delList:
index = curfeed.articles.index(d)
curfeed.articles[index:index+1] = []

if len(movedArticles) > 0:
pfeed = Feed()
pfeed.title = 'Recipes'
pfeed.id_counter = len(movedArticles)
pfeed.articles = movedArticles[:]
feeds.append(pfeed)


I'll try this when I get home today and tweak it a bit more.

Any thoughts?
mean_gene is offline   Reply With Quote
Old 12-10-2010, 10:15 AM   #3
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by mean_gene View Post
So I'm thinking I could use parse_feeds to loop through all articles looking for the feed title and move them (append and delete) to the other title.

Something like this (modified form the reusable code section)...
Any thoughts?
The problem with trying to get the "latest news from fox, cnn, etc" all into one recipe usually has nothing to do with sorting the articles (you've already found the tools in the reusable code section that can do that.) It has to do with the fact that 1) different sites organize the article pages differently, so you need to process them with different code to extract the content you want and remove the unwanted junk, and 2) the latest news from multiple sites tends to be the same "latest news"
Starson17 is offline   Reply With Quote
Old 12-10-2010, 11:06 AM   #4
mean_gene
Enthusiast
mean_gene began at the beginning.
 
Posts: 41
Karma: 10
Join Date: Dec 2010
Device: Kindle 3G/Wifi
Quote:
the latest news from multiple sites tends to be the same "latest news"
Agree completely on the redundant news issue; might not have been the best example. I'd also like to pull feeds from multiple technology sites as well.

Is there a way to process the feeds individually (within a loop maybe) or will the redundant code goof things up?
mean_gene is offline   Reply With Quote
Old 12-10-2010, 11:53 AM   #5
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by mean_gene View Post
Is there a way to process the feeds individually (within a loop maybe) or will the redundant code goof things up?
What do you mean by "process the feeds individually "? The code samples describe how feed parsing is done. There are two distinct ways that feeds are parsed, automatically for RSS feeds and parse_feeds. The code samples show how to mix, match and sort the articles from the parsed feed list if you need to do that.
Starson17 is offline   Reply With Quote
Old 12-10-2010, 11:59 AM   #6
mean_gene
Enthusiast
mean_gene began at the beginning.
 
Posts: 41
Karma: 10
Join Date: Dec 2010
Device: Kindle 3G/Wifi
Ok, that tells me what I need to know; thanks. Just wanted to confirm that what I wanted to do was feasible before I spent half the day coding. I've somehow avoided python during my undergrad and grad work but the API documentation looks pretty good. Here's goes nothing...
mean_gene is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Alien Murders by Stephen Goldin--a science fiction murder mystery sgoldin Self-Promotions by Authors and Publishers 5 07-04-2013 05:22 PM
periodically email combined RSS feeds to Kindle account? bspooky Amazon Kindle 12 11-05-2010 12:33 AM
PRS-300 RSS Feeds denmarks Sony Reader 1 10-06-2009 01:41 PM
Lots of files combined into one - most efficient way? Rasczak Amazon Kindle 7 01-04-2009 02:03 PM
RSS feeds ichor iRex 1 03-01-2008 11:30 PM


All times are GMT -4. The time now is 11:09 PM.


MobileRead.com is a privately owned, operated and funded community.