Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > Miscellaneous > Lounge

Notices

Reply
 
Thread Tools Search this Thread
Old 09-21-2005, 09:33 PM   #1
Alexander Turcic
Fully Converged
Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.
 
Alexander Turcic's Avatar
 
Posts: 17,486
Karma: 10995944
Join Date: Oct 2002
Location: Switzerland
Device: Sony PRS-650 / Nexus 7 / Kindle PW
Track Yahoo Finance message boards via RSS

I just wrote YahooMsg2RSS, a new tool which allows you to track Yahoo Finance message boards via RSS feeds. You can use it for instance to track everything that is going on in the Palm message board.

Your suggestions are always welcomed.
Alexander Turcic is offline   Reply With Quote
Old 09-22-2005, 10:58 AM   #2
Unregistered
Nameless Being
 
very nice! i was wondering when someone would do this (like yahoo).

would something like this be adaptable to, say, treocentral boards, or others very generically?

thanks.
  Reply With Quote
Old 09-22-2005, 11:01 AM   #3
Alexander Turcic
Fully Converged
Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.
 
Alexander Turcic's Avatar
 
Posts: 17,486
Karma: 10995944
Join Date: Oct 2002
Location: Switzerland
Device: Sony PRS-650 / Nexus 7 / Kindle PW
Quote:
Originally Posted by Unregistered
would something like this be adaptable to, say, treocentral boards, or others very generically?
That wouldn't be a trivial task because boards differ in layout and there is no way to predict what data to extract.
Alexander Turcic is offline   Reply With Quote
Old 09-22-2005, 11:25 AM   #4
alta88
Nameless Being
 
yeah, i realized that, then thought it could work like a plugin to a reader and each board could be user configurable, since all boards have certain things in common despite different layout. anyway, just smells like a useful bigger thing to me!

i'm checking palm now, seems updating is sporadic. do you have the bandwidth to run this through mobileread (assuming that's how you do it)?
i use greatnews (and thunderbird and feedreader depending) and the validate feed tool came back with an error (for palm) but a few recent posts updated though not the latest.
  Reply With Quote
Old 09-22-2005, 11:49 AM   #5
Alexander Turcic
Fully Converged
Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.
 
Alexander Turcic's Avatar
 
Posts: 17,486
Karma: 10995944
Join Date: Oct 2002
Location: Switzerland
Device: Sony PRS-650 / Nexus 7 / Kindle PW
Quote:
Originally Posted by alta88
yeah, i realized that, then thought it could work like a plugin to a reader...
Yeah that would be possible In fact, for all my custom feeds I am already using the same underlying engine.

Quote:
i'm checking palm now, seems updating is sporadic. do you have the bandwidth to run this through mobileread (assuming that's how you do it)?
The tool receives the latest 40 posts from Yahoo every time you fetch the feed with your RSS client, but only once every one hour. So if you don't use your RSS client for some time, it's possible that you'll miss some older messages (if there were more than 40 messages posted within this time). But that's the nature of RSS - you only get back as far as the numbers of last items that are returned by the feed. You can easily see if you missed messages by the running ID in front of each item title.

Quote:
i use greatnews
Great choice! I switched from FeedDemon to GN a few months ago.

Quote:
the validate feed tool came back with an error (for palm) but a few recent posts updated though not the latest.
Thanks for the hint! It should valdidate fine now. I switched from using the <author> tag to <dc:creator> since <author> would requires an complete e-mail address.
Alexander Turcic is offline   Reply With Quote
Old 09-22-2005, 12:26 PM   #6
alta88
Nameless Being
 
you and jack should get together on this plugin idea

so, you mean regardless of the interval i have set in greatnews or manual force update, the tool pops max once an hour and then only gets last 40? i mean, i can see why, but..
  Reply With Quote
Old 09-22-2005, 12:38 PM   #7
Alexander Turcic
Fully Converged
Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.
 
Alexander Turcic's Avatar
 
Posts: 17,486
Karma: 10995944
Join Date: Oct 2002
Location: Switzerland
Device: Sony PRS-650 / Nexus 7 / Kindle PW
Quote:
Originally Posted by alta88
you and jack should get together on this plugin idea

so, you mean regardless of the interval i have set in greatnews or manual force update, the tool pops max once an hour and then only gets last 40? i mean, i can see why, but..
I checked some of the most busy Y! Finance boards and once an hour seems to be a good interval to not miss any messages. More advanced clients like Greatnews will notice that the feed hasn't changed (they get a 304 header) and won't try to reload it every time you click on manual update.

The problem is that there are some smart people out there who update their feeds every 10 seconds (!) and that would definitely be a killer for a project like this one (not to mention Y! probably banning our server).
Alexander Turcic is offline   Reply With Quote
Old 09-22-2005, 01:39 PM   #8
alta88
Nameless Being
 
how about once every half hour?

much of yahoo is entertainment only, but some boards are actually amazingly useful and knowledgable..

funny that you're in switz; i'm in rome these days.

btw, you've posted good stuff over on GN.

best.
  Reply With Quote
Old 09-22-2005, 02:33 PM   #9
Alexander Turcic
Fully Converged
Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.
 
Alexander Turcic's Avatar
 
Posts: 17,486
Karma: 10995944
Join Date: Oct 2002
Location: Switzerland
Device: Sony PRS-650 / Nexus 7 / Kindle PW
Lemme explore the logs in the next few days and depending on traffic I'll reduce it to every 30 mins.
Alexander Turcic is offline   Reply With Quote
Old 09-26-2005, 08:30 AM   #10
alta88
Nameless Being
 
alexander,

i've been using your tool for a few days now, and it's changed forever how i read the Y! message boards. really terrific. in GN i can sort by author and get rid of known bad ones/keep known good ones, and save useful stuff. for multiples faster than before.

i don't know your plans for this, but hope you turn it into a generic tool. someday phpBB and the others might have this sort of feature built in, but.. it'll be a one by one progression etc.

msg boards are a different beast, but i wish more sites did what PubMed does, ie allow sending a user customized keyword search as an RSS feed.

some feedback. i know it's only a yahoo message board, and you don't need two-phased commit or checksum architecture to guarantee message integrity. but, you sort of have to guarantee no message gaps. palm eg. is in a very interesting period, and heavy msg volume friday led to hourly gaps. it's the ancient problem of tradeoff between getting, in realtime, both very high volume and very low volume messages.

conversely, on one very lightly used board, i got duplicates after i deleted posts i didn't want. but not all, just a few reappeared. i will watch this carefully again and report a better description.

thanks!
  Reply With Quote
Old 09-26-2005, 12:30 PM   #11
Alexander Turcic
Fully Converged
Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.
 
Alexander Turcic's Avatar
 
Posts: 17,486
Karma: 10995944
Join Date: Oct 2002
Location: Switzerland
Device: Sony PRS-650 / Nexus 7 / Kindle PW
alta, I am happy you like the tool!

Regarding 'generic': It's often up to the webmaster whether to offer RSS feeds for his forum. For instance, Mobileread has RSS feeds for every forum section. Returning RSS feeds based on customized keyword searches sounds useful, but causes some serious scaling issues. As long as you restrict the number of feeds, you can easily cache them for use by multiple people. But if you offer keyword searches, it's possible that your server has to great hundreds if not thousands of individual RSS feeds, definitely putting a lot of stress on any server (bandwidth and CPU).

I am still not sure about refreshing the feeds more than every one hour. So far word hasn't spread much of this tool - but if more people start using it, one hour might be the best frame to limit stress on the servers. Regarding duplicate posts - that issue is most likely related to your feed client (GN) which doesn't keep track of what items you've deleted before and once you refetch the feed, the previously deleted items come back.

update: I've changed the update frequency to every 30 minutes. Let's see how it goes...

Best
Alex
Alexander Turcic is offline   Reply With Quote
Old 09-27-2005, 11:34 AM   #12
alta88
Nameless Being
 
yes, the best way is for all boards to offer a feed. but they don't. surprising about yahoo, they have almost everything else.

as an fyi, check out
http://www.ncbi.nlm.nih.gov/entrez/q...arch&DB=pubmed
where a p53 gene search is returned. Send to dropdown, RSS feed. i'm guessing they have a lot of resources, every reasearcher in the world can do a custom search(es) on a very large database..

yes, it's GN reposting deletes, i recall the bug mentioned now.

i guess the refresh rate is of secondary importance to message gaps.

thanks.
  Reply With Quote
Old 09-27-2005, 11:41 AM   #13
alta88
Nameless Being
 
forgot to mention, feed for EXAS doesn't validate..

thanks.
  Reply With Quote
Old 09-27-2005, 12:42 PM   #14
Alexander Turcic
Fully Converged
Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.
 
Alexander Turcic's Avatar
 
Posts: 17,486
Karma: 10995944
Join Date: Oct 2002
Location: Switzerland
Device: Sony PRS-650 / Nexus 7 / Kindle PW
Quote:
Originally Posted by alta88
forgot to mention, feed for EXAS doesn't validate..

thanks.
Yahoo is having issues with the forum for this stock. See attached screenshot of http://messages.yahoo.com/?action=q&board=EXAS.

Quote:
yes, the best way is for all boards to offer a feed. but they don't.
Best would be to ask the administrator of the board you are interested in to offer RSS.

Quote:
i'm guessing they have a lot of resources, every reasearcher in the world can do a custom search(es) on a very large database..
You're probably right.

Quote:
i guess the refresh rate is of secondary importance to message gaps.
The refresh rate is the only reason for having message gaps since only the latest 40 posts are returned each time you query the updated feed.
Attached Thumbnails
Click image for larger version

Name:	yahoo.gif
Views:	186
Size:	69.2 KB
ID:	857  
Alexander Turcic is offline   Reply With Quote
Old 09-27-2005, 12:57 PM   #15
Alexander Turcic
Fully Converged
Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.Alexander Turcic ought to be getting tired of karma fortunes by now.
 
Alexander Turcic's Avatar
 
Posts: 17,486
Karma: 10995944
Join Date: Oct 2002
Location: Switzerland
Device: Sony PRS-650 / Nexus 7 / Kindle PW
OK, I added some extra sanity checks in 1.01 to remove illegal content from broken boards such as EXAS.
Alexander Turcic is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Hello from WA, love the boards! Ayrahvon Introduce Yourself 11 02-16-2010 07:06 PM
School boards, Religion and Politics... kennyc Lounge 7 02-15-2010 05:11 PM
30 Free Ebooks on Personal Finance Nate the great Deals, Freebies, and Resources (No Self-Promotion) 0 01-28-2008 03:45 PM
Yahoo! Finance News through RSS feeds TadW Lounge 0 01-13-2005 10:06 AM


All times are GMT -4. The time now is 01:22 PM.


MobileRead.com is a privately owned, operated and funded community.