Quote:
Originally Posted by duckblaster
Something that might be useful to some people: an offline metadata cache of entire sites
|
Ah... No.
1) Those sites are _not_ built to make that easy or cheap (bandwidth-wise). It's not like they've got a SOAP/XMLRPC API for metadata queries (although goodness, how that would make our lives easier). Also, they'd follow FFN on the "cut off those leechers" bandwagon really, really fast.
2) I think you seriously underestimate the volume of data involved here. Especially when considering Kaiju-class sites like FFN. You'd have to either deal with a gigabytes-large metadata.db, or have to install a separate instance of MySQL/PostgreSQL. Not quite lightweight, certainly not portable.
3) Where do you get your 1000+ HTTP requests? You're supposed to only update those stories for which you receive an update notice from the hosting site, not have your entire library checked every n days/hours. Even then, I only update those stories I intend to read right away (or close enough), and leave the remaining notices for later.
So, yeah. It's a nice idea, but let's just put it aside for now, 'kay?
N.