|
Alright, initial tests look rather promising.
As far as ps is concerned, I've reduced reduced the rss usage from ~50MB to ~15MB, so a pretty decent improvement I think.
I didn't have to massively overhaul how I deal with metadata (I'm still holding the entire set in memory), but tried to me more memory efficient by preallocating where possible, streaming JSON decoding, and attempting to avoid duplicating data structures as much as possible.
I don't think I've reduced the final working set of memory much, but rather how much 'temporary' memory is required.
|