I cannot replicate that, the memory consumption goes down for me after each job completes (each job keeps the downloaded metadata in memory, when it competes the metadata is written to disk and the memory released).
Since this kind of bulk download is typically a one time thing, I suggest you just do it in a couple of batches. I lack the motivation to try to figure out why the mem isn't being released for you.
|