This may be way off base, but would it make sense to keep the last n files or n days? Then a daily load with "keep 7 days" would give you anything that was generated within the last week. If the date was part of the filename, even better. And still better yet if only posts that haven't been downloaded are included in successive files.
|