Thanks for all the replies so far!
Unfortunately I've tried most of the suggestions and they haven't really worked for me.
I've tried to use tags / fields to keep track of of what files I've imported/added directly and what is an output of a conversion. I came to the conclusion that it would be a lot easier for the software to keep track of if there isn't already some other solution
I'm playing with the idea of just having multiple full copies of my main library. One with only source formats, and a full duplicate used to actually load stuff onto my eReaders. This would probably be the least amount of work if there isn't some more automated solution, though it's far from a perfect solution I think.. *shrug*