As ilovejedd pointed out, keeping the files behind in the add folder is a big performance hit over time if you dont clean it out manually from time to time. It means calibre has to keep track of the files it has already added and compare every file in the folder to the history every time the folder is scanned.
As for duplicates, see
https://bugs.launchpad.net/bugs/926962