Calibre has a library check that can help you with this. This is a common problem if you have manipulated files and folders inside the Calibre library. Perhaps copied a backup over an existing copy of the library.
Before you start, make a full backup of your calibre library, including the "bad" folders. Otherwise you may find that you have lost some titles.
You check for errors like this:
Library | Library Maintenence | Check Library
First the database will be checked, after that you will be asked if you wish to check if the files in your library match the information in the database. Click "Yes," and after a while you will get a report with errors found, and extra titles and so on. From there you can delete the extra titles and fix other problems.
Another way is, like DoctorOhh suggests, to move the library to a new location. Or to create a new empty library with the same structure as the one you currently use. Then select and copy all books to the new library (and delete after copy). Books, files and folders left behind in the old library folder are "junk" that Calibre has lost track of. It may not be junk at all...