[Adoby]

Let Calibre work as it does. And use it! Either surrender your soul to the Calibre way of doing things, or wander around in the barren wastelands outside. There is a plugin to Calibre that will let you search for duplicates. Both by a binary compare of the actual book and by comparing the file names. You can install the plugin from within Calibre.

So before you fetch any metadata to rename your new books, remove any books that already has a binary match (with correct metadata) already in the Calibre library.

In order to get Calibre to bulk fetch metadata, at least book title and author has to be correct. You have to fix that somehow. Perhaps scan it from the filename when you import, unless it can be read from inside the file. Or you will have to actually open and manually copy the title and author from inside the book.