Your name was in my brain earlier today when was writing a post saying how many people helped me personally with calibre. Once that I remember clearly was with importing records way back in 2010
One thing that may help.
The find duplicates plugin will find duplicates across libraries. You can create a temp library using the copy structure option and import x books, use the plugin with the duplicate library option, put
not before marked duplicates in the search bar and you will have a list of all of the books
not in your main library.
What I do at this point is use the copy to library function to copy (and delete after copy) to the main library. Then I copy the duplicates to another temp duplicates library in case I want to compare later. Then it is on to the next batch of x books. It is actually pretty fast and efficient.
Helen