Will the advice of an intermediate user do?
I use the Find Duplicates plugin and it works very well to find duplicates. I look for similar title, similar author and that gets everything for me.
One thing I do to manage space is to set up separate libraries for different books. I have a library for comics, a library for subscriptions, a library for general books, a library for a specific genre, etc. That said, I don't have close to the same number of books that you do.
For storing by series, I have no idea. I know you can have the Series tag. Probably you'd need to change how Calibre saves books to include that?