[Sirtel]

With a really messy library, there's unfortunately no other way to do this than manually. That is, going through it book by book, correcting metadata, deleting duplicates. With correct metadata you can use the Find Duplicates plugin to find and remove duplicates, but the plugin doesn't work when metadata is different between copies. So you'll have to fix the metadata first.

I know it's a lot of work, as I have a huge calibre library myself. But there's just no automated way to do it and get correct results.

Personally I don't bother with detailed or hierarchical genres - I just use a handful of broad genre categories and for more detail there's the synopsis in comments. Anything else would just be too much work when you have thousands of books. But YMMV, of course.

Start going through your books a few a day. Take your time. Don't try to do it all in a couple of weeks or even months. There's no hurry.