[cybmole]

i don't suffer from this problem, but if I did then I'd start with windows utilities & freeware fiile duplicate finders.

something as simple as a windows search of calibre library foir *.epub, then sort results on file size & eyeball the list, would show up most dups. zap them & then use library repair in calibre. with 30,00 books it should have done long before collection got that big but use other search filters & work your way though systematically. or go straight to a good utility program - where you have options to match on file attributes, file content or both

a match on file size , and file extension + a related file name should be enough for manually spotting a duplicate ebooks. most utilities will auto delete fully matching duplicates according to some rule ( like keep oldest, or keep newest )

for a merge - I'd run a windows duplicate files finder utility before adding the folder( to calibre, then zap dups from the "to be added" folder - then do the merge.

PS the dup finder is built into some boost speed / defrag packages -e.g. there is one is auslogics boost speed which free trail ware for 30 days - long enough to get the job done!