Hi Lenny - welcome to MobileRead. And yes reporting a bug on the plugin thread (or via PM/email to me if it is my plugin like this is) is the right thing to do.
If you have "When doing a Binary Compare, automatically remove duplicate formats" checked then it will only remove all but one duplicate out of each group. I've just run it again now to verify and that is what it is doing. Create a new library for testing, add a couple of books to it creating a binary duplicate and give it a whirl.
My only guess as to what is happening is that you have been deleting books directly out of your library folders and your calibre database is hence out of date. Run Library Maintenance -> Check Library all the way through to make sure that you fix any incorrect book records so they reflect the actual formats you have available for a book. Only then should you be running this option. This plugin does not physically reverify all the formats exist on all the records in the group before removing the "other" duplicate formats, so my guess is that it is happening to leave behind records which are invalid.
If this is ringing bells to you (of having deleted stuff directly out of calibre's folders) then thats fine - you know not to do that and to use the "Remove" feature of calibre to remove formats instead.
If you don't believe this is the case, your library maintenance shows all formats are fine and you can replicate the issue with a series of specific steps then post back.