Search for binary duplicates

[Beemer2]

I searched for duplicate books using the Duplicates Plug-in tool. I ticked the remove found duplicates. The search found many hundreds of duplicates which were displayed but there was no delete button or automatic deletion.

How should this removal process work?

[Sabardeyn]

The important part of Find Duplicates>Binary Compare is to set the appropriate Result Options for how you want any duplicates to be handled. Really this comes down to two means of cleaning with Binary Compare:

• Automatic Cleaning: You trust the plugin and just want it to delete any duplicates. Check Automatically Remove.
• Manual Cleaning: You don't fully trust an automated system and want to manually pick and choose what is deleted. Check the options Show All Groups At Once and Sort Groups By Number of Duplicates, at least.

For manual cleaning (sorry, I'm paranoid and would rather choose for myself which items to delete), your book list is set to a Virtual Library (duplicates only) with a set of duplicate books highlighted. A dialog pops up telling you the total number of duplicate file sets that have been found. (It's not a total count of all the duplicate books. You could have three sets which involves 52 books because you entered the same file repeatedly under different info.)

You will notice that their are two or more books highlighted: the book you chose and any duplicate books. You can take a look at the individual Edit Metadata, Book Details or Book Pane for each book to determine which book(s) to keep or delete.

To delete a specific book, highlight it in the book list. Then use the normal Delete menu bar option to have calibre permanently destroy the info in the database (ie, the metadata), while sending the ebook files to the recycle bin.

Note: Be very careful to compare metadata. You might have the same ebook file, but different metadata for each ebook entry. They might have different information (maybe one has an ISBN and another does not). Or maybe you've got two different publishers and published dates. Or the comments are different. So look at everything and make sure that what you're deleting is the one you really, really meant to destroy. Also be careful for things like the book variations: the normal edition or the 20th Anniversary edition, normal or illustrated, 1st or 2nd edition, etc.

[Sabardeyn]

I should have mentioned that in order to get out of the Find Duplicates function, you should click the Find Duplicates>Clear Duplicate Results. Otherwise you're restricted to just the duplicate books in the book listing.

To get out of the Virtual Library, you need to click on the yellow Virtual Library button and choose None. (Just mentioning this one in case you're not familiar with Virtual Libraries - which can get you confused because you cannot find books you know are in the library. Don't panic, just make sure you're not in a Virtual Library.)

[Beemer2]

Sabardeyn,

Thanks for taking the time to give your comprehensive reply. It will help me in future. However if you note the last part of my post it says that the automatic deletion is not working. Also despite numerous duplicates none are highlighted by this plug-in

[Sabardeyn]

I did a bit of testing with a dummy database. With Automatic Deletion (Highlight Authors or Show Groups settings don't matter), only the ebook file formats are deleted. So the ebook entry, and it's metadata within calibre, remains but there are no more files associated with that book.

To delete the ebook's metadata and entry in the book list, you will still need to manually delete the entry as I mentioned previously.

I'm not sure if this "double deletion" process is normal behavior or a bug.

[mbovenka]

Originally Posted by Sabardeyn

I'm not sure if this "double deletion" process is normal behavior or a bug.

I'm guessing normal behaviour. You might want to do something with the metadata afterwards, even if all actual books are gone. And if your library is messed-up enough that you find binary dups in what Calibre thinks are different books (and yes, that has happened to me ), you might even have formats left...

A simple 'formats:false' search will find all empty books left by the deletion process.