Best Duplicate Finder Programmes?

[Dullahir]

So I was wondering: What is THE best Dupe Finder program that will actually find things like .epub and .mobi documents? Because the only Dupe Finders that I know of only find .pdf which is SUPER annoying, PLUS, they delete the original, AND the duplicate files, leaving me with zero copies.

Any ideas?

[Tex2002ans]

The program that I use to check for duplicates is called Fast Duplicate File Finder:

http://www.mindgems.com/products/Fast-Duplicate-File-Finder/Fast-Duplicate-File-Finder-Download.htm

[DedTV]

If I just want to find and delete dupes quickly, I use CloneSpy.

If I need more fine control over what I'm doing, I use AllDup

Both are Free for non-commercial use.
Both can handle any kind of file to find exact dupes. AllDup has some tools and options to make dealing with similar files a bit easier (file preview and the ability to pass files to a program for comparison, etc).
Both programs also allow you to move "deleted" files to a folder rather than actually deleting them which you should ALWAYS do to avoid disastrous mistakes (although making a full backup is even better).

[JSWolf]

Forget all of that. You don't need any of it.

Just use Calibre with the Find Duplicates plugin. That's all you need.

[Dullahir]

Thanks for the excellent suggestions. However some of these won't work.

Also, I'm trying to find the dupes *before* sending them to Calibre, since sometimes books are renamed 'Sin and Seduction' 'Sin and Seduction (2)' which Calibre doesn't locate, neither does DoubleKiller.

[JSWolf]

Originally Posted by Dullahir

Thanks for the excellent suggestions. However some of these won't work.

Also, I'm trying to find the dupes *before* sending them to Calibre, since sometimes books are renamed 'Sin and Seduction' 'Sin and Seduction (2)' which Calibre doesn't locate, neither does DoubleKiller.

Give the Find Duplicates plugin for Calibre a go. It might do what you want other then finding the dupes before loading into Calibre.

[Dullahir]

I would, but I have 8k .epub files.

Also:

When using AllDupe:

I created a folder and in it, I added 'Becca Fitzpatrick - Hush, Hush' and 'Becca Fitzpatrick [Hush Hush Saga 1] Hush Hush' into a single folder and ran a check with the .epub filter.
I receive the same message I got from Doublekiller.

'1 Folder and 2 Files were located, but no duplicates were found.'

Could someone tell me please why this would be?

[Tex2002ans]

Originally Posted by Dullahir

Thanks for the excellent suggestions. However some of these won't work.

Fast Duplicate File Finder will find files that are EXACTLY THE SAME FILE.... aka, a duplicate file. From my reading of your initial post (seems as if DedTV also understood it this way), it seems that is what you were asking for.

If you meant something that specifically looks through the metadata, and tries to point out books that have the same title/author, but are in different formats/different files, then I would then recommend what JSWolf recommended.

Originally Posted by Dullahir

I created a folder and in it, I added 'Becca Fitzpatrick - Hush, Hush' and 'Becca Fitzpatrick [Hush Hush Saga 1] Hush Hush' into a single folder and ran a check with the .epub filter.

Did you try the "Find Duplicates" plugin?

There is no program that will magically handle all this different metadata for you, but that plugin should help wittle down doubles greatly. Cleaning up such a large library will still require lots of manual intervention... Although it should just be a job which will be required to be done once. From then on, just keep your collection clean.

[kacir]

Originally Posted by Dullahir

Also, I'm trying to find the dupes *before* sending them to Calibre, since sometimes books are renamed 'Sin and Seduction' 'Sin and Seduction (2)' which Calibre doesn't locate, neither does DoubleKiller.

Find duplicates plugin in Calibre will find those and give you powerful tools to deal with them.

As for duplicate files, if you set calibre to auto-merge duplicates, those will removed at the moment you import (or drag and drop) them into Calibre.

[Dullahir]

Thanks for the suggestions.

Yes, I tried the Find Duplicate plugin, and no, it wouldn't work for some reason.

[JSWolf]

OK, here's what you can do given the similarity of the Metadata. Import all your eBooks.

Sort by title and manually look through the list.
Sort my author and manually look through the list

That seems to be about all you can do.

[Dullahir]

Thank you. I'll try other methods, as I have 16,032 epub files. See my conundrum?

And with Doublekiller AND AllDupe, sometimes it will find similarly named items. At least it did at first, like that 'Hush, Hush' example I've mentioned, it would have found that right quick. But for some odd reason, that only worked with .pdf and I don't like Adobe.

[DedTV]

Originally Posted by Dullahir

When using AllDupe:

I created a folder and in it, I added 'Becca Fitzpatrick - Hush, Hush' and 'Becca Fitzpatrick [Hush Hush Saga 1] Hush Hush' into a single folder and ran a check with the .epub filter.
I receive the same message I got from Doublekiller.

The one weakness with all such programs have is they will only check for binary duplication. 2 ebook files that render exactly the same way in a viewer can be vastly different "binarily". As far as I know, there's no program that will check for rendered duplicates. If you were searching for binary dupes, that's probably why they didn't come up.

You might try Anti Twin which lets you find files with similar names which sounds like what you were trying to do with Alldup.

[kellygas]

Another alternative also that's worth trying is "Duplicate Files Deleter" ... great piece of software and effective, I'm currently using it to hunt and delete all duplicate files.. hope it helps..