MobileRead Forums - View Single Post - Finding Duplicate files without launching Calibre?

altonaduck · 04-10-2013, 01:00 AM

I can think of a way to do this under Unix/Linux (or Cygwin), but it onvolves the command line.....

Basically, you do something like:

find <name of top level folder> -print | xargs sum | sort > listofbooks.txt

Then you search for lines in listofbooks.txt that have the same checksum entry - they are probably, but not definitely, duplicates. If you want I can make this more robust and automated - it's what I'm going to have to do myself, but not for a week or so. If you want I can post the script once I do - but it will be a Unix/Linux script.

There's probably a way to do it using PowerShell too, but my PowerShell skills aren't that good yet.

04-10-2013, 01:00 AM	#6
altonaduck Junior Member Posts: 3 Karma: 10 Join Date: Oct 2011 Device: Android	I can think of a way to do this under Unix/Linux (or Cygwin), but it onvolves the command line..... Basically, you do something like: find <name of top level folder> -print \| xargs sum \| sort > listofbooks.txt Then you search for lines in listofbooks.txt that have the same checksum entry - they are probably, but not definitely, duplicates. If you want I can make this more robust and automated - it's what I'm going to have to do myself, but not for a week or so. If you want I can post the script once I do - but it will be a Unix/Linux script. There's probably a way to do it using PowerShell too, but my PowerShell skills aren't that good yet.