Finding and Deleting Duplicate Files of different formats

[oggelbe2007]

Ah, it seems that what you really need is some code with Judgemental Heuristics. Which would scan your ebook files (regardless of format, title errors, etc.) and product a list of duplicate ebook files as output.

[Ken Maltby]

Originally Posted by oggelbe2007

Ah, it seems that what you really need is some code with Judgemental Heuristics. Which would scan your ebook files (regardless of format, title errors, etc.) and product a list of duplicate ebook files as output.

The NSA (and perhaps the FBI now a days) could provide such a program,
but wouldn't be inclined to do so. There would not be much of an incentive
for one of their contractors to create such a spin-off product, either.

Luck;
Ken

[oggelbe2007]

Originally Posted by Ken Maltby

The NSA (and perhaps the FBI now a days) could provide such a program,but wouldn't be inclined to do so. There would not be much of an incentive for one of their contractors to create such a spin-off product, either.

Luck;
Ken

They would probably drop by your house/apt and ask you what kind of shenanigans you're up too...but all we need here is a simple adaptive sort;

1) take 2 files: A & B
2) convert them to a common format
3) take a random sentence from file A
4) compare this sentence with all of file B
5) if it hits - remove both files, place in dup file list & log results
6) if it doesn't hit - toss file B & grab another one
7) repeat until finished...or xmas arrives
8) verify and clean the dup file list.
9) bubble sort the remaining file list.
10) Take rest of the day off.

[wannabee]

I'll bet some of those books you got rid of were more than 16 years old. That's how long ago I was storing files on removable Syquest and tape drives that I tossed out.

I saw on telly that the museum of technology here has electronic data storage devises that aren't too old but they have no way of accessing the data because there are no readers that read it anymore.

So I hope you start a backup regime to future proof your collection when you finally get it sorted.

[T.D.02809]

Originally Posted by dpayment

Thanks Susan & Worldwalker, both of these answers are excellent, I hadn't thought to use Calibre to do the comparison, it just never occurred to me, but it makes perfect sense. Even if the file titles are different, the metadata should help to identify "most" of the duplicates, if not all.

Thanks again,
Dan

I am unsure if any of you (Dan, Susan,Worldwalker) will see this; however, I want to say Gracias-Thanks for this information. I have hit a similar difficulty. There are duplicate copies of ebooks on my Kindle 3Gen Keyboard model. When I have the time I have said I would organize them. Then I would say, 'another day'. Thanks to this info I am indeed organizing my ereader's ebooks eliminating duplicate copies. Gracias, T.D.02809