View Single Post
Old 06-22-2011, 02:39 PM   #16
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 12,463
Karma: 8025600
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Quote:
Originally Posted by kiwidude View Post
@Kovid - the Find Duplicates plugin for binary comparison does two passes.

The first is to add candidates to a map by getting the os stat size and modified datetime:
Depending on how far we want to look ahead, we might need to abandon this. I can imagine that various cloud implementation don't have this information. On the other hand, there should be no trouble in supplying an API to give some info from 'stat' to the caller.
Quote:
The second pass is on the reduced subset (where size and modified datetime match) to compute a hash for each of those books:
A pipe would work very well for this, as would a memory file.
chaley is offline   Reply With Quote