Thread: [GUI Plugin] Find Duplicates
View Single Post
Old 08-09-2011, 08:02 AM   #126
mbovenka
Wizard
mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.mbovenka ought to be getting tired of karma fortunes by now.
 
Posts: 2,082
Karma: 14079267
Join Date: Oct 2007
Location: Almere, The Netherlands
Device: Kobo Sage
Plugin starts eating huge amounts of memory
I found out that using 'Soundex' for the Title and 'Ignore' for the Author in an Author/Title duplicate search doesn't work well. With my library (~40K books) the plugin starts eating memory like mad, in the end crashing Calibre when it runs out, which happens in half a minute or less (this on a 2.4GHz Corei5 with 2GB RAM + the same VM)

It doesn't do this when using Soundex for both, or indeed any other combo I have tried (mostly Fuzzy/Fuzzy or Fuzzy/Ignore) or when using ISBN matching.

Those all work just fine, and have weeded out literally thousands of dups (probably close to 4000) from the mess that was my ebook collection
mbovenka is offline   Reply With Quote