[GUI Plugin] Find Duplicates

[kiwidude]

@edwdecarlo - Thanks for the kind words. However I think all you need to do is choose "Next result" from the Find Duplicates menu after you reapply your View Manager view and that will reapply the appropriate sorting for the groups.

[kiwidude]

Originally Posted by MartyTX

I noticed that in Find Duplicates "Cross Library Search Options" the dropdown list of libraries contains a few calibre libraries that I have deleted. I had created a few test libraries as a sandbox for learning calibre; they have been deleted using the "Remove Library" function and the folders have also been deleted.

Is there a relatively easy way to "prune deadwood" from the dropdown library list?

Go to your calibre configuration directory. Close calibre and then carefully edit the history.plist file. Inside there you will see the list of libraries from the history of usage of this combo, within the <asset> under the <key>lineedit_history_find_duplicates_plugin:libra ry_duplicate_combo</key> entry.

[MartyTX]

Originally Posted by kiwidude

Go to your calibre configuration directory. Close calibre and then carefully edit the history.plist file. Inside there you will see the list of libraries from the history of usage of this combo, within the <asset> under the <key>lineedit_history_find_duplicates_plugin:libra ry_duplicate_combo</key> entry.

Thank you for pointing me towards a quick and easy solution.

I noticed my post was moved. Should all plugin questions/comments be posted in the appropriate existing thread, for example: "[GUI Plugin] Find Duplicates" instead of in a new standalone thread? Glad to comply if it's preferred.

Marty

[kiwidude]

@Marty - yes I moved your post, and yes it is preferable to keep the q&a in the specific plugin thread rather than spread around the forum.

[edwdecarlo]

Originally Posted by kiwidude

@edwdecarlo - Thanks for the kind words. However I think all you need to do is choose "Next result" from the Find Duplicates menu after you reapply your View Manager view and that will reapply the appropriate sorting for the groups.

That did it. Thank you much.

[V3n0M]

I'm having a problem with the plugin. Every time I use binary compare calibre crashes. Is it because I have many books in my library (around 50 000)?

[mbovenka]

Originally Posted by V3n0M

I'm having a problem with the plugin. Every time I use binary compare calibre crashes. Is it because I have many books in my library (around 50 000)?

No. I have more (about 59K) and I have run a successful binary compare recently, so it's not the number of books as such. Perhaps running Calibre in debug mode and trying it again will give some useful info?

[Weekendmedic]

Originally Posted by mbovenka

No. I have more (about 59K) and I have run a successful binary compare recently, so it's not the number of books as such. Perhaps running Calibre in debug mode and trying it again will give some useful info?

Agreed - I'm at just about 79k, and while binary compare takes a long time, it does run. I use a scheduled script to move files into my auto-import folder, I need to disable that anytime I run a bigger search (like this), prevents unexpected use of resources during the search...

[saiyuki]

Hi. I have managed to combine my libraries and weeded out most of the duplicates. I am now left with only EPUB and MOBI formats. Using the 3rd party Duplicate File Detective program i can see where i have both EPUB and MOBI files the same, however, i cannot come up with a way of doing the same within Calibre.

I looked at the Duplicate Plug-in, but it gives no option to check duplicate filenames including the extentions - .epub, .mobi. Is there a way of achieving this either using the plug-in, or through Calibre itself so i am left with just EPUB's.

Note: I haven't deleted the duplicates with the 3rd party program.

Regards

[mbovenka]

Originally Posted by saiyuki

I looked at the Duplicate Plug-in, but it gives no option to check duplicate filenames including the extentions - .epub, .mobi. Is there a way of achieving this either using the plug-in, or through Calibre itself so i am left with just EPUB's.

That's because the idea of a 'file name' does not exist within Calibre. Only the metadata database exists, and that's what the Find Duplicates plugins works on. So if your EPUBs and MOBIs are different formats of the same book according to Calibre, it's easy:

formats:epub and formats:mobi

as a search term will give you books that have both a MOBI and an EPUB format. Select them all and do a right mouse button -> Remove Books > Remove all formats from selected books, except... -> Pick EPUB.

If they are not the same book according to Calibre, you'll have to either merge them by hand or play with Find Duplicates.

[saiyuki]

Worked a treat. Many thanks

Regards

[silentguy]

I really love this plugin! I manage a collection of fanfiction in one of my calibre libraries, and working with those can get quite messy and you plugin helps a lot in cleaning up.

One little request: Would it be possible to have an option to have "unknown" similar to every author?

[kiwidude]

@silentguy - it sounds a reasonable request, though I suspect it would be a major problem to implement with the way the comparisons work. From memory it computes a hash using a combination of title and author (depending on the options you have specified). If you do an "ignore author" search, then the hash it generates is done only on the title. However what you are asking for would not fit into this scheme, as you can't mix title only hashes with title/author hashes.

I will take a look at the code sometime to confirm this but I suspect it will be a non-starter because of this. I would have to code up a huge amount of special case logic and given in two years this is the first time this has come up I would rather not do that.

I can only suggest you use the "Ignore Author" searches if you are trying to find duplicates based on title only. Having the author field set to Unknown is not a particularly common thing for calibre users to do, certainly not for the bulk of their library

[silentguy]

Oh, no problem. I can see how this would only come up VERY seldom, I just came up this one time for me so I though I'd propose it cause it sounded like an easy addition...

My current solution is to ignore author and then just highlight results that contain matches with author "unknown"... a little more scrolling but easily manageable...

[crazyfrank]

im sorry but i couldnt find a answer to removing dups
when i search for dups, i use binary compare.
when i see the results, do i Delete all that see, or pick 1 from each listing and delete the rest.
i have 1000+