[GUI Plugin] Find Duplicates

[kiwidude]

Originally Posted by chaley

I will submit the changes to Kovid. The fix is a bit exotic so he might prefer an alternate approach. I will post here once we have a resolution in source.

Nice one chaley. I did consider moving all these posts out into the dev forum as we wandered away from the specifics of Find Duplicates with this one. However clearly your changes (or whatever Kovid approves) is going to benefit everyone with larger libraries and across a number of plugins, so great stuff.

For once it wasn't something I broke...

[kiwidude]

Release Notes:
https://github.com/kiwidude68/calibr...icates-v1.10.3

Sorry I know these releases are like buses at the moment - wait forever and then 2 or 3 arrive at once.

This finishes something I started in 1.10.2 but did not have time to complete.

Now library duplicate searches will display the results using a virtual library restriction, slightly similar to what Find Book Duplicates does. All the same rules apply for if you clear a search/switch virtual library or choose "Clear duplicate results" from the Find Duplicates menu of clearing the marked books/restoring previous virtual library etc.

The documentation pages have been updated here:
https://github.com/kiwidude68/calibr...ind-Duplicates

[dunhill]

Kiwidude there is a screen size problem in Mac Monterrey that we can not reduce it.
The reduction arrows have little travel, hahaha I don't know how to say it

[kiwidude]

@dunhill - I don't have a Mac to try it on. One suggestion it might be - do you have some super long identifiers in your books? Perhaps that ISBN dropdown is being forced very wide by the contents? You could try dropping it down to see if you can spot the long entry/entries, and/or try viewing in the tag browser.

Nothing should have changed in this release regarding the dialog sizing.

EDIT: I've managed to replicate the same issue myself, that is what your problem is being caused by. I will see if I can constrain that combobox width to a maximum in Qt somehow...

[kiwidude]

Release Notes:
https://github.com/kiwidude68/calibr...icates-v1.10.4

Give this a whirl dunhill. I am now truncating the identifier types to less than 50 characters and forcing a maximum width.

There is still one further issue with that dropdown of when you have dozens or hundreds of identifier types, thanks to the garbage urn1231928319 type nonsense that many books may have. Personally I did a clean on my library of all that nonsense so the identifier list is just the handful I care about.

[ownedbycats]

Just curious: How feasible would it be to extend 'find metadata variation' to custom columns? Not something I personally need (yet!) but I can see the use cases.

[kiwidude]

Originally Posted by ownedbycats

Just curious: How feasible would it be to extend 'find metadata variation' to custom columns? Not something I personally need (yet!) but I can see the use cases.

Short answer - more work than I have time for at the moment . And since no-one has asked for it in the last 10 years I think it would be a solution looking for a problem to solve. I totally get where you are coming from though - in theory it fits with what a duplicate finding plugin does with other metadata columns.

[dunhill]

Originally Posted by kiwidude

Release Notes:
https://github.com/kiwidude68/calibr...icates-v1.10.4

Give this a whirl dunhill. I am now truncating the identifier types to less than 50 characters and forcing a maximum width.

There is still one further issue with that dropdown of when you have dozens or hundreds of identifier types, thanks to the garbage urn1231928319 type nonsense that many books may have. Personally I did a clean on my library of all that nonsense so the identifier list is just the handful I care about.

With this version it looks good

[Rellwood]

Originally Posted by kiwidude

Short answer - more work than I have time for at the moment . And since no-one has asked for it in the last 10 years I think it would be a solution looking for a problem to solve. I totally get where you are coming from though - in theory it fits with what a duplicate finding plugin does with other metadata columns.

I want the metadata in custom columns- cleans up old tags.

Can I go back to a previous version of the plugin?

[kiwidude]

Originally Posted by Rellwood

I want the metadata in custom columns- cleans up old tags.

Can I go back to a previous version of the plugin?

You are free to do whatever you like. Just don’t ask here for any help or support for that version - it has bugs and limitations I have no interest/time to deal with. I believe capink added it to the old plugin versions sticky thread. I will only ever support the latest version of all my plugins.

What do you mean by “cleans up old tags” by the way? Give me some concrete examples of custom columns of what type of duplicate search might help.

[ownedbycats]

Originally Posted by kiwidude

Give me some concrete examples of custom columns of what type of duplicate search might help.

I'm not Rellwood, but the example I was thinking was fanfiction category column, especially since Archive of Our Own often has stuff like "Fandom, Fandom - All Media Types."

I may be remembering wrong (or maybe it was another user?), but I seem to recall Rellwood mentioning moving all their old tags to a custom column for a spring cleaning. Well, technically autumn cleaning in the Northern Hemisphere

[try_it20]

Since the last the version of the plugin and of calibre the plugin stopped working by giving the error
IndexError: List index out of range

Windows 11 Pro, fully patched with latest updates:
Edition Windows 11 Pro
Version 22H2
Installed on ‎5-‎10-‎2022
OS build 22621.674
Experience Windows Feature Experience Pack 1000.22634.1000.0

Error Description Details:
calibre, version 6.7.1
ERROR: Unhandled exception: <b>IndexError</b>:list index out of range

calibre 6.7.1 embedded-python: True
Windows-10-10.0.22621-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.22621')
Python 3.10.1
Windows: ('10', '10.0.22621', 'SP0', 'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Find Duplicates (1, 10, 4)
Traceback (most recent call last):
File "calibre_plugins.find_duplicates.dialogs", line 367, in _ok_clicked
File "calibre_plugins.find_duplicates.dialogs", line 162, in selected_value
IndexError: list index out of range

[try_it20]

Looking on the internet it seems the plugin is addressing the wrong index
https://blog.finxter.com/python-inde...-out-of-range/
The error “list index out of range” arises if you access invalid indices in your Python list. For example, if you try to access the list element with index 100 but your lists consist only of three elements, Python will throw an IndexError telling you that the list index is out of range.

[kiwidude]

@try_it20 - thanks for reporting the issue. I can't replicate it as yet though. It seems to be related to the Identifier dropdown on the dialog. What type of search are you doing, do you have "any" identifiers in that dropdown? If you can include a screenshot of that Find Duplicate dialog just before you click the OK button (provided you still get the error when clicking ok) it would be helpful...

EDIT: Ok, I found one way to replicate it - by having no identifier types. Is this on a new library or something by chance, where you haven't as yet assigned any isbn or other identifiers for your books?

It is easy enough to put in protection against this to get a fix out, but it would be nice to know if there is some other scenario which triggers this...

[try_it20]

Originally Posted by kiwidude

@try_it20 - thanks for reporting the issue. I can't replicate it as yet though. It seems to be related to the Identifier dropdown on the dialog. What type of search are you doing, do you have "any" identifiers in that dropdown? If you can include a screenshot of that Find Duplicate dialog just before you click the OK button (provided you still get the error when clicking ok) it would be helpful...

Hi, here's what I have set on the find duplicate plugin. Nothing else.
Image: https://ibb.co/nzh71LF



So basically a plain Title and Author comparison, nothing fancy.

On the settings for the plugin itself, the following is set:
{
"SchemaVersion": 1.7,
"advancedMode": {
"bookDuplicates": {
"lastSettings": {},
"showAllGroups": true,
"sortGroupsByTitle": false
},
"libraryDuplicates": {
"lastSettings": {},
"librariesList": []
},
"metadataVariations": {
"lastSettings": {},
"showVariationBooks": false
},
"savedSettings": {}
},
"authorExemptions": [],
"bookExemptions": [],
"lastLibraryCompare": ""
}