[GUI Plugin] Find Duplicates

[JSWolf]

Originally Posted by jony08

Do you mean that it is possible to filter the books of a certain format from the duplicates found by Find Duplicates?

Yes it is possible. The books that have the format you want is highlighted.

[komali_2]

Hello.

My library is massive, and manually merging duplications was untenable. Therefore, I've added the ability for this plugin to merge all duplication groups through a menu action.

I searched around for how to contribute to this plugin, but didn't find any github or anything like that, so for now, my changes can be seen in this commit: https://github.com/komali2/calibre-f...9881c368dbfd1d

It's not at all my intention to take any credit for the hard work of the primary developer/developers by uploading the plugin's source to my own github profile. Please let me know the best way I can issue a merge request for this change in the manner you prefer.

Usage:

(assuming the plugin with my change is installed. See https://manual.calibre-ebook.com/cre...ugins.html#id2 if necessary)

1. This is my first time touching the code of a Calibre plugin. Therefore, I highly recommend you backup your Calibre library.
2. Click the "Find Duplicates" button on the main Calibre toolbar. A dialog box should display with lots of options for how you wish to search and display duplicates, thanks to the hard work of the primary developers. I stick with the default options, but whatever you choose, click "ok."
3. After duplicates are found and grouped, click the small arrow next to the "Find Duplicates" button. A menu should display.
4. In that menu, click "Merge All Groups." Remember, this option is only available if you are using the "Find Duplicates" plugin with my changes (available at github link above, or maybe when you're reading this, merged into the main plugin).
5. For each of the duplicate groups, Calibre's native dialog boxes will pop up. That is to say, from here on out, the functionality is identical to if you manually selected several books and clicked "m" (or dragged one book onto another). The only difference is that you don't need to manually select books to merge. Try holding ALT and pressing "y" repeatedly to quickly verify each merge looks good to you, or, live dangerously and deselect the "show this confirmation again" checkboxes, resulting in the groups to all merge their books together automatically.

The result should be your various duplicate books each merged into a decently formatted single book. For example, I had:

• Betrayal by Aaron Allston. Tags: Sci Fi, Fiction. Formats: Mobi
• Betrayal by Aaron Allston. Tags: Sci Fi. Formats: Mobi
• Betrayal by Aaron Allston. Tags: . Formats: Epub

And after merging, I had

* Betrayal By Aaron Allston. Tags: Sci Fi, FIction. Formats: Mobi, Epub

Note that if you have two books the same FORMAT (epub / mobi / etc), the format of the first book in the list will be the format that survives the merge. This is due to Calibre's implementation of merging books. For example, if you had

• Betrayal by Aaron Allston. Tags: . Formats: Mobi
• Betrayal by Aaron Allston. Tags: . Formats: Mobi

If the first book's metadata indicated the given title and author, but had literally a blank .mobi file, and the second book had the proper .mobi, the result would be a single book with a blank .mobi file (I believe). This, among other reasons, is why you should definitely backup your Calibre library before doing a mass merge!

[tahone]

Hello everybody,

Sorry, but my English is very basic-
I need your help, " find duplicates" that had always worked normally, now freezes the program, and I need to restart calibre.
Obviously, it does not search or find anything .....

My calibre is up to date and my windows too.
I installed and uninstalled several times, but the problem continues

I can't use "find duplicates".

Do you have any solution for it?
Thanks in advance

[capink]

Start Calibre in debug mode and see whether it prints any errors.

[tahone]

Thanks capink
I start calibre in debug mode and I can see:
calibre Registro de depuración
calibre 5.20 [64bit] embedded-python: True is64bit: True
Windows-10-10.0.19041 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.19041')
Python 3.8.5
Windows: ('10', '10.0.19041', '', 'Multiprocessor Free')
Interface language: es
Successfully initialized third party plugins: Find Duplicates (1, 8, 5)
calibre 5.20 [64bit] embedded-python: True is64bit: True
Windows-10-10.0.19041 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.19041')
Python 3.8.5
Windows: ('10', '10.0.19041', '', 'Multiprocessor Free')
Interface language: es
Successfully initialized third party plugins: Find Duplicates (1, 8, 5)
Turning on automatic hidpi scaling
devicePixelRatio: 1.0
logicalDpi: 96.0 x 96.0
physicalDpi: 100.86162790697674 x 101.07357512953367
Using calibre Qt style: True
[0.00] Starting up...
[0.00] Showing splash screen...
[0.78] splash screen shown
[0.78] Initializing db...
[3.20] db initialized
[3.20] Constructing main UI...
[7.56] main UI initialized...
[7.56] Hiding splash screen
Starting QuickView
[44.97] splash screen hidden

....I can't understand were is the problem!!

[capink]

I cannot see anything unusual in the log. I updated to calibre 5.20 and it is working fine for me.

Maybe something is wrong with the plugin's settings. They are library specific and are stored inside the database so are not easily removed.

You can try to create a new library and make couple of empty identical books and see whether the plugin works fine in the new library.

Note: The plugin settings can be removed if you have can open the customize plugin dialog. But you must replace the settings with curly braces like below:

{}

It is better to try the new library thing first before going down this route. If a new library does not make this go away, there is no benefit in removing the settings.

[BetterRed]

Originally Posted by capink

Maybe something is wrong with the plugin's settings. They are library specific and are stored inside the database so are not easily removed.

The JobSpy plugin includes a tool to remove library specific preferences for some plugins, such as View Manager, but unfortunately not Find Duplicates.

I don't know why it's some plugins and not all.

BR

[capink]

Originally Posted by BetterRed

The JobSpy plugin includes a tool to remove library specific preferences for some plugins, such as View Manager, but unfortunately not Find Duplicates.

I don't know why it's some plugins and not all.

BR

That will be a useful tool, thanks for the hint.

[davidfor]

Originally Posted by tahone

Hello everybody,

Sorry, but my English is very basic-
I need your help, " find duplicates" that had always worked normally, now freezes the program, and I need to restart calibre.
Obviously, it does not search or find anything .....

My calibre is up to date and my windows too.
I installed and uninstalled several times, but the problem continues

I can't use "find duplicates".

Do you have any solution for it?
Thanks in advance

When does calibre freeze? When you click the toolbar button or menu item? If so, what happens if you press the escape key or the enter key? The dialog could be appearing off screen. If that does solve things, you can retrieve the dialog using Windows functions (which I never remember how to do) or edit a config file to remove the stored window location.

[tahone]

Hi!
I will try to explain in my limited English what I did.
1- uninstall find duplicates
2- create a new library with few titles and install caliber again.
Freezes when giving find duplicates toolbar button, so enter key

and the list of duplicates appears ... but only "identical title, identical author" as possible. For all other searches it does not show the dialog box.
But, now I can compare titles in various libraries ... which is fine.

I hope that in the next update everything will return to its original state.

But for the moment ... it works (more or less)

THANK YOU ALL FOR YOUR HELP AND ADVICE !!

[capink]

Version 1.8.7

• Update: Restore the whole sort order instead of just one column (bound by maximum_resort_levels tweak).
• Advanced mode: misc improvements.

[jony08]

Originally Posted by capink

Version 1.8.7

• Update: Restore the whole sort order instead of just one column (bound by maximum_resort_levels tweak).
• Advanced mode: misc improvements.

The latest version found by calibre is 1.8.5. When will this version be available from calibre?

[jony08]

Would it be possible to add an option to only display book posts of different formats, same format and both, when doing a Title/Author search?

[capink]

Originally Posted by jony08

The latest version found by calibre is 1.8.5. When will this version be available from calibre?

This is a beta version posted here for testing. If no problems are reported, it should be available in a week or two.

Originally Posted by jony08

Would it be possible to add an option to only display book posts of different formats, same format and both, when doing a Title/Author search?

Edit: In the latest version the advanced mode has the formats added to the list of available field, so that they can be used in duplicate search.

[Terisa de morgan]

Hello, I've updated the plugin to the release 1.8.8 and I've lost all the configuration I had. The rules are still there (by name) but when I try to apply them, the plugin says that they are not valid. I don't know if it's related to the fact that I have calibre in Spanish: The window dialog appears in Spanish but, when I add an option, aside of having to "guess" how to add an algorithm (there is only a white line, and I've guessed that pressing with the mouse expand the list), the available algorithms are in English.