10-20-2012, 05:55 AM | #331 |
calibre/Sigil Developer
Posts: 4,617
Karma: 2124234
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
@rotanal - changing a binary comparison to not actually be a binary comparison will not *ever* happen with this plugin.
I really don't see a use case for some kind of fuzzy comparison. You already have all the fuzzy comparisons required based on *metadata* to identify whether two books are duplicates. The only reason for doing a binary comparison is to most quickly be able to delete the corresponding duplicate with 100% assurance that you are not accidentally losing something. Scanning say two epubs and then deciding that based on their content they are "mostly similar" tells you nothing useful (and bear in mind that is just an easier to compare format, let alone all the others). The one exception being if you have screwed up your library metadata and given the book a title/author which it isn't. But that is such a niche case it isn't remotely worth the enormous effort to cater for. You can find duplicates using the existing metadata based functions. Having found those duplicates, deciding which ones to merge is another matter. Again if you think that "a few bytes" difference means one can safely and automatically deleted as being "almost binary" you are mistaken. As ilovejedd points out those "minor" differences could be the difference between a corrupt and a non corrupt book. Or one that is formatted to your liking versus one that is not. Or one that has been proofed for errors versus one that has not. Or a later edition. Or a different cover. Or one which has encoding a set correctly to make readable, etc, etc. You can't reliably automate those evaluations to determine what is the best version to keep. You have to open them up side by side and decide based on your own personal criteria. As I have had said repeatedly from the time I created this plugin ages ago, there is a space for someone to write a separate smart merge plugin. To allow a user who having been given the duplicate results from this plugin to make better informed merge decisions to help with deciding which to keep. For instance if two epubs are being merged, it could examine the zip files and compare contents files to tell you which differ etc. But such a plugin does not exist and I have no personal interest in writing it as I have no need for it. |
10-20-2012, 12:34 PM | #332 |
hopeless n00b
Posts: 5,111
Karma: 19597086
Join Date: Jan 2009
Location: in the middle of nowhere
Device: PW4, PW3, Libra H2O, iPad 10.5, iPad 11, iPad 12.9
|
Feature request:
Would it be possible to add searching for duplicates based on other identifiers aside from the ISBN? In particular, uri/url? I've got a lot of fanfics in my library and I've encountered cases where both the title and author's pen name for a fanfic have changed while the fanfic url (ergo identifier) remains the same. Thanks! |
10-26-2012, 06:37 AM | #333 |
calibre/Sigil Developer
Posts: 4,617
Karma: 2124234
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
@ilovejdd - apologies for the delay but Sigil development has vamped nearly every waking hour for months. To answer your question - that sounds a good suggestion to me. Change "ISBN Compare" to "Identifier" with a dropdown next to it for the user to choose which type from.
|
10-26-2012, 07:45 AM | #334 |
calibre/Sigil Developer
Posts: 4,617
Karma: 2124234
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
Beta for v1.6.0
This version lets you find duplicates using any identifiers rather than only ISBN. The list of identifiers comes from whatever identifiers exist in the books you have in your library.
As per usual let me know if any issues found before I officially release it... Last edited by kiwidude; 10-29-2012 at 04:38 PM. Reason: Removed attachment as officially released |
10-29-2012, 04:39 PM | #335 |
calibre/Sigil Developer
Posts: 4,617
Karma: 2124234
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
v1.6.0 Released
Changes in this release:
To the few of you that tried the beta above, please just force a refresh to pickup the latest addition as the version nubmer is the same but I did make some additions. |
10-30-2012, 03:25 AM | #336 |
Junior Member
Posts: 2
Karma: 10
Join Date: Oct 2012
Device: kindle touch
|
Thanks for this wonderful plugin!
And I have a question: Would it be possible to implement a duplicate search for 2 libraries? With a source and a destination library and the duplicates will be marked in the source library. Cause I would like to compare 2 libraries without merging them. |
10-30-2012, 04:14 AM | #337 |
calibre/Sigil Developer
Posts: 4,617
Karma: 2124234
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
@alexd - welcome to MobileRead.
You can already to that with this plugin, choose the second option down of "Find Library Duplicates..." |
10-30-2012, 07:24 AM | #338 |
Junior Member
Posts: 2
Karma: 10
Join Date: Oct 2012
Device: kindle touch
|
THANK YOU for the hint!
I use the plugin for ages, but always with the icon only! *shame on me* |
11-10-2012, 12:21 PM | #339 |
Groupie
Posts: 174
Karma: 126824
Join Date: Dec 2008
Location: Out There
Device: K3 W/3G (Fixed screen!) & Paperwhite Wifi
|
Has something changed reticently with the plugin?
I use find duplicates (Binary compare) all the time. (every time I add more books, most times I do not find any. But with the Baen Books Monthly Bundles, sometimes they repeat books, so I find a few) In any case, previously when I did a search (2-3 months ago?) it would take a few seconds to do a search (5-10? seconds maybe a little more? But it was not an excessive length of time) But lately it will take 2-3 minutes!! to do the search, and I still have roughly the same number of books in my library. AFAIK I have only done a couple things reticently I added a bunch of PDFs to my library awhile ago (maybe 200-300 books?) Would PDFs cause a slow down? The program used to take a couple of minutes to start up, until I saw a post about speeding up program starting times, that suggested not including formats in the tag browser. (Now it takes about 10 seconds to start up!!) Could the plugin now need to do whatever the tag browser used to do before it can complete the search? Thanks, |
11-10-2012, 12:38 PM | #340 |
Well trained by Cats
Posts: 29,703
Karma: 54369092
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
|
Any format should not affect startup time
UNLESS you have a weird custom column that requires FILE level access. How many Column Coloring rules do you have? (That has a BIG impact on my startup time) BTW did you change/version update your Anti-virus program? |
11-11-2012, 12:31 AM | #341 |
Groupie
Posts: 174
Karma: 126824
Join Date: Dec 2008
Location: Out There
Device: K3 W/3G (Fixed screen!) & Paperwhite Wifi
|
Hmm I think you miss read my post.
Calibre used to take a couple of minutes to start up, until I saw a post about speeding up program starting times, that suggested NOT including formats in the tag browser. (Now it takes about 10 seconds to start up!!) (I don't use any Column Coloring rules) The problem I have is the Duplicate's plugin, 2-3 months ago it would take a few seconds to do a search (5-10? seconds maybe a little more? But it was not an excessive length of time) Bit now it will take 2-3 minutes!! to do the search, and I still have roughly the same number of books in my library. My theory was maybe the plugin now needs to do (whatever the tag browser was doing with formats) before it can complete the search? My Antivirus is Norton. I have no clue when it updates. |
11-11-2012, 08:58 AM | #342 | |
Well trained by Cats
Posts: 29,703
Karma: 54369092
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
|
Quote:
IIRC User plugins just get initialized (they don't process the DB/files until invoked). Note: many plugins are evolving. There are basic speed-ups , then more/better checks tend to use up that performance gain 'Yellow box' products used to default to Hourly update checks (Actual updates only happened when found) Out of the chute, their default settings were always too aggressive (cpu/file system abusive) for my tastes My systems are dialed way back, as I can usually smell a scam link (I watch the actual URL that is displayed in the status line in Firefox). My wife tends to click first , so I have hers set closer to defaults. YMMV use settings that best reflect your awareness. BTW the duplicates plugin has settings that will affect times. When I use Similar/Similar, I see the checks complete in seconds (4K+ books). If you do a Binary search, that possibly requires that each FILE be opened (Speaking of reading Files: Defrag , when was it last done?) to calculate the values. |
|
11-12-2012, 01:31 PM | #343 |
Country0129
Posts: 55
Karma: 506306
Join Date: Apr 2012
Location: Louisiana
Device: Kindle, Kindle Fire, PC
|
Icon Missing On Toolbar
Something weird! I've been using "Find Duplicates" plugin for two years. Since last updating Calibre versions and any plugin updates (always regularly do this,) my "Find Duplicate" icon is missing from the toolbar.
Checking for installed plugins, it's still an active plugin, but I have no way to access its properties of which I know. Any ideas? |
11-12-2012, 02:22 PM | #344 |
Grand Sorcerer
Posts: 12,124
Karma: 73448616
Join Date: Nov 2007
Location: Toronto
Device: Nexus 7, Clara, Touch, Tolino EPOS
|
Maybe just head over to Preferences | Toolbar and select either Main Toolbar or Main Toolbar when a device is connecterd (or both) and add Find Duplicates to the icons displayed,
|
12-02-2012, 06:06 PM | #345 |
Enthusiast
Posts: 32
Karma: 10
Join Date: May 2012
Device: android
|
I installed v 1.6 and now the plugin fails to find duplicates. Anyone else is experiencing this problem? Is there an archive for old plugin? Thanks
Lucia |
Tags |
cross library duplicates, in library duplicates |
Thread Tools | Search this Thread |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
[GUI Plugin] Open With | kiwidude | Plugins | 403 | Yesterday 08:39 AM |
[GUI Plugin] Quality Check | kiwidude | Plugins | 1171 | 03-23-2024 05:18 AM |
[GUI Plugin] View Manager | kiwidude | Plugins | 413 | 03-17-2024 12:01 AM |
[GUI Plugin] Generate Cover | kiwidude | Plugins | 811 | 03-16-2024 11:31 PM |
[GUI Plugin] Plugin Updater **Deprecated** | kiwidude | Plugins | 159 | 06-19-2011 12:27 PM |