Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 07-21-2012, 08:23 AM   #286
Stampercam
Connoisseur
Stampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to behold
 
Stampercam's Avatar
 
Posts: 86
Karma: 19674
Join Date: Jan 2011
Device: ipad, MiGear
I know it is weird... I've been duplicate searching for days, and now when it is cleaned up I tried running it again and suddenly the error. Yes the library is huge but that hasn't stopped it from working until now.

I just tried again, and it worked for identical title and identical author, but still getting error with fuzzy title and identical author.
Stampercam is offline   Reply With Quote
Old 07-21-2012, 08:31 AM   #287
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,600
Karma: 2092290
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
Well I have no idea what your definition of "huge" means but on a 31,000 book library I have no issues - the search runs in a couple of seconds without problems.

The only thing that has changed in that part of the code would be for fuzzy *author* searches, for which I no longer generate reversals of author names which helps reduce the number of false positives. However that will have zero impact on fuzzy title searches.

How many exemptions do you have setup? Has your "cleaning up" process involved adding to those? Select "Show all book duplicate exemptions" to see what you have. You can clear the exemptions by selecting them and choosing "Remove selected exemptions". See if that helps you out.
kiwidude is offline   Reply With Quote
Advert
Old 07-21-2012, 08:37 AM   #288
Stampercam
Connoisseur
Stampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to behold
 
Stampercam's Avatar
 
Posts: 86
Karma: 19674
Join Date: Jan 2011
Device: ipad, MiGear
Huge means 39000, and issues were for the first time today. I cleared the exemptions and that fixed it. I hadn't used the exemptions before today so that would explain the different behaviour. Thanks

By the way, I love the new metadata variations feature to sort for series. That works fantastically.

Cam
Stampercam is offline   Reply With Quote
Old 07-21-2012, 08:46 AM   #289
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,600
Karma: 2092290
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
The exemptions feature I can only imagine being a potential issue if you marked pretty much your whole database as exempt (e.g. you ran a very fuzzy search and chose "Mark all groups as exempt"). How many exemptions did you have? Without reviewing the code again I can imagine that to cause the logic a few headaches as it then tries to "reorganise" a large number of fuzzy groups.

Thanks for the feedback on the metadata variations - not had many people comment on that specifically as yet so glad to hear someone else is finding it useful.
kiwidude is offline   Reply With Quote
Old 07-21-2012, 08:54 AM   #290
Stampercam
Connoisseur
Stampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to beholdStampercam is a splendid one to behold
 
Stampercam's Avatar
 
Posts: 86
Karma: 19674
Join Date: Jan 2011
Device: ipad, MiGear
I had 1700 exemptions marked. I had been running searches with title similar and author ignore to try and pick up the many titles that had unknown authors. This worked but I had marked a lot of exemptions because I had to restart a few times and it had frustrated me. As you say, probably causing the logic headache. Anywho, happy now, it's not broken anymore.
Stampercam is offline   Reply With Quote
Advert
Old 07-21-2012, 09:16 AM   #291
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,600
Karma: 2092290
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
v1.5.2 Released

Changes in this release:
  • When using "Find library duplicates" clear the current search in order to compare the entire restricted library
  • When using "Find metadata variations" and showing books, fire the search again to ensure results reflect the search

Hopefully that is the last of the quirks out of the "Find library duplicates" behaviour. If you had a search on screen, previously it would only be comparing those books against the other library if you did a Title/Author search. Now it clears the current search (but not any search restriction) before doing the comparison. If you only want to compare a subset of your library, then just like "Find Duplicates" do a search, apply it as a restriction and then start the Find... option of your choice.
kiwidude is offline   Reply With Quote
Old 07-22-2012, 07:46 PM   #292
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 20,457
Karma: 26645808
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by kiwidude View Post
Changes in this release:
  • Add a "Save log" button for the "Find library duplicates" result screen.
- whew, that was quick, and thanks for the explain re library names

cheers BR
BetterRed is offline   Reply With Quote
Old 07-25-2012, 01:43 AM   #293
lsilver
Junior Member
lsilver began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Jul 2012
Device: ipad
I'm having a problem with the Find Duplicates plugin - when scanning for binary duplicates, all formats of the duplicate entry is deleted. I can recover from this, but it seems like a bug to me.

If this isn't the right way to post a bug report or the right place, someone please correct me.

thanks
Lenny Silver
lsilver is offline   Reply With Quote
Old 07-25-2012, 04:54 AM   #294
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,600
Karma: 2092290
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
Hi Lenny - welcome to MobileRead. And yes reporting a bug on the plugin thread (or via PM/email to me if it is my plugin like this is) is the right thing to do.

If you have "When doing a Binary Compare, automatically remove duplicate formats" checked then it will only remove all but one duplicate out of each group. I've just run it again now to verify and that is what it is doing. Create a new library for testing, add a couple of books to it creating a binary duplicate and give it a whirl.

My only guess as to what is happening is that you have been deleting books directly out of your library folders and your calibre database is hence out of date. Run Library Maintenance -> Check Library all the way through to make sure that you fix any incorrect book records so they reflect the actual formats you have available for a book. Only then should you be running this option. This plugin does not physically reverify all the formats exist on all the records in the group before removing the "other" duplicate formats, so my guess is that it is happening to leave behind records which are invalid.

If this is ringing bells to you (of having deleted stuff directly out of calibre's folders) then thats fine - you know not to do that and to use the "Remove" feature of calibre to remove formats instead.

If you don't believe this is the case, your library maintenance shows all formats are fine and you can replicate the issue with a series of specific steps then post back.
kiwidude is offline   Reply With Quote
Old 07-26-2012, 10:26 PM   #295
lsilver
Junior Member
lsilver began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Jul 2012
Device: ipad
Thank you - that was it - I never noticed that checkbox before
lsilver is offline   Reply With Quote
Old 07-29-2012, 04:42 AM   #296
DedTV
Enthusiast
DedTV began at the beginning.
 
Posts: 28
Karma: 10
Join Date: Dec 2009
Device: PRS-505; Galaxy Tab 7
Quote:
Originally Posted by kiwidude View Post
it will find you variations like "Arthur C Clark", "Arthur C. Clark" and "Clark, Arthur C." for instance. Fuzzy is the loosest search, it tries to flatten names out by taking the author last name and the first initial of their first name, in order to catch "A. Clark" and "Arthur Clark". However as you have found it will produce a lot of false positives from this approach.
Would it be possible to have a modifier for the Fuzzy search to include the whole first name rather than just the first initial so Arthur C. Clarke would match with Arthur Charles Clark or Arthur Clark but not Amy, Aaron, Adam (Etc) Clark?

Not a huge need as Fuzzy works well enough for finding those pesky middle name/initial variants but cutting down the false positives when that's what we're looking for would be a big help if it's not too much of a PITA (or impossible. Or already possible and I'm too dense to have found it ).
DedTV is offline   Reply With Quote
Old 08-03-2012, 06:02 AM   #297
odinokij
Enthusiast
odinokij began at the beginning.
 
Posts: 29
Karma: 10
Join Date: Jul 2012
Device: Kindle 3
Hello and thanks for your work,

I'd like to point a possible improvement for the Find Duplicates plugin: After doing a "Find library duplicates" you end up with a log of duplicate books. It would be nice if those duplicated books remained selected so we could delete them easily.

It's just a hint, I really would appreciate it.

Thank you very much.

Odinokij.
odinokij is offline   Reply With Quote
Old 08-03-2012, 06:34 AM   #298
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 29,688
Karma: 54369090
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
Quote:
Originally Posted by odinokij View Post
Hello and thanks for your work,

I'd like to point a possible improvement for the Find Duplicates plugin: After doing a "Find library duplicates" you end up with a log of duplicate books. It would be nice if those duplicated books remained selected so we could delete them easily.

It's just a hint, I really would appreciate it.

Thank you very much.

Odinokij.
They Are

The search bar has turned green and contains:
Code:
marked:duplicate_group_0001
In addition: The Filter value
Code:
*marked:duplicates (or *current search)
(does not work (for Pair))


Note the Result option section on the Search option Dialog.
The Left Choice is: All at once
The Right choice is: one Group at a time
theducks is offline   Reply With Quote
Old 08-03-2012, 07:49 AM   #299
odinokij
Enthusiast
odinokij began at the beginning.
 
Posts: 29
Karma: 10
Join Date: Jul 2012
Device: Kindle 3
Hi,

I'm talking about "Find library duplicates" that opens the "Cross Library Search Options" windows, in which there is no Result option section.

The normal "Find book duplicates" (duplicated books in the same library) works fine.

Thank you.
odinokij is offline   Reply With Quote
Old 08-03-2012, 09:29 AM   #300
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 29,688
Karma: 54369090
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
Quote:
Originally Posted by odinokij View Post
Hi,

I'm talking about "Find library duplicates" that opens the "Cross Library Search Options" windows, in which there is no Result option section.

The normal "Find book duplicates" (duplicated books in the same library) works fine.

Thank you.
I hadn't tried that feature.
I have 2 Libraries: My main and a Test (old, subset of Main where everything should be duplicates )
theducks is offline   Reply With Quote
Reply

Tags
cross library duplicates, in library duplicates

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Quality Check kiwidude Plugins 1171 03-23-2024 05:18 AM
[GUI Plugin] View Manager kiwidude Plugins 413 03-17-2024 12:01 AM
[GUI Plugin] Open With kiwidude Plugins 402 03-16-2024 11:44 PM
[GUI Plugin] Generate Cover kiwidude Plugins 811 03-16-2024 11:31 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 04:51 AM.


MobileRead.com is a privately owned, operated and funded community.