Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 07-12-2012, 12:59 PM   #256
DMee
Connoisseur
DMee began at the beginning.
 
Posts: 90
Karma: 10
Join Date: Jun 2011
Location: Stockport, UK
Device: ipad, iphone, kindle
What an amazing addition!!! Many thanks yet again :-)))

Trouble is "Similar" authors produces nothing and "Soundex" or "Fuzzy" produce loads with some errors hidden amongst lots of "cruft" I beleive you call it :-)

e.g. Arthur Charles Clark was buried 10 screens down - is it possible to manually adjust the settings to limit A.Bronte and A.Brown clashing but pick up Arthur C. Clark and Arthur Charles Clark ??
DMee is online now   Reply With Quote
Old 07-12-2012, 01:06 PM   #257
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Ok, there seem to be a few things here. In the first case you are wanting to make a direct edit of books. I have made the plugin window modal deliberately, because if you were to make changes in the calibre window it will make the results displayed in the plugin window out of date. The primary use case I had in mind was where you would be perfectly happy renaming the authors via the plugin window. However in your case you had found a completely kakked metadata situation, and renaming the "A.N. Author" was not your desired action. I don't see that modality changing to be honest, I think it is an extreme situation and would cause too many headaches if I were to change it.

However your next point about that was that as you clicked cancel you found the plugin cleared the search so you could no longer see those rows to edit them outside the plugin. The intention on my part to restore whatever search the user had displayed before they ran the plugin, to ensure that data is refreshed visually. However I can see that in your situation you would instead want those results to be kept on screen. That is something I will take a look at changing - if a suer has "Show books on screen" checked, there is no issue with leaving them on screen when Close is clicked.

Your final KeyError message looks a concern as that looks a genuine bug. It looks like somehow the plugin was dealing with stale data. However to fix it I am going to need to be able to replicate it - if you or someone else comes across it again and can give me some steps it would be much appreciated.
kiwidude is offline   Reply With Quote
Old 07-12-2012, 01:10 PM   #258
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
The rules of what "Similar", "Soundex" and "Fuzzy" do are no different for Authors between the new screen and the old Find duplicate books approach.

Similar does not "do nothing" - it will find you variations like "Arthur C Clark", "Arthur C. Clark" and "Clark, Arthur C." for instance. Fuzzy is the loosest search, it tries to flatten names out by taking the author last name and the first initial of their first name, in order to catch "A. Clark" and "Arthur Clark". However as you have found it will produce a lot of false positives from this approach.
kiwidude is offline   Reply With Quote
Old 07-12-2012, 01:36 PM   #259
DMee
Connoisseur
DMee began at the beginning.
 
Posts: 90
Karma: 10
Join Date: Jun 2011
Location: Stockport, UK
Device: ipad, iphone, kindle
1) I see where you're coming from re Modal form
2) Yes, if you could leave search active then for weird cases like mine its easily fixed (see you're so good at plugins it does something you didn't plan! ;-) )
3)OK, here's what I did before getting error - dunno if recreatable.....

Plugin flags

John Doe & Jr.
Tom Smith & Iii

Authors Jr. and Iii are considered "Similar"

I Close plugin

I type Author:"Jr." or Author:"Iii" into search

I realise there are a few where the suffix has been taken as surname

I open Tag Browser - Manage Authors - search for Jr. and then Iii

In both cases I "Find" all cases and ensure the Author and Author Sort are corrected.

I close Manage Authors and rerun Find Duplicates - Metadata

Authors - Similar.... Search - boom error, close Calibre, reopen, works fine for every type of search on every item.


4) My "does nothing" wasn't slagging off plugin - its my bad shorthand for "Didn't produce any results i.e. I had no "Similar" authors, but I have loads Soundex and Fuzzy Authors :-)

5) In Find Duplicates you can set a length to the Soundex - that was what I meant by tweak in Metadata :-)

Keep up the amazing work :-)
DMee is online now   Reply With Quote
Old 07-14-2012, 10:13 AM   #260
pelikano
Junior Member
pelikano began at the beginning.
 
pelikano's Avatar
 
Posts: 4
Karma: 10
Join Date: Jul 2012
Device: sony prs-t1, iPad, HTC
Split unicates and duplicate

Hi, I'm using the plugin and it works great. But is there a way to split the duplicates in two groups: unicates(first found or original) and duplicates(all the other copies)
So I can delete all the duplicates at once?
pelikano is offline   Reply With Quote
Old 07-14-2012, 11:49 AM   #261
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,428
Karma: 5560777
Join Date: Aug 2009
Location: The (original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by pelikano View Post
Hi, I'm using the plugin and it works great. But is there a way to split the duplicates in two groups: unicates(first found or original) and duplicates(all the other copies)
So I can delete all the duplicates at once?
But what if the first found is not the best version
theducks is offline   Reply With Quote
Old 07-14-2012, 01:35 PM   #262
DMee
Connoisseur
DMee began at the beginning.
 
Posts: 90
Karma: 10
Join Date: Jun 2011
Location: Stockport, UK
Device: ipad, iphone, kindle
Quote:
Originally Posted by pelikano View Post
Hi, I'm using the plugin and it works great. But is there a way to split the duplicates in two groups: unicates(first found or original) and duplicates(all the other copies)
So I can delete all the duplicates at once?
The first may not be the best - however, once you've decided on the best (Highlight line, click V to view, repeat....), you can then click the one you want, then hold CTRL and click all the other duplicates, then rightclick, Edit Metadata, Merge books, and pick which option suits (usually the first)

If you will always pick the first merge option then pick best book, CTRL click rest and press M
DMee is online now   Reply With Quote
Old 07-15-2012, 08:20 AM   #263
pelikano
Junior Member
pelikano began at the beginning.
 
pelikano's Avatar
 
Posts: 4
Karma: 10
Join Date: Jul 2012
Device: sony prs-t1, iPad, HTC
I'm aware of
Quote:
The first may not be the best
but I can MAKE the first one the best. There are more lovely plugins to enrich the metadata. I know the procedure to merge the meta-data but I have 4196 duplicates....Althought it is a rainy sunday I prefer an automated procedure.
It should be lovely if the split function comes with the plugin...
pelikano is offline   Reply With Quote
Old 07-15-2012, 09:15 AM   #264
DMee
Connoisseur
DMee began at the beginning.
 
Posts: 90
Karma: 10
Join Date: Jun 2011
Location: Stockport, UK
Device: ipad, iphone, kindle
The metadata isn't the problem. What if one file is a crappy OCR that hasn't worked well and the second is a beautifully formatted copy. Just because first one has better metadata doesn't mean it's the one you want to keep :-)
DMee is online now   Reply With Quote
Old 07-15-2012, 10:20 AM   #265
pelikano
Junior Member
pelikano began at the beginning.
 
pelikano's Avatar
 
Posts: 4
Karma: 10
Join Date: Jul 2012
Device: sony prs-t1, iPad, HTC
I think in opportunities not in threaths. But I just count and sorted out 50 duplicates(all with isbn, thanks to the plugin "Extract ISBN") and the book quality was the same. I just want to get rid of the doubles.
pelikano is offline   Reply With Quote
Old 07-15-2012, 10:28 AM   #266
DMee
Connoisseur
DMee began at the beginning.
 
Posts: 90
Karma: 10
Join Date: Jun 2011
Location: Stockport, UK
Device: ipad, iphone, kindle
:-) unfortunately there's no "which one reads best " plugin. Maybe there's your opportunity ;-)

I'm just doing same so sympathise but there's no easy way :-(
DMee is online now   Reply With Quote
Old 07-15-2012, 11:40 AM   #267
pelikano
Junior Member
pelikano began at the beginning.
 
pelikano's Avatar
 
Posts: 4
Karma: 10
Join Date: Jul 2012
Device: sony prs-t1, iPad, HTC
Hi DMee, I'm not looking for a "which one reads best" plugin. Ik just would like to suggest a feature request for the duplicate plugin. All the functionality is already present in the plugin. The only extended functionality I need is a marking the doubles in a group or label. So I can decide not showing them in the database, check them or delete all at once.
pelikano is offline   Reply With Quote
Old 07-15-2012, 02:13 PM   #268
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@pelikano - to be honest I am not exactly sure what you are asking for?

Originally you asked to somehow separate the first match in each duplicate group. As others have explained I can't see how that is useful, because neither the formats nor the metadata associated with any particular book in a duplicate group can be assumed to be the one you should "keep", be it the first, last or any other. The only way to decide which one to keep is to manually investigate the associated formats and judiciously merging/deleting until you only have one book left. There are no shortcuts in this process without screwing up the quality of your library. I'm afraid you created your 4196 duplicates, so it is you that has to clean up your mess .

Then in your last post you talk about "marking the doubles in a group or label". Again I don't know what you mean by that. The plugin already "marks" each group behind the scenes temporarily for the purposes of stepping through the result groups so you can resolve them, seeing only the duplicates in that group. You can step through the results and defer decisions on on group, and you can just run the plugin again to bring the results back up. So what is it that it doesn't do?
kiwidude is offline   Reply With Quote
Old 07-15-2012, 02:22 PM   #269
DMee
Connoisseur
DMee began at the beginning.
 
Posts: 90
Karma: 10
Join Date: Jun 2011
Location: Stockport, UK
Device: ipad, iphone, kindle
@pelikano

The plugin shows all the duplicates, group by group - you tweak whether its one group at a time or every duplicate group in the settings. Once you've dealt with a group reclicking the "Find Duplicates" button on the toolbar will highlight the next group of duplicates to deal with.
DMee is online now   Reply With Quote
Old 07-16-2012, 10:19 AM   #270
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Quote:
Originally Posted by DMee View Post
3)OK, here's what I did before getting error - dunno if recreatable.....
Thanks for that. I have managed to replicate it. If you remove an author that would have been a duplicate candidate and then re-run the metadata variations search it will blow up. It is because of stale data in the calibre database. I query the database to ask for a list of authors, but that will still include authors which actually aren't linked to any books any more (e.g. an author like III which you had just eliminated).

I'll PM chaley - it is something we have discussed previously and I thought had been "fixed" in such a way that I did not have to do a "cleanup" call before I do every query but obviously not. At least I can see why it is happening...
kiwidude is offline   Reply With Quote
Reply

Tags
cross library duplicates, in library duplicates

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Generate Cover kiwidude Plugins 489 08-15-2014 09:39 AM
[GUI Plugin] Quality Check kiwidude Plugins 738 08-02-2014 10:06 PM
[GUI Plugin] View Manager kiwidude Plugins 82 08-01-2014 12:37 PM
[GUI Plugin] Open With kiwidude Plugins 228 07-31-2014 01:06 AM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 07:02 AM.


MobileRead.com is a privately owned, operated and funded community.