Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 05-18-2011, 07:35 PM   #46
Narcio
Junior Member
Narcio began at the beginning.
 
Posts: 2
Karma: 10
Join Date: May 2011
Device: none
A binary compare for duplicates during merging is exactly what I thought would be helpful to avoid all of the trouble I'm having now. So yes, I see your point on the merge plugin side of things.

In the second paragraph you said that you think its unnecessary for a merging plugin to allow a user to look and choose between metadata from either of the potential duplicate books. I think this actually would be a useful option, and a merge plugin like what you described with this feature would solve 99% of my problems before they happened.

I agree with you that the root of the problem is in the merging.

Your plugin is extraordinarily useful nonetheless.
Narcio is offline   Reply With Quote
Old 05-18-2011, 08:20 PM   #47
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Quote:
Originally Posted by Narcio View Post
In the second paragraph you said that you think its unnecessary for a merging plugin to allow a user to look and choose between metadata from either of the potential duplicate books. I think this actually would be a useful option, and a merge plugin like what you described with this feature would solve 99% of my problems before they happened.
My concern with trying to offer flexible options for merging metadata would be coming up with a UI design which doesn't make it all too horribly complicated to use.

Not that I would completely ignore metadata with an initial plugin version of course (at risk of sounding like I am writing this here but I'm not - well not yet at least!). Perhaps a few (state persisted) checkboxes could give a little more control/flexibility over what you have in Calibre today.

For instance I know Starson17 loves and defends this behaviour, but I disagree with the concept of merging comments where they differ (rather than just where there are none). There are also scenarios where people do not want to merge tags. I think these two stand in isolation from the rest of merging metadata, as everything else is considered as a "replace if empty".

Starson17 was kind enough to put in a "merge formats only" option into Calibre to replace my own local patching of the code. I think this smart merge plugin could just have a few checkboxes in it giving options like "Merge formats only" and if unchecked some granular choices of merge behaviour for the comments and tags fields. I think perhaps something like that would satisfy the 98% scenario.

You could potentially also consider covers - should the user be able to choose which of multiple covers should survive, or should it just that of the "master" (first selected record chosen before starting the merge operation).

It's a slippery slope and a challenge I think to find the right balance between keeping the merge sufficiently simple and non-intimidating versus flexibility. The existing merge operations in calibre choose the "simple" route, with some flexibility provided by mutually exlusive merge menu options. I would like a smart merge plugin to act as a superset of those in a single dialog. A power user who knows that they want to do a merge copy or whatever can still use the existing calibre menu options, but anyone else who wants a bit more control and assistance with understanding which format of multiple to keep etc could use the smart merge plugin.

As with a number of my posts, it is just me thinkout out loud at this point... I'm glad you are finding the duplicates plugin useful anyways.
kiwidude is offline   Reply With Quote
Old 05-19-2011, 02:18 PM   #48
Partzz
Junior Member
Partzz began at the beginning.
 
Posts: 9
Karma: 10
Join Date: Jul 2010
Device: Blackberry
Thank-you. Plug-in works FAST. Did a similar search using title and author, took about 10 seconds to go through 24,000+ titles with 360 dupes found! Good Job!
Partzz is offline   Reply With Quote
Old 05-19-2011, 11:47 PM   #49
speakingtohe
Wizard
speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.
 
Posts: 4,569
Karma: 24987036
Join Date: Apr 2010
Device: sony PRS-T1 and T3, Kobo Mini and Aura HD, Tablet
Perfection. Thank you
Helen
speakingtohe is offline   Reply With Quote
Old 05-20-2011, 08:34 AM   #50
geordiejohn
Connoisseur
geordiejohn began at the beginning.
 
geordiejohn's Avatar
 
Posts: 50
Karma: 10
Join Date: Dec 2010
Location: London
Device: Kindle 3
after reading through all the topics i have finally got it to work and it is now showing on my toolbar,thank you very much.
geordiejohn is offline   Reply With Quote
Old 05-20-2011, 12:44 PM   #51
geordiejohn
Connoisseur
geordiejohn began at the beginning.
 
geordiejohn's Avatar
 
Posts: 50
Karma: 10
Join Date: Dec 2010
Location: London
Device: Kindle 3
i spoke too soon,when i connect my Kindle the find duplicates goes away,i have tried to setup up so when Kindle is connected to show duplicates.
any ideas what i am doing wrong please as i would like to find duplicates on my Kindle ?
thank you.
geordiejohn is offline   Reply With Quote
Old 05-20-2011, 12:48 PM   #52
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Go to Preferences->Toolbar, and change the dropdown on the top right to "The main toolbar when a device is connected". Or similarly for the context menu if that is where you put it. There are different toolbars/menus that you customise based on whether a device is or is not connected.
kiwidude is offline   Reply With Quote
Old 05-20-2011, 01:08 PM   #53
geordiejohn
Connoisseur
geordiejohn began at the beginning.
 
geordiejohn's Avatar
 
Posts: 50
Karma: 10
Join Date: Dec 2010
Location: London
Device: Kindle 3
i went to pref-toolbar and main toolbar when device connected but i cannot apply as it is highlighted,i have to cancel ?
geordiejohn is offline   Reply With Quote
Old 05-20-2011, 01:15 PM   #54
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Once you have switched the dropdown to "Main toolbar when a device is connected", you still need to actually make changes to that toolbar before the Apply button will not be greyed out. So you will want to select the "Find Duplicates" plugin from the left side of Available Actions, click the right arrow to put it onto that particular toolbar you chose in the dropdown. Once you have done that you will be able to click Apply.
kiwidude is offline   Reply With Quote
Old 05-20-2011, 01:20 PM   #55
geordiejohn
Connoisseur
geordiejohn began at the beginning.
 
geordiejohn's Avatar
 
Posts: 50
Karma: 10
Join Date: Dec 2010
Location: London
Device: Kindle 3
by jove i think i have done it.
thank you.
geordiejohn is offline   Reply With Quote
Old 05-22-2011, 05:17 AM   #56
drMerry
Addict
drMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmos
 
drMerry's Avatar
 
Posts: 293
Karma: 21022
Join Date: Mar 2011
Location: NL
Device: Sony PRS-650
Hi Kiwidude,

An interesting error:
I did a bit compare and found these duplicates:


The interesting part is, I had 500 duplicates. I selected every first one and deleted them (it's bit, so true-duplicate).
When I found these duplicates, I got curious because of the many dups. I did see the different size of books and did an external duplicate check.
It occurred that only the epubs are duplicates.

Is there a way to tell the user there is no complete match on all formats, or to show which formats are dups?

Some cases with (by me) desired output (assume same type is duplicate):
Book A - txt
Book A - txt, epub
Book A - epub

return on dupcheck:
Book A - txt
Book A - txt, epub - 50% match
Book A - epub

return on dupcheck of txt files:
Book A - txt
Book A - txt, epub - 50% match

So, when 100% dup, no extra info needed, otherwise there is more info
An other option could be to create several filters:
marked:complete_duplicate_group_x
markedartial_duplicate_group_x
marked:all_duplicate_group_x
drMerry is offline   Reply With Quote
Old 05-22-2011, 09:06 AM   #57
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@drMerry - I believe this is the same issue that I have recently discussed in several posts starting at the end of the previous page. This is a merge issue, not a duplicate finding issue. Duplicate find has done its job and told you which books are duplicates. Whether they contain duplicate formats or complementing formats and how you get those down to a single book record is now all about the merging, which I posted my thoughts on.
kiwidude is offline   Reply With Quote
Old 05-22-2011, 12:16 PM   #58
drMerry
Addict
drMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmos
 
drMerry's Avatar
 
Posts: 293
Karma: 21022
Join Date: Mar 2011
Location: NL
Device: Sony PRS-650
You are right about that.
I was only thinking about a possibility for extra warnings at this point. But it will be a hard nut to crack I suppose. And at the other hand, most of the functions (except this one) show POSSIBLE dups, so it is up to the user to check...
drMerry is offline   Reply With Quote
Old 05-24-2011, 03:29 AM   #59
htweedie
Junior Member
htweedie began at the beginning.
 
Posts: 2
Karma: 10
Join Date: May 2011
Device: Kindle
I seem to be having an error when I try and run the plugin in Calibre 0.8.2 on OSX.

It seems to have installed okay (find the installer plugin) and on the title bar.

When I click the title bar I get the error RuntimeError: maximum recursion depth exceeded while calling Python object with the following traceback/details:

Appreciate any help :-)

calibre, version 0.8.2
ERROR: Unhandled exception: <b>RuntimeError</b>:maximum recursion depth exceeded while calling a Python object
Spoiler:

Traceback (most recent call last):
File "site-packages/calibre/gui2/library/views.py", line 765, in search_proxy
File "site-packages/calibre/gui2/library/models.py", line 298, in search
File "site-packages/calibre/library/caches.py", line 773, in search
File "site-packages/calibre/library/caches.py", line 791, in search_getting_ids
File "site-packages/calibre/utils/search_query_parser.py", line 193, in parse
File "site-packages/calibre/utils/search_query_parser.py", line 201, in _parse
File "site-packages/calibre/utils/pyparsing.py", line 1089, in parseString
File "site-packages/calibre/utils/pyparsing.py", line 962, in _parseNoCache
File "site-packages/calibre/utils/pyparsing.py", line 2681, in parseImpl
File "site-packages/calibre/utils/pyparsing.py", line 962, in _parseNoCache
File "site-packages/calibre/utils/pyparsing.py", line 2536, in parseImpl
File "site-packages/calibre/utils/pyparsing.py", line 962, in _parseNoCache
...
File "site-packages/calibre/utils/pyparsing.py", line 293, in __init__
RuntimeError: maximum recursion depth exceeded while calling a Python object

Last edited by kiwidude; 05-24-2011 at 05:04 AM. Reason: please don't EVER post thousands of repeated lines as it makes a thread unreadable
htweedie is offline   Reply With Quote
Old 05-24-2011, 05:09 AM   #60
drMerry
Addict
drMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmos
 
drMerry's Avatar
 
Posts: 293
Karma: 21022
Join Date: Mar 2011
Location: NL
Device: Sony PRS-650
Is it possible to change settings on preferences->(advanced) Plugins->User Interface Action Plugins->Duplicate Check

===

EDIT (I did mean Duplicate off-course)

Last edited by drMerry; 05-24-2011 at 05:33 AM.
drMerry is offline   Reply With Quote
Reply

Tags
cross library duplicates, in library duplicates

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Quality Check kiwidude Plugins 730 Today 02:50 AM
[GUI Plugin] Generate Cover kiwidude Plugins 479 07-10-2014 07:35 PM
[GUI Plugin] View Manager kiwidude Plugins 77 07-09-2014 07:40 PM
[GUI Plugin] Open With kiwidude Plugins 223 07-09-2014 02:42 AM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 03:17 AM.


MobileRead.com is a privately owned, operated and funded community.