Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 09-09-2012, 08:10 PM   #316
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,870
Karma: 5654321
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by BelgarionNL View Post
please tell me there is a way to say: DELETE ALL DUPLICATE FOUND!

I am currently have 1 library with around 40 procent dublicates...

they are 1 on 1 duplicates from an old library!! how do I remove the copies?

finding is great but I am not deleting them 1 by 1


if there is way to remove the duplicates PLEASE tell me!

thx
Which of the 2 duplicates do you delete?
Are the the same format or different (and a candidate to be merged)?


+IF+ you have the list show all at once (grouped), you can Multi-select (select the first to delete, then hold the Ctrl key and select more) lines, then tap Del
theducks is offline   Reply With Quote
Old 09-09-2012, 08:16 PM   #317
BelgarionNL
Member
BelgarionNL began at the beginning.
 
Posts: 21
Karma: 10
Join Date: Jul 2011
Device: Sony PRS 650
all same format etc but now I have the duplicates marked but I cannot sort them to top...

so now group selecting

right now it is book:

A
A
B
B
C
C
D
D

now how do I remove 1 a/b/c/d etc ???

Last edited by BelgarionNL; 09-09-2012 at 09:00 PM.
BelgarionNL is offline   Reply With Quote
Old 09-09-2012, 09:06 PM   #318
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,870
Karma: 5654321
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by BelgarionNL View Post
all same format etc but now I have the duplicates marked but I cannot sort them to top...

so now group selecting

right now it is book:

A
A
B
B
C
C
D
D

now how do I remove 1 a/b/c/d etc ???
click one of each as I outlined above, then tap the Del key
theducks is offline   Reply With Quote
Old 09-09-2012, 09:12 PM   #319
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,870
Karma: 5654321
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by BelgarionNL View Post
all same format etc but now I have the duplicates marked but I cannot sort them to top...

so now group selecting

right now it is book:

A
A
B
B
C
C
D
D

now how do I remove 1 a/b/c/d etc ???
Moderator Notice
Don't start a new thread on this subject. Stay in the same thread and/or request it be moved to another section. I deleted the New one you just started
theducks is offline   Reply With Quote
Old 09-09-2012, 10:07 PM   #320
JSWolf
Resident Curmudgeon
JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.
 
JSWolf's Avatar
 
Posts: 37,711
Karma: 18475602
Join Date: Nov 2006
Location: Roslindale, Massachusetts
Device: Sony Reader PRS-650, iPad, nook STR
Can this plugin be made such that it does a CRC compare so we know that the duplicates are the exact same version?
JSWolf is offline   Reply With Quote
Old 09-09-2012, 10:38 PM   #321
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 8,861
Karma: 12755553
Join Date: Feb 2009
Location: North Carolina
Device: Nexus 7
Quote:
Originally Posted by JSWolf View Post
Can this plugin be made such that it does a CRC compare so we know that the duplicates are the exact same version?
An exact binary compare is already part of this plugin.
DoctorOhh is offline   Reply With Quote
Old 09-10-2012, 03:10 AM   #322
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,228
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@BelgarionNL - you said you created the duplicates by moving them from another library. Are these the exact same books, or slight variations (e.g. different formats, different versions of the same epub etc).

If they are binary identical, then you can use the Binary Compare search, and tick the box which automatically removes duplicates. That will remove the book format files, but still leave you with the book rows in calibre. If indeed those rows were created purely from identical copies of the book then one of each of those duplicate pairs will now have no book formats associated with it. So you can do a search for formats:false and then delete those to finish the cleanup operation. The plugin does not automatically do this because there might be metadata on those rows (like a better cover) that you may want to keep by manually merging them.

However if the books you merged in are not binary identical, then it is entirely up to you to go through one by one and decide which version of a format you want to keep. There is no tool on the planet that can make that decision for you - only a human eyeballing them can decide which one is of better quality since it is so subjective.
kiwidude is offline   Reply With Quote
Old 09-10-2012, 07:35 AM   #323
BelgarionNL
Member
BelgarionNL began at the beginning.
 
Posts: 21
Karma: 10
Join Date: Jul 2011
Device: Sony PRS 650
yes they were 100 procent duplicates! So I did what you said! thx!

still I would prefer to have auto delete then to go through 3000 books...
BelgarionNL is offline   Reply With Quote
Old 09-10-2012, 08:12 AM   #324
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,228
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@BelgarionNL - I don't see this plugin won't be changing to cater for such an extremely rare (and silly) thing to do .

I explained above the reasons why the plugin does not auto-delete book records. It would have to compare every single field and custom column of the book record, and compare image contents to be 100% sure that two records are identical, it is such a minor edge case I cannot be bothered trying to write the code for it. Any thing less than that strict, and deleting becomes an arbitrary decision which may result in someone losing their perfect retail ePub copy and being left with some corrupted calibre conversion or whatever. It is not a responsibility I have any intention of taking any possibility of blame for.

As I mentioned several times on the original development thread for this plugin I always envisaged there would perhaps be one day a more intelligent "Smart Merge" plugin that would partner this one. So having identified the duplicates, you can use Smart Merge to help figure out how to resolve them and do auto-merging where safe etc. But no-one has written it yet, and it isn't on my todo list in the forseeable future either. By being more intelligent with your workflow of adding books to your library you can minimise the likelihood of duplicates occurring in the first place, and then just periodically scan with this plugin to catch any failures in that process.
kiwidude is offline   Reply With Quote
Old 09-14-2012, 01:20 AM   #325
chis
Junior Member
chis began at the beginning.
 
Posts: 5
Karma: 10
Join Date: Jul 2012
Device: Kindle Touch
@BelgarionNL - I think your issue now is that while Find Duplicates can automatically remove binary duplicate formats, you're left with all the book entries with no book formats (ie. book files) attached. You'd like to delete all those empty book entries because they're duplicates of their matching book entry. If that is your problem, I have a solution.

Add a custom field "format". It's a standard custom field so real easy to add.
Choose to display that new field in your book list.
Now you can see which books have book formats and which don't (because FindDuplicates nicely removed them for you).
Sort by format.
Now you can select and delete all the book entries with no formats real simply.
chis is offline   Reply With Quote
Old 09-14-2012, 01:28 AM   #326
chis
Junior Member
chis began at the beginning.
 
Posts: 5
Karma: 10
Join Date: Jul 2012
Device: Kindle Touch
Quote:
Originally Posted by kiwidude View Post
@chis,

Thanks for the kind words and glad the plugins have been useful to you.

The problem with the selection idea is that there is already a purpose for clicking on the right-side - it controls the ability to select/deselect items. As it is possible that you don't want to rename all of the variations that it found.

The way I saw people could handle the "this is not the name you were looking for" issue is that if you scroll down the list on the left hand side and find that variation there. All the permutations are available in that list.

I can't think of an alternative way just at the moment, though if someone has a suggestion I will consider it.

Enjoy the plugins.
How about right-clicking on the right-side list. That doesn't seem to do anything on that list at the moment (Win7x64)

Having this feature is very useful when you have a lot of variable authors and will work through them on several occasions. I can just work through the authors from top each time, not having to remember that I've already thought about some of them and put them aside to fix further down the list.
chis is offline   Reply With Quote
Old 10-15-2012, 06:26 AM   #327
RotAnal
Enthusiast
RotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheese
 
Posts: 41
Karma: 1234
Join Date: Sep 2012
Device: Onyx Boox M92
Dear Sirs,
I have noticed that many e-books files of mine only differ for a very few bytes. Is it possibile to use the plugin so that it finds file binary search in a adjustable fuzzy way?
Thanks for the attention.
RotAnal is offline   Reply With Quote
Old 10-16-2012, 10:56 PM   #328
ilovejedd
hopeless n00b
ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.
 
ilovejedd's Avatar
 
Posts: 2,312
Karma: 5761596
Join Date: Jan 2009
Location: in the middle of nowhere
Device: PRS-350, Nexus S, Galaxy S, Nook Color, iPhone4, iPT4, iPad 2012
Quote:
Originally Posted by RotAnal View Post
I have noticed that many e-books files of mine only differ for a very few bytes. Is it possibile to use the plugin so that it finds file binary search in a adjustable fuzzy way?
Assuming you're dealing with epub or similarly compressed files, those few bytes in file size can mean quite a big difference in the actual binary content. For any meaningful comparison to be made, you'd need to decompress/unpack the ebooks which I reckon, would take too much time unless you've got a tiny library.

Last edited by ilovejedd; 10-16-2012 at 10:59 PM.
ilovejedd is offline   Reply With Quote
Old 10-18-2012, 03:24 AM   #329
RotAnal
Enthusiast
RotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheese
 
Posts: 41
Karma: 1234
Join Date: Sep 2012
Device: Onyx Boox M92
thank you for your answer, ilovejedd, but maybe I did not let myself understood. When I wrote "only differ for a very few bytes" I did not mean differences in length, but typical, minimal differences in actual bytes.
However, I ignore the way by which that comparison is performed in the plug in.
RotAnal is offline   Reply With Quote
Old 10-20-2012, 04:40 AM   #330
RotAnal
Enthusiast
RotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheeseRotAnal can extract oil from cheese
 
Posts: 41
Karma: 1234
Join Date: Sep 2012
Device: Onyx Boox M92
Anyway, changing the duplicate binary comparison algorithm, so that it can also provide some bits of fuzzy logic would be a cool thing.
RotAnal is offline   Reply With Quote
Reply

Tags
cross library duplicates, in library duplicates

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Generate Cover kiwidude Plugins 502 10-10-2014 06:47 AM
[GUI Plugin] Open With kiwidude Plugins 232 10-09-2014 12:38 AM
[GUI Plugin] Quality Check kiwidude Plugins 785 10-06-2014 05:25 PM
[GUI Plugin] View Manager kiwidude Plugins 83 09-24-2014 07:00 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 12:25 AM.


MobileRead.com is a privately owned, operated and funded community.