[GUI Plugin] Find Duplicates

[obvious]

Would it be feasible to add an option to find duplicates by content? CPU/IO heavy no doubt but what do you think?

Just a thought

[BookJunkieLI]

Originally Posted by obvious

Would it be feasible to add an option to find duplicates by content? CPU/IO heavy no doubt but what do you think?

Just a thought

That would be extremely CPU/IO heavy and likely to crash larger libraries. More feasible, I'd think and wouldn't mind seeing, is the ability to compare content once a duplicate has been found via the title/author search. I haven't upgraded yet but it does look like 2.0 has the option to compare two books side by side in the book editor. That might be an option once you've found duplicates that are identical except for page/word count.

[ymmv]

Is it possible to make this plugin find duplicates just for the currently selected books instead of all books?

[BetterRed]

Originally Posted by ymmv

Is it possible to make this plugin find duplicates just for the currently selected books instead of all books?

One way is to run Find Duplicates within a Virtual Library - such as

Mark the books to be considered and create a VL Books4FDCheck for marked:true or mark the books to be excluded and create a VL Books4FDCheck for not marked:true

Run Find dups while you're in the Books4FDCheck VL

BR

[rebl]

Originally Posted by obvious

Would it be feasible to add an option to find duplicates by content? CPU/IO heavy no doubt but what do you think?

Just a thought

I've been trying to use the "count words" plugin instead, in order to identify different formats with the same content.
If the content is the same, the word count should be the same, right?.... err... wrong . My results with that plugin were inconclusive for the moment (e.g. books with supposedly same content (such us donwloaded from the same source like guttenberg, in different formats) actually had a way different words count. Still testing to see what's wrong.

[AnselmD]

Originally Posted by theducks

Did you fianally understand "eyeball MkI" ?

It means, give them a fast look. In Grid view (no Details or Tags showing)you can usually see 15 to 60 (17" mon) covers at a glance. Page down for the next batch. It takes 5 seconds on newer CPU to paint all the covers on the page

Click on a bad one then Tap E to edit (or right click for more options)
Not blazing fast, but that is how I do it

Thank you for the hint of "eyeball MkI".

I marked them in the Grid view and then i load the covers with "load metadata and covers".

[AnselmD]

I like to use the "find duplicates" plugin to eliminate binary duplicates.

But the "calibre bookmark file" in the epub files makes books, which are "binary equal", unequal.

So, add the same book to calibre, view it with the viewer and scroll a little bit down and it is not "binary equal" any more (comparing to the original).

Going into the edit mode, check the book and remove the "book mark file" and the files are still no "binary equal".

Does anyone have a good idea to avoid this behaviour?

[DoctorOhh]

Originally Posted by AnselmD

Does anyone have a good idea to avoid this behaviour?

Open a book in calibre's viewer. Then go to the viewer's Preferences - Miscellaneous options and uncheck "Remember the current page when quitting" then click OK.

[AnselmD]

Originally Posted by DoctorOhh

Open a book in calibre's viewer. Then go to the viewer's Preferences - Miscellaneous options and uncheck "Remember the current page when quitting" then click OK.

Thank you, this helps me for the future.

[AnselmD]

Originally Posted by AnselmD

I like to use the "find duplicates" plugin to eliminate binary duplicates.

But the "calibre bookmark file" in the epub files makes books, which are "binary equal", unequal.

So, add the same book to calibre, view it with the viewer and scroll a little bit down and it is not "binary equal" any more (comparing to the original).

Going into the edit mode, check the book and remove the "book mark file" and the files are still no "binary equal".

Does anyone have a good idea to avoid this behaviour?

Ok, i think i should not delete the bookmark file in Calibre edit. It makes some changes in a metainfo file.

To delete the bookmark file to revert it to the original file, i uses zip.

To do it at all epub files in the library i used:

find . -name "*.epub" -exec zip -d "{}" META-INF/calibre_bookmarks.txt \;

[theducks]

Originally Posted by AnselmD

Ok, i think i should not delete the bookmark file in Calibre edit. It makes some changes in a metainfo file.

To delete the bookmark file to revert it to the original file, i uses zip.

To do it at all epub files in the library i used:

find . -name "*.epub" -exec zip -d "{}" META-INF/calibre_bookmarks.txt \;

You can use Unpack: (delete the file): Rebuild

Also, you can configure the Editor to NOT update metadata.

IN THE EDITOR: Edit: Preferences: <integration with Calibre>: change the tick

[Mashkai]

I hope this is in the right place:/

So, I used Find Duplicates and I had roughly 7k duplicates. (Not a s'prise, really) After an hour or so of clicking on individual books and deleting, I thought maybe I could make it a bit easier. I did a binary search and checked "When doing a binary compare, automatically remove duplicate formats" thinking this would make my life much easier. Unfortunately, now I have the same problem as I did to begin with, I am back to lengthy time spent clicking on each individual book. The main difference being that half of them are empty folders so I must slow down a bit further to pay attention and make sure I am deleting the empty folder rather than the one containing an ebook. Is there a faster or better way to do this? Or at least a way to find and remove the now empty file folders?

-so frustrated-

Edit: I found the solution and having spent -hours- on this now feel like an idiot for not seeing it sooner:/

[AnselmD]

Originally Posted by theducks

Also, you can configure the Editor to NOT update metadata.

IN THE EDITOR: Edit: Preferences: <integration with Calibre>: change the tick

Oh! Thanx, i changed it.
Is this a good idea to have this checked as calibres default behaviour?

Originally Posted by theducks

You can use Unpack: (delete the file): Rebuild

I am not sure, if i understand it correctly:
I understand: unpack manually with an external tool. delete the file. pack ths staff again?

Is there i reason why you emphasize the unpack and underline the U?


Edit: Now i understand, i found the keyboard shortcut for unpack. But it does not work. I am using calibre 1.48 for windows and linux. But i can put the Unpack into the context menu and use it with right mouse click.

For me my version is the easy way. Use the zip -d (delete file) command recursive over all directories to delete all the bookfiles in all epub files.

[AnselmD]

Originally Posted by Mashkai

Edit: I found the solution and having spent -hours- on this now feel like an idiot for not seeing it sooner:/

I do not know your solution, but i wanted to give you some hints.

Check the "one group in a file" in the "find duplicates" menu.

Define keyboard shortuts (preferences->keyboard->Find Duplicates) for "next result" "previous result".

and

Define keyboard shortuts (preferences->keyboard->Edit Meta Data)
Merge Book Records

Use the Merge Book Records to merge the empty records with the stiil existing ones. Title, author, series, image, will be replaced (taking the first marked book). Tags, comments etc are appended.

Edit: Or you can search all records with no formats (books) with
formats:false
and delete them all

[MidwestJen]

Every time I open Calibre, my library opens with all of my duplicate exemptions marked. Is there a way for it to remember that without re-marking them each time? Obviously it just takes a few seconds to 'clear all marked books', but it's a tad annoying. It's also a bit baffling that I couldn't really find anyone else mentioning it (then again, I could be using a bad set of keywords to search on).

Is there a setting/preference I'm missing somewhere or is this normal and necessary behavior?

Thank you for any advice or help you can give on this. =)