Find duplicated items in library

[dricciardi]

Hi all,

i have a question that i didn't find any answer in the forum or in the Faq.

I bulk imported a full ebook library of a lot of books in any formats. How can i find duplicates after import has finished?

thank you

DR

[itimpi]

It all depends on the state of the metadata! If the titles are OK then you can simply sort by title to get duplicates next to each other. Then you could try using the "merge" facility to get these down to a single entry.

If the metadata is not OK so that the titles are all over the place then I do not think there is any easy way of doing this.

[dricciardi]

Thank you a lot. I think a long long data entry work is waiting me :-)

[Starson17]

Originally Posted by dricciardi

I think a long long data entry work is waiting me :-)

I find duplicate titles by opening up the metadata.db file in sqlitespy and using an sql query.

Alternatively, this will also do it:

calibre-debug -c "from calibre.library.database2 import LibraryDatabase2; db = LibraryDatabase2('/path/to/library/folder');dupes = db.conn.get('select title from books group by title having count(*) > 1;');print dupes;">dupes.txt

Copy that long line, change the path to point to your library folder and paste it into a command window (terminal window, dos box, whatever you call it) and it will produce a file in the current directory called dupes.txt with all your duplicate titles in it. It's a list of dupe titles in unicode format.

[Sabardeyn]

Starson17,

Originally Posted by Starson17

calibre-debug -c "from calibre.library.database2 import LibraryDatabase2; db = LibraryDatabase2('/path/to/library/folder');dupes = db.conn.get('select title from books group by title having count(*) > 1;');print dupes;">dupes.txt

This will only find exact title matches, possibly ignoring uppercase, correct? So it cannot find any titles that are slightly different.

[Starson17]

Originally Posted by Sabardeyn

This will only find exact title matches, possibly ignoring uppercase, correct? So it cannot find any titles that are slightly different.

Correct. It's just an SQL query and finds exact title matches only. I suppose you could revise the query to do some fuzzy matching (but then the line would get really long.)