Help With Duplicate Book Titles

[Starson17]

Originally Posted by pghaworth

How does it decide which book to merge into which? Plus, with my different devices that have the same file name it would be really bad to merge those.

There's another thread going on about whether I should integrate the Merge function (which merges all metadata) into the Copy to Library function. Currently only the autosorting function is integrated. That means that you will lose metadata found only in the second format if it has metadata not found in the first format. The order they are sent over is by sort order in the source Library.

If you are intentionally keeping books with the same author and title separate, then you don't want to do this.

[pghaworth]

Originally Posted by Starson17

The order they are sent over is by sort order in the source Library.

So before, if I were to do this, if I sort by date, oldest first, would it merge the newer records into the older, or vice versa?

[Starson17]

Originally Posted by pghaworth

So before, if I were to do this, if I sort by date, oldest first, would it merge the newer records into the older, or vice versa?

newer->older

Records are sent sequentially by sort order in the source library. With the option on, it checks the destination library before sending to see if formats in the record it's sending should go into an existing "matching" record.

You should be able to send some records with autosort on (thereby merging duplicates), then turn it off, send some more (thereby keeping duplicates) and have control by running searches selecting the records you send with it on versus off and changing sort order.

[pghaworth]

So, since what I need is older->newer I should then sort newest to oldest by date, correct?

I can't thank you enough for all your help. I have a lot going on this week but can't wait to test this out in the next week or so.

[Starson17]

Originally Posted by pghaworth

So, since what I need is older->newer I should then sort newest to oldest by date, correct?

Yes

I can't thank you enough for all your help.

You're welcome.

[brewjono]

Originally Posted by Starson17

I find duplicate titles by opening up the metadata.db file in sqlitespy and using an sql query.

Alternatively, this will also do it:

calibre-debug -c "from calibre.library.database2 import LibraryDatabase2; db = LibraryDatabase2('/path/to/library/folder');dupes = db.conn.get('select title from books group by title having count(*) > 1;');print dupes;">dupes.txt

Copy that long line, change the path to point to your library folder and paste it into a command window (terminal window, dos box, whatever you call it) and it will produce a file in the current directory called dupes.txt with all your duplicate titles in it. It's a list of dupe titles in unicode format.

Thanks very much for this. It worked for me, but took a while to work out which part was the path.

For people a bit slow like me I've included my path to help.

calibre-debug -c "from calibre.library.database2 import LibraryDatabase2; db = LibraryDatabase2('c:/users/frank/documents/my ebooks/calibre');dupes = db.conn.get('select title from books group by title having count(*) > 1;');print dupes;">dupes.txt

Using Vista I went to the start menu, selected "Run". When that came up I typed in "cmd" and enter. That brought up a traditional DOS window.

I copied the text from above into word where I edited it with my path, then pasted it into the dos window.

It then ran and I did a search for "dupes.txt". It was in the C: root. Opened it and manually went through Calibre.

I have almost 10,000 books and found almost 100 legitimate dupes.

Brewjono.

[Starson17]

Originally Posted by brewjono

Thanks very much for this.

You're welcome. Thanks for posting your steps, they may help others who have the default library location.

[PCreighton]

Hi thanks this helped a lot, quesiton any way it can show author as well I am finding duplicate books in dupe.txt however they really are different authors. Am I asking for to much.

[Starson17]

Originally Posted by PCreighton

Hi thanks this helped a lot, quesiton any way it can show author as well I am finding duplicate books in dupe.txt however they really are different authors. Am I asking for to much.

Try this:

calibre-debug -c "from calibre.library.database2 import LibraryDatabase2; db = LibraryDatabase2('c:/users/frank/documents/my ebooks/calibre');dupes = db.conn.get('select title, author_sort from books group by title having count(*) > 1;');print dupes;">dupes.txt

[PCreighton]

that works great thank you

[Starson17]

Originally Posted by PCreighton

that works great thank you

You're welcome. It probably should have been that way from the beginning, but it was getting awfully long and intimidating.