Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 09-03-2010, 02:14 PM   #1
pghaworth
Zealot
pghaworth doesn't litterpghaworth doesn't litter
 
Posts: 128
Karma: 128
Join Date: Feb 2009
Device: Kindle
Question Help With Duplicate Book Titles

I'm trying to clean up my Calibre library and merge formats. Is there any way I can tell Calibre to look for and bring up only duplicate [or similar titles], other than sorting by title?

I've tried a couple of things but nothing is getting me where I need to go yet. All help and ideas appreciated.

Thanks a bunch!
pghaworth is offline   Reply With Quote
Old 09-03-2010, 03:03 PM   #2
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by pghaworth View Post
I'm trying to clean up my Calibre library and merge formats. Is there any way I can tell Calibre to look for and bring up only duplicate [or similar titles], other than sorting by title?
I find duplicate titles by opening up the metadata.db file in sqlitespy and using an sql query.

Alternatively, this will also do it:

Code:
calibre-debug -c "from calibre.library.database2 import LibraryDatabase2; db = LibraryDatabase2('/path/to/library/folder');dupes = db.conn.get('select title from books group by title having count(*) > 1;');print dupes;">dupes.txt
Copy that long line, change the path to point to your library folder and paste it into a command window (terminal window, dos box, whatever you call it) and it will produce a file in the current directory called dupes.txt with all your duplicate titles in it. It's a list of dupe titles in unicode format.
Starson17 is offline   Reply With Quote
Old 09-03-2010, 03:14 PM   #3
pghaworth
Zealot
pghaworth doesn't litterpghaworth doesn't litter
 
Posts: 128
Karma: 128
Join Date: Feb 2009
Device: Kindle
Thanks. I don't think this will get me where I need to go but maybe I'm not understanding it's full use. I was hoping to do this from inside Calibre.

If all else fails I will use this method but Calibre is such a clever program I feel sure there's something, that will bring a list of my duplicates up in Calibre, that I'm just missing or don't know how to do.
pghaworth is offline   Reply With Quote
Old 09-03-2010, 03:18 PM   #4
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by pghaworth View Post
Thanks. I don't think this will get me where I need to go but maybe I'm not understanding it's full use. I was hoping to do this from inside Calibre.
Sort by title and manually looking is the only way inside the GUI. There is no "press a button to show duplicates" function. I've posted my two workarounds.
Starson17 is offline   Reply With Quote
Old 09-03-2010, 03:21 PM   #5
pghaworth
Zealot
pghaworth doesn't litterpghaworth doesn't litter
 
Posts: 128
Karma: 128
Join Date: Feb 2009
Device: Kindle
Yes, and I found your answer in another thread, much older, as well. I'm not very good with either of the options in your answer. It's a lot like reading Latin for me at this point. I think I will have to limp along in Calibre at this point unless I learn Latin. Thanks again.
pghaworth is offline   Reply With Quote
Old 09-03-2010, 03:27 PM   #6
peterselizabeth
Junior Member
peterselizabeth began at the beginning.
 
Posts: 1
Karma: 10
Join Date: Sep 2010
Device: Kindle
pghaworth, have you tried selecting one book. right click, choose similar books in the context menu, then choose books by the same author. All books by that author will show in the list and its a matter of just selecting the dups and merging or deleting the ones with the same format? It may not be much better than sorting by title, but it depends on how many books you have. Good luck.
peterselizabeth is offline   Reply With Quote
Old 09-03-2010, 03:30 PM   #7
pghaworth
Zealot
pghaworth doesn't litterpghaworth doesn't litter
 
Posts: 128
Karma: 128
Join Date: Feb 2009
Device: Kindle
Hmmm... this may be just what I'm looking for. My list is ginormous and to go thru book by book is daunting, even sorted by titles. I'll give this a try and see if it gets me where I need to go any quicker. Thanks!
pghaworth is offline   Reply With Quote
Old 09-03-2010, 03:32 PM   #8
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by pghaworth View Post
Yes, and I found your answer in another thread, much older, as well. I'm not very good with either of the options in your answer. It's a lot like reading Latin for me at this point. I think I will have to limp along in Calibre at this point unless I learn Latin. Thanks again.
Duplicates were very annoying to me. That "much older" thread preceded the Merge function, which I wrote so I could merge them once I found them.

It also preceded the autosort/automerge function I wrote to avoid creating duplicates in the first place. Note that the autosort/automerge function is located in Preferences|Add/Save|"If books with similar authors and titles found, merge automatically" and it's off by default. That means you will often create duplicates if you add two different formats for the same book. You may want to turn it on to prevent them from being formed in the first place. I seldom see them any more.
Starson17 is offline   Reply With Quote
Old 09-03-2010, 03:39 PM   #9
pghaworth
Zealot
pghaworth doesn't litterpghaworth doesn't litter
 
Posts: 128
Karma: 128
Join Date: Feb 2009
Device: Kindle
I don't want to turn it on because I want the different versions to show and be added for the time being. I want to be able to view my device versions and I have set up things so that I can differentiate.

I do have a question for you tho' that is unrelated to above. I have an old Kindle and a new one and I have versions of a given book for each device. Can I merge those two azw file records but keep the device integrity?
pghaworth is offline   Reply With Quote
Old 09-03-2010, 03:47 PM   #10
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by Starson17 View Post
Sort by title and manually looking is the only way inside the GUI. There is no "press a button to show duplicates" function. I've posted my two workarounds.
OK, I lied. Just last week I wrote some special code that implements the autosorting/automerge function when doing a Copy to Library. That code was released minutes ago as 0.7.17.

You can do this:

1) create an empty library
2) make sure that the autosort/automerge option is on (the one with the long name referred to above in Preferences|Add/Save)
3) select all your books and right click then Copy to Library

This should cause all of your duplicates to be merged during the transfer. You should check carefully, as I've only tested it once. I hesitated to mention it as it hadn't even been released until I checked a minute ago.

If you're happy with the merge, use the new library.

Yes, Calibre changes minute by minute.

Full disclosure: When I say "I wrote ... code" what I really mean is I submit code that works (sort of) then Kovid tells me he's "refactored" it. That's a polite way of saying he's taken my crappy code and cleaned it up so it meets the high standards he keeps for Calibre.
Starson17 is offline   Reply With Quote
Old 09-03-2010, 03:50 PM   #11
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by pghaworth View Post
I don't want to turn it on because I want the different versions to show and be added for the time being. I want to be able to view my device versions and I have set up things so that I can differentiate.
If you want the dupes, why do you want to find them?

Quote:
I do have a question for you tho' that is unrelated to above. I have an old Kindle and a new one and I have versions of a given book for each device. Can I merge those two azw file records but keep the device integrity?
I'm not sure what you're asking, but if you're asking about keeping two files with the same extension in one record, the answer is no. I know little about the Kindle or azw files.
Starson17 is offline   Reply With Quote
Old 09-03-2010, 03:51 PM   #12
pghaworth
Zealot
pghaworth doesn't litterpghaworth doesn't litter
 
Posts: 128
Karma: 128
Join Date: Feb 2009
Device: Kindle
LOL Good one.

Calibre is an amazing program - I continue to be impressed daily. I will never be without it again in my life. The only real problem with it, in fact, is the ID10T error. I wish Kovid could take my code and clean it up.

I'll see if I can digest your instructions and apply without trashing my library.
pghaworth is offline   Reply With Quote
Old 09-03-2010, 03:56 PM   #13
pghaworth
Zealot
pghaworth doesn't litterpghaworth doesn't litter
 
Posts: 128
Karma: 128
Join Date: Feb 2009
Device: Kindle
Quote:
Originally Posted by Starson17 View Post
If you want the dupes, why do you want to find them?
If I have an epub and an azw, for instance, I want to temporarily keep them so that I can choose when one gets merged into the other. I did a lot of metadata clean-up of my records - well over 1K books - and then upgraded. So if I bring in a record with incomplete data then I can just tell it which one to merge into. I like control.


Quote:
Originally Posted by Starson17 View Post
I'm not sure what you're asking, but if you're asking about keeping two files with the same extension in one record, the answer is no. I know little about the Kindle or azw files.
Yes, two records, one extension. I have identifiers for which device it goes to. I don't want a separate library so for those records I will have to have duplicate titles.
pghaworth is offline   Reply With Quote
Old 09-03-2010, 03:59 PM   #14
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by pghaworth View Post
I'll see if I can digest your instructions and apply without trashing my library.
It's perfectly safe to try it, but don't give up your original library until you're sure the new one is in good shape. Remember, it will merge together duplicate books. A "duplicate book" is one which has the identical author and nearly the same title (ignoring capitalization, spaces and certain indefinite articles - "a," "an" or "the" at the beginning of the title.)
Starson17 is offline   Reply With Quote
Old 09-03-2010, 04:01 PM   #15
pghaworth
Zealot
pghaworth doesn't litterpghaworth doesn't litter
 
Posts: 128
Karma: 128
Join Date: Feb 2009
Device: Kindle
How does it decide which book to merge into which? Plus, with my different devices that have the same file name it would be really bad to merge those.
pghaworth is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Show Duplicate Titles NoBiscuit Calibre 13 05-30-2011 03:20 AM
Duplicate series titles? hairybiker Calibre 2 03-17-2010 06:27 AM
PRS 300: Duplicate titles on reader (and growing!) 747sardine Sony Reader 4 02-19-2010 04:37 PM
PRS 300 and duplicate titles rypleigh Introduce Yourself 1 12-27-2009 03:16 PM
Duplicate titles on PRS700 with SD card herbdb Sony Reader 5 01-26-2009 09:40 AM


All times are GMT -4. The time now is 08:23 PM.


MobileRead.com is a privately owned, operated and funded community.