Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 12-22-2009, 08:10 PM   #1
pauldadams
Junior Member
pauldadams began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Dec 2009
Device: WinMobile
Angry Duplicates

I just started using Calibre. After reading in all of my ebooks and updating the meta data for each one, I copied (since I have a WinMobile device) my Calibre directory to my device. When I opened my library with Mobipocket I saw that I had 2 of every book.

When I looked through my Calibre directory on my XP notebook I found that I had at least 2 and in many cases 3 or 4 copies of the same book. I don't believe this should ever be allowed to happen, let alone for each book!

I took the time to import all books including duplicates back in to Calibre and went through each entry and deleted those copies that I don't want. Calibre warns that this will delete the file from my computer but this is exactly what I wanted.

All of the duplicates are still here. Additionally, the detail view of many books show the wrong book cover although the meta data view shows the correct cover. If I save the meta data the detail image will update to the correct cover until I move away and come back. Then the old, incorrect image appears in the detail view, although the meta data is still correct.

I now have a horrible mess on my hard disk which I will need to go through to delete one by one. I imagine a lot of correct meta data will also be deleted. If I go through to update the meta data, will I gain new duplicates?

This is definitely NOT what I expected to get!
pauldadams is offline   Reply With Quote
Old 12-24-2009, 04:35 AM   #2
polly
Evangelist
polly ought to be getting tired of karma fortunes by now.polly ought to be getting tired of karma fortunes by now.polly ought to be getting tired of karma fortunes by now.polly ought to be getting tired of karma fortunes by now.polly ought to be getting tired of karma fortunes by now.polly ought to be getting tired of karma fortunes by now.polly ought to be getting tired of karma fortunes by now.polly ought to be getting tired of karma fortunes by now.polly ought to be getting tired of karma fortunes by now.polly ought to be getting tired of karma fortunes by now.polly ought to be getting tired of karma fortunes by now.
 
polly's Avatar
 
Posts: 454
Karma: 270240
Join Date: Aug 2009
Device: Sony PRS 650, PocketBook 360, Astak PocketPro (RIP), Tungsten T3
It sounds like you went mucking about in the files maintained by Calibre. That's guaranteed to make a mess because Calibre is only aware of the changes it makes there.

The trick with your Calibre library is to leave it alone. Don't even open it to look inside. Hide it someplace deep in your directory if it offends you. Just leave it alone and let Calibre deal with it.

If you want to extract books from Calibre, Save to Disk will put the desired books wherever you want. Trust the metadata that you see within Calibre. It only gives me duplicate books when I add the same book twice, and then it warns me that that book is already in Calibre. Or were you talking about the same book in different formats?
polly is offline   Reply With Quote
 
Enthusiast
Old 12-25-2009, 11:23 AM   #3
pauldadams
Junior Member
pauldadams began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Dec 2009
Device: WinMobile
No, I didn't touch anything in the Calibre directory. Calibre doesn't recognize when my WinMobile device is plugged in (even though Windows does and I could save files there via Windows) so I had to do a copy of the Calibre directory to my device to move my books there. When I opened the library with Mobipocket mobile to read my books I noticed there were duplicates and then found the duplicate directories on my hard drive. I had multiple sub-directories for the same book just with a different number in parentheses at the end of the directory name.

I imported a the whole directory back in to Calibre and deleted the duplicates. Some I had to manually delete via Windows. My hard disk is now cleaned up but I continue to have issues with the wrong cover appearing in the detail view. The metedata view is correct and the jpg file in the directory is correct. So this is Calibre mis-reading data from metadata.db file, which I assure you I've never touched.
pauldadams is offline   Reply With Quote
Old 12-26-2009, 09:29 AM   #4
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by pauldadams View Post
had to do a copy of the Calibre directory to my device to move my books there.
You didn't have to copy the Calibre directory. You should have done a "save to disk" to a temporary directory on your hard drive, and then copied *that* directory. As polly said, you shouldn't ever even look inside the Calibre Library directory with Windows Explorer. Calibre will keep multiple copies of the same book in the same folder, but these aren't really duplicates - they are just different formats.

Quote:
I had multiple sub-directories for the same book just with a different number in parentheses at the end of the directory name.
The different numbers indicate that you imported the same book more than once. (The number is the ID that Calibre uses to identify a book) Calibre warns you when you try to do this. These are true duplicates. You can get rid of the second copy by deleting it, or if it is a different format from the first copy, by saving the second to disk, deleting it from Calibre and bringing it back as a different format of the first book.

Edit: BTW, I have four Win Mobile Devices, and the newest one can optionally connect through either ActiveSync or as a USB drive to Windows. Calibre can only talk directly to it when it's connected as a USB drive. Windows can only sync to it when connected via ActiveSync. You may want to check the manual for your Win Mobile device to see if it has this feature. It asks me when I first connect the USB cable how I want to connect.

Last edited by Starson17; 12-26-2009 at 09:48 AM.
Starson17 is offline   Reply With Quote
Old 04-29-2010, 11:52 AM   #5
Darryl_Gittins
Junior Member
Darryl_Gittins began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Jan 2010
Device: Sony Reader
I've already done this (mucked with the files) and totally botched my entire book collection. I now have a huge mess of duplicates.
Can anyone offer suggestions on how to start from scratch and clean up the mess?

Darryl_Gittins is offline   Reply With Quote
Old 04-29-2010, 04:07 PM   #6
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by Darryl_Gittins View Post
I've already done this (mucked with the files) and totally botched my entire book collection. I now have a huge mess of duplicates.
Can anyone offer suggestions on how to start from scratch and clean up the mess?

I suggest:

Get your Calibre GUI interface to the library cleaned up, so that you have one and only one entry for each book you want, with all the book formats for each and all metadata set correctly. You can stop there ... or ...

You can then Save to Disk with the .opf and .jpg options turned on (they are by default), then turn off Calibre, change the name of your Calibre library (so Calibre can't find the library) and Add Books all of the Books you Saved to Disk. That will give you a clean library, with all your metadata.

If you stopped before the last steps, you'd have some extra junk in your library, but you wouldn't see it and it wouldn't cause any harm other than eating up some space.

Edit: If you have duplicates in the database that appear in the GUI, you may want to wait until the next release of Calibre when the Merge function will be introduced. You'll be able to merge those dupes, with all the metadata.
Starson17 is offline   Reply With Quote
Old 04-29-2010, 06:32 PM   #7
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,022
Karma: 777817
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
Quote:
Originally Posted by Starson17 View Post
Edit: If you have duplicates in the database that appear in the GUI, you may want to wait until the next release of Calibre when the Merge function will be introduced. You'll be able to merge those dupes, with all the metadata.
The merge function is already there (in the 0.6.49 release). I was using it extensively late yesterday and earlier today.
itimpi is online now   Reply With Quote
Old 04-29-2010, 08:19 PM   #8
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by itimpi View Post
The merge function is already there (in the 0.6.49 release). I was using it extensively late yesterday and earlier today.
You're running it in the source, not the binary. Kovid develops directly into the source, then does a snapshot and upgrades the version number. Anyone running the source from a few days after 0.6.49 was released has it, but anyone running the 0.6.49 binary does not, since that snapshot was taken just before the Merge code was added. I assumed the OP wasn't running source in a development environment.

(I hope you like it. I put a lot of effort into that code ... even if Kovid did have to fix it up)
Starson17 is offline   Reply With Quote
Old 04-30-2010, 01:53 AM   #9
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,022
Karma: 777817
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
Quote:
Originally Posted by Starson17 View Post
You're running it in the source, not the binary. Kovid develops directly into the source, then does a snapshot and upgrades the version number. Anyone running the source from a few days after 0.6.49 was released has it, but anyone running the 0.6.49 binary does not, since that snapshot was taken just before the Merge code was added. I assumed the OP wasn't running source in a development environment.
OK - that will be it as I tend to download the source as well whenever I download a new release. . A shame that there is not a way of displaying in the GUI that one is running from source as a reminder. I have raised ticket #5417 to request this as it should be an easy enhancement. In fact I may look at the source myself later today to see if I can figure out the exact change reuqired - having raised the request I might as well see if I can also figure out the fix

Quote:
(I hope you like it. I put a lot of effort into that code ... even if Kovid did have to fix it up)
It is definitely a significant improvement - already saves me a lot of work as I can now simply import any book without regard to format and sort them into a single entry afterwards when required.

Last edited by itimpi; 04-30-2010 at 02:05 AM.
itimpi is online now   Reply With Quote
Old 04-30-2010, 07:59 AM   #10
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by itimpi View Post
A shame that there is not a way of displaying in the GUI that one is running from source as a reminder.

I have raised ticket #5417 to request this as it should be an easy enhancement. In fact I may look at the source myself later today to see if I can figure out the exact change reuqired - having raised the request I might as well see if I can also figure out the fix
I think that would be an excellent addition. I didn't even realize until a couple of weeks ago that the source differed from the binaries. Thanks for tackling it.

Quote:
It is definitely a significant improvement - already saves me a lot of work as I can now simply import any book without regard to format and sort them into a single entry afterwards when required.
Yes, that's why I wrote it (or as much of it as Kovid kept) I just wish I could code well enough that he didn't have to completely rewrite my efforts. I hesitate to ask for an improvement, thinking I should do it myself, then hesitate to do it, thinking he'll just have to rewrite it anyway.
Starson17 is offline   Reply With Quote
Old 04-30-2010, 09:32 AM   #11
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,371
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Quote:
Originally Posted by Starson17 View Post
I hesitate to ask for an improvement, thinking I should do it myself, then hesitate to do it, thinking he'll just have to rewrite it anyway.
Rewriting is always easier than writing from scratch, so don't hesitate
kovidgoyal is online now   Reply With Quote
Old 05-01-2010, 08:44 AM   #12
Jaelle
Junior Member
Jaelle began at the beginning.
 
Posts: 9
Karma: 10
Join Date: Mar 2010
Device: Kindle 2
I have a related question on this that I've been wondering about. From reading the thread above it appears that I probably caused the issue with first importing all my books into calibre, and then using the save to disk button, creating duplicates.


When I look at the calibre library with Windows Explorer, there are two folders for every author. There is a folder with the authors name, first name then last name. And there is a folder with the authors name, last name comma first name.

The folders that have the first name first have each book in subfolders with a number in parentheses. These folders contain the ebook file and a cover.

The folders that have the last name first contain the ebook file, jpeg for the cover, and the .opf file.

I don't want every book stored on my hard drive twice. So I have two questions.

1) Which set of folders should be deleted without doing any harm to my calibre library?

2) How do I avoid everything being duplicated in the future? Stop using Save to Disk? (I was under the impression I guess that you should do a save after importing into calibre.)

Thanks for any help!
Jaelle is offline   Reply With Quote
Old 05-01-2010, 09:37 AM   #13
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,022
Karma: 777817
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
The folders with a number in the name are those that belong in the Calibre library. Any without numbers have been created by some method other han importing into Calibre.

The Save to Disk facility is used for when you want to export a file from the Calibre library. No need to do it as a matter of course. Sounds as if you used it and then gave the same location for the Save to Disk as your Calibre library and thus the duplication.
itimpi is online now   Reply With Quote
Old 05-01-2010, 09:41 AM   #14
Jaelle
Junior Member
Jaelle began at the beginning.
 
Posts: 9
Karma: 10
Join Date: Mar 2010
Device: Kindle 2
Thanks! I thought that was the case, but I wanted to make sure before I start deleting the extra folders.
Jaelle is offline   Reply With Quote
Old 05-01-2010, 10:21 AM   #15
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 8,770
Karma: 12516053
Join Date: Feb 2009
Location: North Carolina
Device: Nexus 7
Quote:
Originally Posted by Jaelle View Post
Thanks! I thought that was the case, but I wanted to make sure before I start deleting the extra folders.
The save to disk feature is a great way to back up your books, just make sure to pick a different folder next time.
DoctorOhh is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Calibre Duplicates mitch13 Calibre 5 11-13-2010 06:42 AM
Possible Bug on Duplicates Giuseppe Chillem Calibre 3 05-06-2010 07:26 PM
Duplicates... jaxx6166 Sony Reader 5 07-09-2009 09:13 PM
duplicates in database RJA Calibre 3 06-22-2009 09:06 AM
Duplicates found! Moejoe Calibre 32 06-12-2009 12:20 AM


All times are GMT -4. The time now is 12:19 PM.


MobileRead.com is a privately owned, operated and funded community.