duplicates on hard drive

LarryRup · 10-24-2016, 01:04 PM

Hi: Can use help. My hard drive has many, many duplicate directories, and mostly, but not always, duplicate book formats taking up much valuable space. Here is an example: calibre library->author Good up to here. Then, Under the author we have three directories for the same book title (spelled identically) however each directory/book title is followed by a number in parenthesis for example:

book title (1)
book title (4)
book title (8)

I assume this number has reference to adding the book multiple times, however I do use the duplicate plug in then delete dups using delete within Calibre. Looking in each of the three directories, the first only has a file named metadata.opf, the second and third directories have four files: a cover, the same metadata file, and a book file in a epub and mobi format.

The duplicate book title directories are present for the vast majority of author directories with titles duplicated two, sometimes three times.

I do not edit the calibre library directory in explorer, and no longer run Calibre on different machines using the same Calibre library which is on a windows shared drive, just not being used by calibre simultaneously.

When this happened previously, the only solution I could find was closing Calibre, renaming the library directory and using add books to a new calibre library.

As info, I have in configuration the option to check for duplicates.

What could be causing this and is there a better way to fix other than creating the library from scratch? Very much appreciate help!

Regards,

Larry

theducks · 10-24-2016, 01:31 PM

Look at the Find Duplicates Plugin. That helps find duplicates.

Then you can resolve what to do by viewing, Merging (formats)or deleting

pootytang · 10-24-2016, 02:10 PM

Thanks for the response. Not sure I understand. I took a look at the duplicates plug in, but it seems to only deal with duplicates in the library i.e. the books that show in the Calibre display. Calibre database is clean... no duplicates. The duplicates are only on the hard drive. This directory was created by Calibre and is maintained by it. Since the duplicates do not show in Calibre (and the dup plug in confirms there are no dups in the calibre database) I don't think and cannot figure out how to use it to remove all the duplicates on the hard drive.

I'm likely missing something or mis-understanding.

Regards,

Larry

JSWolf · 10-24-2016, 02:12 PM

Are you looking for duplicates not just in your Calibre library? If so, you are in the wrong forum.

chaley · 10-24-2016, 02:28 PM

Try running calibre's database check. Click the "Library" button, choose "Library Maintenance", then "Check library", then "Yes" when calibre asks if you want it to check if the files in your library match the database. Calibre will then show you all "books" in your library that it does not have in its database.

If there are lots of "extra books" then you need to look for why. The most common reasons are:

Having your library on a network drive (you seem to be doing this).
Having your library in a folder synced to the cloud.
Having some sync/backup program running.
Aggressive antivirus.

MerlinMama · 10-25-2016, 05:36 AM

Quote:

Originally Posted by LarryRup

When this happened previously, the only solution I could find was closing Calibre, renaming the library directory and using add books to a new calibre library.

I want to make sure I get the problem correctly. Here is what I understand:

You had duplicate books in a library, but rather than remove one copy that you didn't need, you created a new library and added all your books to that. What did you do with the first library? I might be wrong but as far as I know, adding books won't delete the 'original' copy, so unless you delete it, it stays there.

itimpi · 10-25-2016, 05:54 AM

Quote:

Originally Posted by LarryRup

When this happened previously, the only solution I could find was closing Calibre, renaming the library directory and using add books to a new calibre library.

Probably a better option is to use the Calibre option to move all the books in the current Calibre library to a new library. When that completes, all the folders/files that are still present in the old Calibre library are ones that were not referenced by the Calibre database. You now have to decide what (if anything) you want to do with these.

pootytang · 10-28-2016, 06:54 PM

What I finally did was use the Calibre library check and deleted all the extra titles. What i found was all of the duplicates appeared to NOT be in the Calibre database. While I understand this effectively deleted the files (books), it did clean up the hard drive. I'll assume I did something funky, which I think was restoring some books from a cloud based backup I have of my books. I believe that is where all the duplicates came from on my hard drive. Sorry for taking some of your time If I was the cause ( I did not realize this!).

If a network share is not a good thing, and a cloud based backup is also not good....how are you guys handling backup and convenience?

Thanks

Pooty

theducks · 10-28-2016, 08:31 PM

Quote:

Originally Posted by pootytang

What I finally did was use the Calibre library check and deleted all the extra titles. What i found was all of the duplicates appeared to NOT be in the Calibre database. While I understand this effectively deleted the files (books), it did clean up the hard drive. I'll assume I did something funky, which I think was restoring some books from a cloud based backup I have of my books. I believe that is where all the duplicates came from on my hard drive. Sorry for taking some of your time If I was the cause ( I did not realize this!).

If a network share is not a good thing, and a cloud based backup is also not good....how are you guys handling backup and convenience?

Thanks

Pooty

Backing up on anything other than Google drive is fine. GD plays with the file name in a way that messes with the linkage to the DB.

Any 2 way Sync needs to be suspended (calibre Library folder) while Calibre is running

10-24-2016, 01:04 PM	#1
LarryRup Junior Member Posts: 1 Karma: 10 Join Date: Oct 2016 Device: Kindle	duplicates on hard drive Hi: Can use help. My hard drive has many, many duplicate directories, and mostly, but not always, duplicate book formats taking up much valuable space. Here is an example: calibre library->author Good up to here. Then, Under the author we have three directories for the same book title (spelled identically) however each directory/book title is followed by a number in parenthesis for example: book title (1) book title (4) book title (8) I assume this number has reference to adding the book multiple times, however I do use the duplicate plug in then delete dups using delete within Calibre. Looking in each of the three directories, the first only has a file named metadata.opf, the second and third directories have four files: a cover, the same metadata file, and a book file in a epub and mobi format. The duplicate book title directories are present for the vast majority of author directories with titles duplicated two, sometimes three times. I do not edit the calibre library directory in explorer, and no longer run Calibre on different machines using the same Calibre library which is on a windows shared drive, just not being used by calibre simultaneously. When this happened previously, the only solution I could find was closing Calibre, renaming the library directory and using add books to a new calibre library. As info, I have in configuration the option to check for duplicates. What could be causing this and is there a better way to fix other than creating the library from scratch? Very much appreciate help! Regards, Larry

10-24-2016, 02:10 PM	#3
pootytang Member Posts: 11 Karma: 10 Join Date: Jun 2012 Device: Kindle	Duplicates on hard drive Thanks for the response. Not sure I understand. I took a look at the duplicates plug in, but it seems to only deal with duplicates in the library i.e. the books that show in the Calibre display. Calibre database is clean... no duplicates. The duplicates are only on the hard drive. This directory was created by Calibre and is maintained by it. Since the duplicates do not show in Calibre (and the dup plug in confirms there are no dups in the calibre database) I don't think and cannot figure out how to use it to remove all the duplicates on the hard drive. I'm likely missing something or mis-understanding. Regards, Larry

10-24-2016, 02:28 PM	#5
chaley Grand Sorcerer Posts: 12,447 Karma: 8012886 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	Try running calibre's database check. Click the "Library" button, choose "Library Maintenance", then "Check library", then "Yes" when calibre asks if you want it to check if the files in your library match the database. Calibre will then show you all "books" in your library that it does not have in its database. If there are lots of "extra books" then you need to look for why. The most common reasons are: Having your library on a network drive (you seem to be doing this). Having your library in a folder synced to the cloud. Having some sync/backup program running. Aggressive antivirus.

10-28-2016, 06:54 PM	#8
pootytang Member Posts: 11 Karma: 10 Join Date: Jun 2012 Device: Kindle	Much Thanks What I finally did was use the Calibre library check and deleted all the extra titles. What i found was all of the duplicates appeared to NOT be in the Calibre database. While I understand this effectively deleted the files (books), it did clean up the hard drive. I'll assume I did something funky, which I think was restoring some books from a cloud based backup I have of my books. I believe that is where all the duplicates came from on my hard drive. Sorry for taking some of your time If I was the cause ( I did not realize this!). If a network share is not a good thing, and a cloud based backup is also not good....how are you guys handling backup and convenience? Thanks Pooty

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
KoboExtStorage on Hard drive	Nascarfan1972	Kobo Reader	9	06-17-2013 08:47 PM
hard drive and Calibre???	sleepster69	Calibre	1	10-22-2012 10:42 PM
New Hard Drive and Calibre	taxesed	Library Management	6	03-24-2012 12:39 AM
External Hard Drive and PE	bunodosoma	enTourage eDGe	6	09-28-2011 07:19 AM
Easy hard drive data archiving with a USB hard drive adapter	Bob Russell	Lounge	24	02-20-2007 04:15 PM

10-24-2016, 01:31 PM	#2
theducks Well trained by Cats Posts: 31,057 Karma: 60358908 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	Look at the Find Duplicates Plugin. That helps find duplicates. Then you can resolve what to do by viewing, Merging (formats)or deleting

10-24-2016, 02:12 PM	#4
JSWolf Resident Curmudgeon Posts: 79,756 Karma: 145864619 Join Date: Nov 2006 Location: Roslindale, Massachusetts Device: Kobo Libra 2, Kobo Aura H2O, PRS-650, PRS-T1, nook STR, PW3	Are you looking for duplicates not just in your Calibre library? If so, you are in the wrong forum.

Advert

Advert