How to fix "Extra Titles" in Database in " Check Library - Problems Found

[chaley]

Originally Posted by millan

Update:
Moving the OPF file to the folder with the book and cover
or moving the book and the cover to the folder with opf

still gives this look



that means book and opf did not match
perhaps because of the slight difference in the book name
that also means opf has some sort of link to the book title
moving opf or other way around break that link I suppose

Sorry, but I don't know what you are telling me.

If there is an OPF in the folder along with a book file, and if you choose (as I suggested) the "One book for folder" option, then the metadata in the OPF wins. If the metadata is incorrect then it is incorrect. There is no magic way of fixing it.

The calibre user manual describes what this add book function does.

Example: I copied a folder out of my production library and put it on my desktop. It contains a (correct) metadata.opf, a cover, and 2 format files. I renamed the formats aa.epub and aa.mobi. I then opened my test library and did an Add books One book per directory, aiming it at that folder. The book was added to my library perfectly, with the correct cover and the correct metadata.

You must use the "One book per directory" option for this to work.

[chaley]

BTW: the image you posted violates forum rules because it is too big. You should resize it to be less than 600 pixels in any dimension. Even better would be to add it as an attachment instead of using an img link.

[millan]

I'm sorry it's my misunderstanding that I missed "One book for folder" part
I am not familiar with "One book for folder"
is there such setting if so how can I get to that setting
or I should remove the extra folder manually please

- sorry about the image size I'll remove them asap

Originally Posted by chaley

Sorry, but I don't know what you are telling me.

If there is an OPF in the folder along with a book file, and if you choose (as I suggested) the "One book for folder" option, then the metadata in the OPF wins. If the metadata is incorrect then it is incorrect. There is no magic way of fixing it.

The calibre user manual describes what this add book function does.

Example: I copied a folder out of my production library and put it on my desktop. It contains a (correct) metadata.opf, a cover, and 2 format files. I renamed the formats aa.epub and aa.mobi. I then opened my test library and did an Add books One book per directory, aiming it at that folder. The book was added to my library perfectly, with the correct cover and the correct metadata.

You must use the "One book per directory" option for this to work.

[chaley]

Originally Posted by millan

I'm sorry it's my misunderstanding that I missed "One book for folder" part
I am not familiar with "One book for folder"
is there such setting if so how can I get to that setting
or I should remove the extra folder manually please

It isn't a setting, but is instead an option when adding books. See the attached screen capture.

[millan]

All the books I have added so far was drag and drop onto calibre interface.
is it too late to change the previous ones now or can I do anything about it right now please

[chaley]

Originally Posted by millan

All the books I have added so far was drag and drop onto calibre interface.
is it too late to change the previous ones now or can I do anything about it right now please

You can delete and re-add the books unless you deleted the folders they came from. If you did delete the folders then you should be able to recreate them from your other copy.

[millan]

Originally Posted by chaley

You can delete and re-add the books unless you deleted the folders they came from. If you did delete the folders then you should be able to recreate them from your other copy.

I cannot thank you enough for all the guidance you have given me

Here's what I have to do I think:
==========================
1) select one entry at a time - and every entry to follow) in the calibre book interface
2)Click edit metadata
3) add a format to this book - locate the book and add it
4) add the cover - locate the cover and add it
it works
the I delete the second folder which contents already copied (from the step 3 and 4) to the folder where the opf is
==========================

I guess there's no faster way of doing it since I do not want to loose the metadata

is there ?

[chaley]

Originally Posted by millan

I cannot thank you enough for all the guidance you have given me

Here's what I have to do I think:
==========================
1) select one entry at a time - and every entry to follow) in the calibre book interface
2)Click edit metadata
3) add a format to this book - locate the book and add it
4) add the cover - locate the cover and add it
it works
the I delete the second folder which contents already copied (from the step 3 and 4) to the folder where the opf is
==========================

I guess there's no faster way of doing it since I do not want to loose the metadata

is there ?

That isn't the way I suggested you do it, but if it works for you then fine.

What I suggested is that you create a new empty library. You then combine the contents of the 3500 or so folders together so that the book file(s), the cover, and the metadata.opf for one book are in the same folder. In the newly-created library you use the "Add ... One book per directory" to add the book and its metadata in one step. If you make a folder containing these combined folders then you can add that "super-folder" (again, one book per directory) to add all the books in the subfolders in one step.

But to say again, at this point the best thing for you to do is what you know how to do and what works for you.

[millan]

Your steps are less time consuming
I'll follow that right away, Chaley

Originally Posted by chaley

That isn't the way I suggested you do it, but if it works for you then fine.

What I suggested is that you create a new empty library. You then combine the contents of the 3500 or so folders together so that the book file(s), the cover, and the metadata.opf for one book are in the same folder. In the newly-created library you use the "Add ... One book per directory" to add the book and its metadata in one step. If you make a folder containing these combined folders then you can add that "super-folder" (again, one book per directory) to add all the books in the subfolders in one step.

But to say again, at this point the best thing for you to do is what you know how to do and what works for you.

[millan]

Just reporting an update:

I'm at the stage one
combining book, opf and cover into single folder
-I did 166 books=2.96gb (out of 3505 books=48gb) and
added to the newly-created library.
seems ok, no error,
enough for the day

May need weeks to complete the whole thing I suppose

all the best

[millan]

I cannot thank you enough chaley:

I have finally completed this mammoth task.
Any suggestions/cautions I don't have to do this again in the process I do as follows:

I have around ( 3480 books/articles/thesis) (about 45gb)
and add several more every week

I have one original and one backup copy:
Lib1 original (at pc)
Lib2 backup (external usb disk) (identical copy of Lib1)

This was I was doing it before I mucked it up:
1)adding new ones to Lib1 (via drag and drop from a folder)
2) Copy to Lib2 (via Calibre interface)

In order to fix the corrupted database,
1)I have added all the items (add books -one book per directory)
2) then I copy the whole lot to external usb disk
so no problem so far.

From now on:
Should I
a)drag and drop the new ones from a content folder and include them in Lib1, then Copy to Lib2 (via Calibre interface)

or should I
b) add books - one book per directory
and then Copy to Lib2 (via Calibre interface)

if I pick option b) , it adds only one item rather than the whole folder I have noticed, so I must be doing something wrong I assume

I appreciate your any further suggestions and cautionary tips please

all the best

[eschwartz]

1-book-per-directory is meant to only import one book per folder. No wonder it adds only one book.
If you have multiple books per folder, don't use that option.
The primary use of that option is for books that have been in a calibre library before, and are already grouped with OPF/cover/multiple formats.

For new books yo can just drag-and-drop -- unless they are currently organized in folders already.

Do note: You can select a folder full of folders, for the one-book-per-directory option.




For backup, I would suggest using something like FreeFileSync/Synctoy to sync the library folder to your backup drive.

[chaley]

Originally Posted by millan

I cannot thank you enough chaley:

I have finally completed this mammoth task.
Any suggestions/cautions I don't have to do this again in the process I do as follows:

I have around ( 3480 books/articles/thesis) (about 45gb)
and add several more every week

I have one original and one backup copy:
Lib1 original (at pc)
Lib2 backup (external usb disk) (identical copy of Lib1)

This was I was doing it before I mucked it up:
1)adding new ones to Lib1 (via drag and drop from a folder)
2) Copy to Lib2 (via Calibre interface)

There is nothing obvious in that process that would have caused the problems.

The usual way that problems like yours arise is having calibre on a network drive. They can also happen if the file system on a local drive isn't "native". Backup-like programs can also cause problems, in particular cloud backups, because they can undo calibre's renames. You should look at your system to verify that the file systems on the disks are normal for your operating system, and also look at what programs you have running in the background.

In order to fix the corrupted database,
1)I have added all the items (add books -one book per directory)
2) then I copy the whole lot to external usb disk
so no problem so far.

From now on:
Should I
a)drag and drop the new ones from a content folder and include them in Lib1, then Copy to Lib2 (via Calibre interface)

or should I
b) add books - one book per directory
and then Copy to Lib2 (via Calibre interface)

if I pick option b) , it adds only one item rather than the whole folder I have noticed, so I must be doing something wrong I assume

I appreciate your any further suggestions and cautionary tips please

all the best

There are two questions here: 1) how to add books to calibre, and 2) how to backup your calibre library.

As for 1), add the books in any of the ways supported by calibre. None of them are more dangerous than the others. And eschwartz's comments about one book per folder are spot on.

Regarding 2) again eschwartz is right. If what you want is a backup then use software that makes backups. There are many available. As it is a backup you probably want to use a "mirror deletes" copy mode so that the backup is an accurate copy of the library, not the union of changes over time with folders left behind.

I go a step further and use a backup program (Acronis) that supports incremental updates and history browsing so that I can restore something from (say) a week ago, but these programs typically cost money and can be 'interesting' to set up. I also am backing up much more than my library, storing the backups on my local multi-terabyte Synology NAS. Out of paranoia I also backup (almost) everything to a cloud backup service (Livedrive) so that I will have something available if disaster strikes at my apartment, destroying all equipment and copies. And finally, a few important things are also kept on Dropbox. And yes, over the years I have had to recover information from all three of these backup locations, once because the NAS-of-the-moment failed just when I needed it and once because I had misconfigured the cloud backup. Bottom line: think carefully about what you are trying to achieve with a backup strategy, set it up, then test it.

[millan]

I appreciate your kind help and thank you eschwartz,

I'm kind of relieved that I could continue to drag and drop the new items
I have also installed the FreeFileSync now.
It's a great suggestion.

On the other hand, I have a question on copying through Calibre:
After I copy the latest entries in calibre and copy through calibre interface,
I seem to get "cleaner" Lib2
ie: Less number of folders (Perhaps all the empty folders are left in Lib1
So the total byte size of the Lib1 and Lib2 are exactly the same after copying although I could see less number of folders etc in Lib2

does that mean copying items from Lib1 to Lib2 gives a cleaner library all
together if done through calibre interface?

Originally Posted by eschwartz

1-book-per-directory is meant to only import one book per folder. No wonder it adds only one book.
If you have multiple books per folder, don't use that option.
The primary use of that option is for books that have been in a calibre library before, and are already grouped with OPF/cover/multiple formats.

For new books yo can just drag-and-drop -- unless they are currently organized in folders already.

Do note: You can select a folder full of folders, for the one-book-per-directory option.

For backup, I would suggest using something like FreeFileSync/Synctoy to sync the library folder to your backup drive.

[millan]

I appreciate your kind help I couldn't have solved it just by myself. Thank you again chaley,

I'm happy to hear that the ones I mentioned woudl not have caused the problem

Thank you also for the cautionary tips I definitely need them.

Your cloud copy is also another ideal and further step although I am hesitant to do it because of the costs involved for my close to 50gb and growing library,
at the same time I could argue with myself that the loss time I have had recently was much more costly.

Originally Posted by chaley

There is nothing obvious in that process that would have caused the problems.

The usual way that problems like yours arise is having calibre on a network drive. They can also happen if the file system on a local drive isn't "native". Backup-like programs can also cause problems, in particular cloud backups, because they can undo calibre's renames. You should look at your system to verify that the file systems on the disks are normal for your operating system, and also look at what programs you have running in the background.
There are two questions here: 1) how to add books to calibre, and 2) how to backup your calibre library.

As for 1), add the books in any of the ways supported by calibre. None of them are more dangerous than the others. And eschwartz's comments about one book per folder are spot on.

Regarding 2) again eschwartz is right. If what you want is a backup then use software that makes backups. There are many available. As it is a backup you probably want to use a "mirror deletes" copy mode so that the backup is an accurate copy of the library, not the union of changes over time with folders left behind.

I go a step further and use a backup program (Acronis) that supports incremental updates and history browsing so that I can restore something from (say) a week ago, but these programs typically cost money and can be 'interesting' to set up. I also am backing up much more than my library, storing the backups on my local multi-terabyte Synology NAS. Out of paranoia I also backup (almost) everything to a cloud backup service (Livedrive) so that I will have something available if disaster strikes at my apartment, destroying all equipment and copies. And finally, a few important things are also kept on Dropbox. And yes, over the years I have had to recover information from all three of these backup locations, once because the NAS-of-the-moment failed just when I needed it and once because I had misconfigured the cloud backup. Bottom line: think carefully about what you are trying to achieve with a backup strategy, set it up, then test it.