Calibre library doesn't show all books after PC formatting and Calibre reinstallation

[lovedj1]

Hello everyone. I hope you're all doing great :-)

Yesterday, after performing a backup to an external hard drive, I formatted my Laptop and installed a clean OS (Windows 11). When I reinstalled Calibre and copied the old folder with all the books to C:\Biblioteca Calibre 2024 and told Calibre to use a preexisting library at that folder, it only imported some of the books. The original library had over 3000 books and the new one only had over 2400. For some reason, it was only importing books added to Calibre before July 2024 (or so I think, as that is the closest date on the library when I organise it by date). Everything added since then wasn't showing.

After reading through another thread here, I performed a library restore and now I have even less books in Calibre (1912). But after the restore, it did import some (or maybe all, no way of knowing) of the new ones (added after July 2024).

Also, I performed a library check and I do have those black triangles in some of the entries (image attached).

I have checked some of the books that I remember to be missing and they are in the Calibre library folder if I use Windows Explorer to look for them, just not showing on Calibre.

It's the first time in many years this has happened after an OS restore. I have followed the same procedure I always have (backup Calibre library folder to external hard drive, format PC, install all relevant software, and Windows updates, Install Calibre and tell it where the library is).

I am NOT synching the library with any cloud software either, I learned that lesson a year ago (the hard way).

Can any of you help with this, please? I do have a clean backup copy of the library in the external HD (luckily I didn't delete it after copying it to the PC).

[lovedj1]

It's been 3 days since i posted this and 209 views, but nobody has offered any help. I would very much appreciate it if someone could help, please. I'm desperate. Only solution I can think of is to copy each book one at a time and that would take forever.

[Sirtel]

Originally Posted by lovedj1

It's been 3 days since i posted this and 209 views, but nobody has offered any help. I would very much appreciate it if someone could help, please. I'm desperate. Only solution I can think of is to copy each book one at a time and that would take forever.

There's not really any 1-click method to fix a messed-up library. You could try to copy all the books displaying in the library view to another, empty library and then delete them from the old library. What's left in the old library folder will be the books Calibre is not aware of, and those must be added back manually.

Note that unless you find out what caused the mess, it could happen again. Have you added or deleted books directly to/from the library folder, without going through the calibre GUI?

[kovidgoyal]

I recommend creating a new empty library, then right click the add books button, choose add from folder and subfolders->click yes for assume all files ina single folder are the same book and finally choose the old library folder.

That should re-create your library with all book metadata and files intact.

[lovedj1]

Thank you @sirtel and @Kovidgoyal. I will try what Kovid suggested first and I will report back.

[lovedj1]

Originally Posted by kovidgoyal

I recommend creating a new empty library, then right click the add books button, choose add from folder and subfolders->click yes for assume all files ina single folder are the same book and finally choose the old library folder.

That should re-create your library with all book metadata and files intact.

OK, so, several things.

1) You saved my library. I now have 3175 books (which should be either all or most of them). So thanks a lot.

2) For some reason, while importing the library, it detected 84 books already in Calibre (which it shouldn't have, as the library was clean and these books were added into an empty library). SO, I don't know what happened there. No worries, though, I will clean it again if necessary. Just wanted to let you know.

For example, it detected different formats (one of them being OPF), and some of the books already in Calibre had a format name [none], which is weird.

Again, no worries, I will clean the library.

Aside from this, only other caveat is that the library now is in a different order and most books appear as added today.

But I got my whole library back, and that's the best.

Thanks a lot, mate

[lovedj1]

Oh, just another FYI. For some reason, even though I did get my library back, the metadata in many (most) of the books, has disappeared (covers and descriptions).

[JSWolf]

Originally Posted by lovedj1

Oh, just another FYI. For some reason, even though I did get my library back, the metadata in many (most) of the books, has disappeared (covers and descriptions).

If you still have the library that's got the problem, click on the library icon on the toolbar and select Library maintenance and then Restore library. That will go through the entire library and pull out the data from the OPF files and restore the database. If that works well enough, you will have your metadata and covers back.

[lovedj1]

Originally Posted by JSWolf

If you still have the library that's got the problem, click on the library icon on the toolbar and select Library maintenance and then Restore library. That will go through the entire library and pull out the data from the OPF files and restore the database. If that works well enough, you will have your metadata and covers back.

Thanks, JSWolf. I'm a bit afraid of doing that, as the last time I did it, it erased several hundred books.

[JSWolf]

Originally Posted by lovedj1

Thanks, JSWolf. I'm a bit afraid of doing that, as the last time I did it, it erased several hundred books.

But that won;t be an issue as you already have your eBooks in another library. Also, you can backup your original library just to be safe.

[lovedj1]

Originally Posted by JSWolf

But that won;t be an issue as you already have your eBooks in another library. Also, you can backup your original library just to be safe.

Well, I perfomed the restore, it was successful, but nothing changed as far as I can see. There are still a lot of books without covers and without summaries. But at least no books were lost. It's OK, I'll rebuild it slowly, 10 or 20 books at a time.

[theducks]

Originally Posted by lovedj1

Well, I perfomed the restore, it was successful, but nothing changed as far as I can see. There are still a lot of books without covers and without summaries. But at least no books were lost. It's OK, I'll rebuild it slowly, 10 or 20 books at a time.

Glad that was (mostly) successful.

Restore uses 2 additional files in the book folder: metadata.opf and cover.jpg.
These take priority (they have the changes YOU added to the Grid view) over what is found in the book files. The book may have some of these, but thase may be stale. (Calibre only automatically embeds the current data into books sent to devices)

Some of your losses, may be due to previous fails of your recovery having time to update (that is a background job) some of the the older recovery files.
(books that use the same tag or other common data may get updated when a change to that is made. Status is can be viewed on the Library icon menu)

[lovedj1]

Originally Posted by theducks

Glad that was (mostly) successful.

Restore uses 2 additional files in the book folder: metadata.opf and cover.jpg.
These take priority (they have the changes YOU added to the Grid view) over what is found in the book files. The book may have some of these, but thase may be stale. (Calibre only automatically embeds the current data into books sent to devices)

Some of your losses, may be due to previous fails of your recovery having time to update (that is a background job) some of the the older recovery files.
(books that use the same tag or other common data may get updated when a change to that is made. Status is can be viewed on the Library icon menu)

Thanks for the info, theducks. I will slowly rebuild it to how it was before (with all info, descriptions, titles and authors and covers) and then I will read the manual again to see how best to keep it all updated properly for future backup and restores.

[PeterT]

I'd suggest getting into the backup mindset before completing the updating of info phase.

Look at a simple solution for backup; something using a tool like FreeFileSync to do a one way copy from your PC to an external storage device (USB memory stick) of the calibre preferences folder along with the actual library folder.

While it's possible to use the calibre import / export command, we've seen too many issues with people havibng problems restoring from that form of backup.

Recovery from FreeFileSync to your PC is just a matter of either manually copying the data back to the correct locations, or using FreeFileSync to do a sync DROM the USB stick to the PC.

[Sirtel]

It's even better to back up to more than one external device, as external drives are known to die (are you sure that your external drive was not the cause of your corrupted back-up?)

Personally I keep the copies of my library and configuration folders on three external drives and in Dropbox. I've used my backups several times without issues.