Books in Calibre folder don't turn up in search

[montalex]

I recently discovered that many of the books in my Calibre folder will not show up in the Calibre search window. There are hundreds of them! I've tried 'restore database", but this doesn't help. Also, when I select all books, delete books of a certain format, there are hundreds of unidentified formats!

Help!

[alanHd]

Have you ran check library under Library maintenance? Let it check your library files against the database.

[montalex]

Yes, I've run check library. It finds no errors. Along with books that won't show up in search, are many books that are actually missing. I search for a book, and it turns up, but there is nothing in the book folder but metadata -- no book! What I mess! I have lost many dozens of books, and other are there, but not findable via Calibre. And all books were added through the Calibre interface. There are hundreds of errors listed in restore database.

[alanHd]

Im no Calibre guru, but one way of getting out of this situation occurs to me. You could move your Calibre library to another location and then re-add anything left behind.

Im sure someone with more knowledge will be along fairly shortly.

[montalex]

I'm wondering if using my Calibre book folder as a Dumptruck folder is the problem. I never actually use the Dumptruck online service, I simply backup my books there. No one else has access to it. Weird and frustrating. Hundreds of books gone missing. There are folders with the book title that contain a metadata file but no book. And then there are folders which do contain books, but I cannot access them through Calibre. I have to use Explorer to find them (we are talking over 2,000 books). Without going through everything book by book, it is hard to even know what is missing.

[Adoby]

Do what alanHD suggested. Move the books over to a new library, using the GUI. Then you can see what is left behind.

It is possible to corrupt the calibre library by running backup tools while calibre is working, or by restoring parts of the calibre library. Turn off any cloud, sync or other backup tools working on the library, while you use calibre, to avoid problem.

[montalex]

That means going through thousands of files and adding books one at a time! I see in the error log from "restore database" that there are 148 folders that were "ignored". There are the books that I can't find through the Calibre interface, but which are actually in their folders. Then, there are the missing books.

What a mess! Maybe I'll just use Calibre for conversion, and store my books elsewhere.

[theducks]

Originally Posted by montalex

I'm wondering if using my Calibre book folder as a Dumptruck folder is the problem. I never actually use the Dumptruck online service, I simply backup my books there. No one else has access to it. Weird and frustrating. Hundreds of books gone missing. There are folders with the book title that contain a metadata file but no book. And then there are folders which do contain books, but I cannot access them through Calibre. I have to use Explorer to find them (we are talking over 2,000 books). Without going through everything book by book, it is hard to even know what is missing.

Synching on a Active Calibre Library folder is asking for issues.

(Backup) Synch AFTER Calibe shuts down. Things happen in Calibre in transactions, not all at once. Folder get created/deleted, file added/deleted, DB records get updated.

[montalex]

@theducks. That makes sense. I'm sure it is something I've done and not some flaw in Calibre. I've loved Calibre, and donated.

[PeterT]

If you do do syncs, if possible make them one way... PC to cloud for instance.

[DoctorOhh]

Originally Posted by montalex

That means going through thousands of files and adding books one at a time!

No that doesn't mean you have to add books back 1 at a time. Once the library is moved via the GUI you can simply use the Add books from directories, including sub-directories option under add books. Point it to the main folder that was left behind and let it add those books back.

Originally Posted by montalex

What a mess! Maybe I'll just use Calibre for conversion, and store my books elsewhere.

Maybe you should stop allowing DumpTruck to access calibre while you are actively working on it. Some active sync software plays nicer than others but corruption like you've experienced is the possible result if you allow another program access to calibre while calibre is active.

[montalex]

Originally Posted by DoctorOhh

No that doesn't mean you have to add books back 1 at a time. Once the library is moved via the GUI you can simply use the Add books from directories, including sub-directories option under add books. Point it to the main folder that was left behind and let it add those books back.

That fixed the problem of books not found via Calibre, but caused another headache. My books had been painstakingly sorted by author last name. Now the sort is random. I guess it doesn't matter. I can always search for what I want.

Thanks for the help.

[montalex]

O.K., I started over again, with a virgin Calibre folder, and imported books from a backup. It worked perfectly! Thanks so much DrOhh.

And a big thanks to Kovid!

I wrote too soon. Still a big mess. It is funny, I can go to an old backup I made, and all folders are in perfect alphabetical order by author's last name. The latest Calibre folder is chaos, most authors listed by first name. Unfortunately, the older backup, on Dropbox, is missing, many, many books that I added since moving to Dumptruck to get more space.

I think the lesson is to back up locally.

[eschwartz]

There are two options to add books from subdirectories. The one that assumes "one folder per book, each file is the same book in a different format" will try to find a metadata.opf file (calibre creates these) to get the metadata from. This would preserve your old metadata.

[montalex]

Originally Posted by eschwartz

There are two options to add books from subdirectories. The one that assumes "one folder per book, each file is the same book in a different format" will try to find a metadata.opf file (calibre creates these) to get the metadata from. This would preserve your old metadata.

That's the selection I used. Interestingly, although my author sort is all messed-up still, I have about 250 more titles listed in my new Calibre book folder than I had in the old one. So, at least this method found the missing books. I guess how the author is listed makes little difference. If I search for an author, I get all iterations of his name anyway. I can always make the repair when I transfer a book to my Kindle.