Books dissapear from Library

[Corwyn]

Hi everyone,

I've been using Calibre for a while now and recently i've been noticing that some of the books that I add to the library will be gone the next time I load Calibre.
If I check the files themselves lo and behold there is the missing book. But to view it in Calibre I have to re-add the book.
When I go back to the files, Calibre will have added a second entry for the same book in the directory. (just the numbers at the end of the folder will be different.)

Why does this happen and how can I fix it?

If this question has been asked and answered srry for the extra thread but I couldn't find anything similar.

[theducks]

Got Cloud (like Dropbox) running on the Library?

[Corwyn]

No the books are on a fixed hd in my computer. I ran a restoration of the library database but that did nothing to restore the books.
They are "physically" in the library's files location but somehow it doesn't seem able to read them.

[eschwartz]

Originally Posted by Corwyn

No the books are on a fixed hd in my computer. I ran a restoration of the library database but that did nothing to restore the books.
They are "physically" in the library's files location but somehow it doesn't seem able to read them.

Well, yes, but are you using any syncing solutions, for example Dropbox, syncing the library folder to another location e.g. Cloud?

If so, you may have edit clashes. It is advisable to only sync when calibre is not running.

If not, perhaps something is restricting the view? Is it possible you are accidentally using a virtual library? That would make sense, if the book is physically present in the library folders and Check Library doesn't flag it, but you cannot see it in the GUI.

[BetterRed]

- such as local file syncing with Apple Time Machine, FFS RealTime, GoodSynch OnChange etc.

BR

[theducks]

A Library folder structure looks like:

Library_name/Author_name/title_name(ID#)/formats,metadata.opf,cover.jpg

If you don't see a title(###) folder, things is really busted

Open a metadata.opf with Notepad, that ws the backup of your metadata for that title. If that is trashed, empty (btw that would also say USER action, not the update), which would make the recover operation fail. You need to import the books into a NEW-EMPTY library

[Corwyn]

No not syncing with anything. And I don't think I'm using a virtual library.

The library folder structure look exactly like you describe it and how it is supposed to be.
At the moment I don't have an example of a book that is missing (that I know of) When I find one I will update this post with what I find in the metadata file.

Metadata.opf file seems to fine for the missing books.

[Corwyn]

Another thing I just noticed as I was looking for missing books to investigate the metadata file was a entry in Calibre with parts of the metadata form one book and parts form an other book. the author and title were wrong, they didn't fit with the cover art, and in the author sort field in the edit metadata window in calibre was the right author.

The book from witch it copied the metadata was also in the library unfortunatly the book file itself disappeared the folder of that book only had the cover and the metadata file. Why would calibre delete the book but not the metadata or the folder ( not that I had given that order) I rarely delete any books.

[itimpi]

You should run the Library Maintenance option to check that the Calibre metadata database is consistent with the files stored in the Calibre library, and correct any errors/issues that are identified. That way you will know that they are currently in step with each other.

[Corwyn]

I'm doing that right now I had run the restore function in the Library Maintenance option.

Wouw,

That found a ton of books and authors it had as files but not in the library. Going to tel calibre to fix that.

Any idea why it lost those books?

Ok The Library check -- problems found screen appears
It has a long list of books that seem not to be added to the library but for with it did find files in the directory.
At the bottom it has a few buttons one delete marked and one of them is Fix marked.
Only the delete option works the fix one stays grayed out. As I don't want to throw these books away I'm not going to delete them anyone know why the fix option is grayed out?

[chaley]

Originally Posted by Corwyn

I'm doing that right now I had run the restore function in the Library Maintenance option.

Wouw,

That found a ton of books and authors it had as files but not in the library. Going to tel calibre to fix that.

Any idea why it lost those books?

Something is scribbling over your metadata.db file. I have no idea what, except that it isn't calibre. What plugins do you use/have installed? Do you ever use the command line? Do you ever do anything to the database outside of calibre such as run programs to analyse/fix up the database?

Have you verified that the disc holding the calibre library has a healthy file system?

Ok The Library check -- problems found screen appears
It has a long list of books that seem not to be added to the library but for with it did find files in the directory.
At the bottom it has a few buttons one delete marked and one of them is Fix marked.
Only the delete option works the fix one stays grayed out. As I don't want to throw these books away I'm not going to delete them anyone know why the fix option is grayed out?

That function will never add books back to the database, which is why "Fix" is greyed out. The books must be manually added. The best way to do this is to move the "title" directory (the one containing cover.jpg, metadata.opf, and the format files) out of calibre's library to some other directory then tell calibre to add books from that directory using the "recursive, one book per directory" option.

[Corwyn]

As far as I know the drive the library is on is good has been for some time.
I don't use the command line or touch the database outside of calibre and I don't have any analitical/fixing programs running.

list of plugins
Doc input (1.0.1)
kobotouchextended (2.2.20)

barns and nobels
beam
goodreads

kobo utilities
manage series
resize covers
search the internet

thanks for the explenation I had found it after reading all the text so I'm following your advice there.

[theducks]

Originally Posted by Corwyn

As far as I know the drive the library is on is good has been for some time.
I don't use the command line or touch the database outside of calibre and I don't have any analitical/fixing programs running.

list of plugins
Doc input (1.0.1)
kobotouchextended (2.2.20)

barns and nobels
beam
goodreads

kobo utilities
manage series
resize covers
search the internet

thanks for the explenation I had found it after reading all the text so I'm following your advice there.

None of those PI should trash metadata.db (without a LOT of help from you. eg Pulling the power during an DB write)

1) your HD is failing?
2) your PSU is putting out trash on the rails/failing (bad fan)
3) Malware
4) An anti Virus that misbehaves (acts like Malware)
5) You installed a (IMHO snake oil) System speed up program that caches so much it gets trashed

As chaley indicated. something is stepping on metadata.db, big time.

[Corwyn]

it's a brand new system with almost nothing installed
I'll move the database to another disk and see if that helps.

[theducks]

Originally Posted by Corwyn

it's a brand new system with almost nothing installed
I'll move the database to another disk and see if that helps.

Ever hear the terms: 'Bathtub curve' and 'Infant Mortality'?

New stuff can 'pass' QA and ship.
Soon it fails. That is why most electronics have warranties of 60-90 days.
If it gets to the bottom of the 'Tub', it runs a long time, till it finally dies of old age (assumes you keep it in service).

We used to use 'Burn in shelves' in the 70's,. Where stuff ran 24/7 for a week or longer before the 'Final' QA. Manufacturing volumes are lots higher now, and stuff does fail less, so it is cheaper to ship and deal with the tiny fallout.

There are still 'Lemons'