Books deleted by Calibre (without my intervention !)

[alouest]

Hello,

First, sorry for my bad english.

Calibre is awesome and I am using it since a while.

But I discovered the lost of a large part of my books !
I'm sur I didn't erase them by myself.

Several times, I have noticed one book is missing : it was shown in the liste but the epub was not here anymore (no format) !
Possibly after modifying metadatas but I 'm not sur.
As the collection was stored on a NAS, I was thinking that it was a network problem. So I put my collection on my pc's hard drive and continued the harmonization (authors, titles, series).
Yesterday I saw that a lot of my epub have disappeared : there remain only the directories with the covers and the metadata.opf files.

Can anyone tell why, and how avoid that it happen again ?

I'm using Calibre with Linux.



Al.

Oops!
I post in the wrong section.
Please move this topic (sorry)

[theducks]

Originally Posted by alouest

Hello,

First, sorry for my bad english.

Calibre is awesome and I am using it since a while.

But I discovered the lost of a large part of my books !
I'm sur I didn't erase them by myself.

Several times, I have noticed one book is missing : it was shown in the liste but the epub was not here anymore (no format) !
Possibly after modifying metadatas but I 'm not sur.
As the collection was stored on a NAS, I was thinking that it was a network problem. So I put my collection on my pc's hard drive and continued the harmonization (authors, titles, series).
Yesterday I saw that a lot of my epub have disappeared : there remain only the directories with the covers and the metadata.opf files.

Can anyone tell why, and how avoid that it happen again ?

I'm using Calibre with Linux.



Al.

Oops!
I post in the wrong section.
Please move this topic (sorry)

I think the 'Library Management' section is OK

Books being deleted?
Did any have the tag 'News'?
If you insist on using that tag:
Preferences Behavior: remove the tick from: 'Delete news from Library...'

If the 'book' is really News (from the Schedule News download icon), check that the Advanced (Tab) setting (per News Source)is : Keep All issues

[alouest]

Yes, books are deleted from my pc, not my reader.
It remain only their directories and opf files (sometimes covers files).
I check hard drive and trash can but I can't find de lost epubs.

I don't use the tag "news".
I don't download news from www.
I only use epubs.

'Delete news from Library...' isn't tick...

[Adoby]

Unfortunately I don't think that calibre deleted the books without your intervention.

It isn't impossible, but highly unlikely.

I think it is much more likely that you caused the problems when you stored the calibre library on a NAS, or possibly when you moved the library to or from the NAS. There are for instance well known problems with using the calibre library on a networked filesystem and on operating systems that have different case sensitivity. If you ever did some work on the library from a Windows computer, that might have been the cause. Or if you used SMB or CIFS on the NAS. One symptom is exactly what you described. The books are listed in the GUI, but when you try to access them they seem to be missing. Another cause could be that you have changed the files in the library using some other tool than calibre, and then inadvertently made some minor change in the filename. Unfortunately, there can be quite a while between whatever you did that caused the problem and you discovering it, so even backups may have the same problem.

I have successfully stored my calibre library on a NAS, but I used NFS, not SMB or CIFS. And I once updated books from Windows, and experienced your symptoms. I was able to restore everything without problems. The books were there, just with the wrong case in the paths and the filenames.

Usually the books are not deleted but remain as abandoned "orphans" somewhere in the calibre library. They are different from what is stored in the database, or stored in a folder with a different name, so calibre can't recognize them. Perhaps with a slightly different name. But if you have copied the library from one system to another you may have inadvertently deleted those orphans, that wasn't copied with the rest of the library.

One way to check is to use the calibre database maintenance functions. Check the library. Another is to simply move the library to another folder using the calibre GUI, and see what files remain in the old calibre library folder. All on a local filesystem of course. It may even be possible to use the maintenance functions to fix the problems automatically, if you are really lucky. It might be a good idea to backup the calibre library before you try to fix it. That way you can be sure not to loose more books.

[DoctorOhh]

Originally Posted by alouest

Yes, books are deleted from my pc, not my reader.
It remain only their directories and opf files (sometimes covers files).

Just because directories have no books in them doesn't mean the book is missing. Now that the library is not on the network how did you recently discover that books were missing?

The only way to find out if books are missing is by looking them up in the GUI. For various reasons there may be orphaned directories in the database tree. Noticing books missing by looking in the actual file tree doesn't prove that any books are missing. This would be the case if you had broken books on your NAS and never ran library maintenance after moving the library to the local drive.

[BetterRed]

@alouest - if your using Windows on the PC then grab a file finder like Everything or Agent Ransack. Have it search for files & folders whose name containing one of the 'missing' author names or titles eg if one of the missing books was Fallen Order by Karen Liebreich I would look for Liebreich in the name. Do the search on each local and NAS drive.

Its highly probably that the format files are there somewhere. Once you find them then you will probably be able to work out why they are where they are, and not where you think they ought be.

BR

[alouest]

Thanks for your responses.

@DoctorOhh:
I discovered that books were missing by clicking "view" in Calibre 's interface : I got a message "no available formats". So, I applyed a filter "formats:false" ... and cried.

@BetterRed :
I perfomed a search using parts of author's name and of titles. I find nothing but I don't try all missing files (too many).

@Adoby :
The NAS is mounted as cifs.
I don't store the library on NAS anymore but I don't know when books disappeared (before or after moving library). I think it's after, because it seem to lack more books that when I have decided to move library on my pc but maybe I'm wrong.

I have move the library (using Calibre) : I found some books remaining in the old directory (only few, not all). But it's a good thing to know !

As I have a backup of the library, find disappeared books is not my priority.
I just want to understand why they disappear to avoid that a such thing happen again.

I understand that differents filesystemes / OS can lead some problems. I experienced such thing under linux by decompressing some archives created on a windows xp system.

I guess this is how it happened:

First, I rename and corrects the titles on a PC (windows xp).
Then I move the library on a NAS and use Calibre from an other PC (Linux Ubuntu). I continue my work on the library.
Finally, I move the library on my PC (Linux Ubuntu).
All library's moves where made by copying / pasting directly the library's directory.
All this is a good set of disorder among the files.
I think Calibre has deleted files by trying to modified them (because of the previous multiple moves).
Do you think it 's possible ?

Al.

[BetterRed]

I guess its vaguely possible that calibre deleted the files.

But its highly probably that files were lost when copying them across a network, between three different operating environments - XP, NAS (NOT supported by calibre) & Ubuntu - using different file systems - NTFS, Ext3/4, and whatever the NAS uses. And probably with lightweight copy programs.

The losses could have occurred in any one or all of the copy operations and what you see now is the cumulative effect.

BR

[DoctorOhh]

Originally Posted by alouest

@DoctorOhh:
I discovered that books were missing by clicking "view" in Calibre 's interface : I got a message "no available formats". So, I applyed a filter "formats:false" ... and cried.

I understood this is what happened when you were running from the NAS.

How did you move the library to the local machine? After you moved the library to the local machine did you run library maintenance on the library? On your local machine did you place your library in Dropbox or other cloud sync software?

Calibre does not delete books, so we need to figure out what on your system is causing this to occur.

[Adoby]

I don't think calibre deleted any books. I suspect that they were deleted as the library was copied/moved between different filesystems, and that calibre was used to copy the library at one step. And some books then were left behind and later deleted.

Keep the library on a local filesystem and don't switch between Linux and Windows. Then you should have no problems. If you update the library in Linux then Windows has no problem using a copy of that library, but if you update the library in Windows, then Linux may not be able to access some books.

[eschwartz]

Or else rebuild the database under Library Maintenance after every time you update in Windows.

[alouest]

I confess I do not use the "Library Maintenance" function.
I will.

Well, it now seems clear that the cause of the suppression was not Calibre.
Before opening a topic here, I searched if many other cases exist, and I had not found.
I wanted to be sure.
Now I am : As usual, the problem is between the keyboard and the chair ...


I considere this topic can be closed.

Thanks again for your answers.

@ DoctorOhh :
I copy / past the whole library's directory using files manager. I did'nt run library maintenance. I don't use cloud synchronisation (I use cloud for backup only).