Did Calibre really just delete my entire collection?!

[GavinZac]

Its 4am and I've been working on organising my book collection via Calibre for the last 6 hours. I've just had an SQLite error, and now Calibre shows no files - and neither does my file manager.

Please tell me Calibre would not delete every single file on the basis of an error?

[kovidgoyal]

calibre never deletes files on receiving an error.

[GavinZac]

I don't think its wise to make such assumptions!

If Calibre found itself with an empty database suddenly, would it clear out the chosen Calibre eBook directory? Because after the error my library was gone, and unfortunately so were 400 eBooks.

Is there a log file where I can find out the exact error?

Calibre seemed like such fantastic software that I'm actually probably going to try again - the alternative is going through a ton of LIT files by hand. But I'd like to isolate what this problem was, so as to avoid it in the future (and obviously, im going to back up the books which had yet to be imported to Calibre, this time )

[itimpi]

I can tell you from experience that Calibre does NOT clear out the library folder if there are files in there it does not have in its metadata database.

When Calibre is about to delete any files it has a confirmation dialog before it proceeds, so again it seems unlikely it can go mad deleting files.

I guess a lot depends on what caused the orignal error. If it was at the file-system level it could have been Windows that deleted all the files!

Having said that, I do back up the Calibre database and associated files at regular intervals as a lot of effort has gone into organising my collection and getting all the metadata and cover art as I want them, let alone the conversions into formats suitable for my e-book readers.

[wallcraft]

I suggest searching your entire system for the ebook files. Perhaps they got moved somehow. What operating system are you running under?

[pilotbob]

Originally Posted by GavinZac

I don't think its wise to make such assumptions!

I'm pretty sure Kovid, they guy that wrote calbire, isn't assuming here.

Did you look in your Recycle Bin. It's easy to make a slip of the mouse sometimes.

BOb

[kovidgoyal]

Originally Posted by GavinZac

I don't think its wise to make such assumptions!

If Calibre found itself with an empty database suddenly, would it clear out the chosen Calibre eBook directory? Because after the error my library was gone, and unfortunately so were 400 eBooks.

Is there a log file where I can find out the exact error?

Calibre seemed like such fantastic software that I'm actually probably going to try again - the alternative is going through a ton of LIT files by hand. But I'd like to isolate what this problem was, so as to avoid it in the future (and obviously, im going to back up the books which had yet to be imported to Calibre, this time )

The only times that calibre deletes anything is when you ask it to delete a book and when you change the metadata on a book. On the latter ocassion, the book is first copied to a new folder and then the original folder is deleted.

What exactly is deleted? The library folder, the metadata.db file, all the individual book folder? Only the book files?

[GavinZac]

Originally Posted by wallcraft

I suggest searching your entire system for the ebook files. Perhaps they got moved somehow. What operating system are you running under?

I'm using Ubuntu Intrepid, everything up to date. They definitely aren't in any trash or temp folder, in fact I don't have any LBR files on the computer - I had been using Calibre to convert my LIT collection to LBR

Originally Posted by pilotbob

I'm pretty sure Kovid, they guy that wrote calbire, isn't assuming here.

It's not an accusation on my part toward Kovid, but as a software developer myself I know that assuming your software doesn't do something is a quick way to find out what it can do! Like one Windows Mobile program which sync'd with not just the host PC but each on the network...

Originally Posted by kovidgoyal

The only times that calibre deletes anything is when you ask it to delete a book and when you change the metadata on a book. On the latter ocassion, the book is first copied to a new folder and then the original folder is deleted.

The only thing I can think of so, is that perhaps it tried to update metadata, thought it had succeeded but it hadn't. Again, is there a log file I could go through?

What exactly is deleted? The library folder, the metadata.db file, all the individual book folder? Only the book files?

The folder is there, the metadata.db file is there, and one other folder, a folder called "Nora Roberts" (don't laugh, it's my girlfriend's honest ) with absolutely nothing in it

Is there a way to check if there's anything in the metadata.db file? I've assumed its empty because Calibre doesn't show anything, but I could be wrong.

Anyway, I've reassembled my collection and will give Calibre another go, retaining a backup. I'll report any further errors I get here.

Please don't think I'm trolling. I'd love to be the guy trolling as I'd still have my books. I'd just like to not have this happen again to me, or others I guess. But mostly to me

[kovidgoyal]

Originally Posted by GavinZac

The only thing I can think of so, is that perhaps it tried to update metadata, thought it had succeeded but it hadn't. Again, is there a log file I could go through?

That would at worst have removed a single folder, not all of them.

The folder is there, the metadata.db file is there, and one other folder, a folder called "Nora Roberts" (don't laugh, it's my girlfriend's honest ) with absolutely nothing in it

I really can't think of a situation that would lead to this, and I've never heard of it happening before. What linux filesystem is your database stored on?

Is there a way to check if there's anything in the metadata.db file? I've assumed its empty because Calibre doesn't show anything, but I could be wrong.

The database is a sqlite database. You should be able to use any of a dozen different tools to browse the database.

Best way to get a log is to simply start calibre in a terminal. It will print any unhandled errros to stdout (it also shows them in the GUI).

[GavinZac]

Originally Posted by kovidgoyal

That would at worst have removed a single folder, not all of them.
I really can't think of a situation that would lead to this, and I've never heard of it happening before. What linux filesystem is your database stored on?

Ok. I'm using EXT3, no fancy EXT4 for me yet.

The database is a sqlite database. You should be able to use any of a dozen different tools to browse the database.

I'll take a look, but I've actually restarted importing them so it's probably pointless.

Best way to get a log is to simply start calibre in a terminal. It will print any unhandled errros to stdout (it also shows them in the GUI).

Ok cool. Maybe as a feature for next release it could output a crash dump or something on errors? I'll try to run it from terminal from now on and if I encounter this again I'll pop the output into this thread.

As I said,I'm going to give it another go. It's a great program, and I've just introduced my girlfriend to it. She told me to say it's fantastic, and was surprised when the guy writing it was also the guy answer questions on a forum

[pilotbob]

Originally Posted by GavinZac

Anyway, I've reassembled my collection and will give Calibre another go, retaining a backup. I'll report any further errors I get here.

Sign up for dropbox (getdropbox.com) and then move your calibre library to your dropbox. This will give you automatic offline backup. Also, dropbox revs stuff so if it is deleted from your pc you can recover it.

BOb

[Andybaby]

and I know that an error like this is possible. it has happened to me before. in windows. with another program.

I was moving Data using a well known file manager from one drive to another, neither of them the C drive, I hit cancel, and it took that to mean to delete my c:/windows folder.

Im entirely sure that it was a freak accident and will probably never happen again. with that program, or calibre

[kovidgoyal]

Originally Posted by GavinZac

Ok cool. Maybe as a feature for next release it could output a crash dump or something on errors? I'll try to run it from terminal from now on and if I encounter this again I'll pop the output into this thread.

Like I said, unhandled errors are displayed in the GUI and printed to stdout. Since calibre is written in python, it doesn't usually crash, and even when it does, a dump wouldn't provide much useful information.

As I said,I'm going to give it another go. It's a great program, and I've just introduced my girlfriend to it. She told me to say it's fantastic, and was surprised when the guy writing it was also the guy answer questions on a forum

Actually, answering questions on the forum is a big part of what keeps me motivated to continue developing calibre.