How to Rebuild Database

[apswartz]

After a recent upgrade I discovered that the database was corrupted. It only showed 3 entries even though the directory contained all of the ebooks. I deleted the database file and restarted calibre thinking it would rebuild the database. oops! bad assumption on my part. I have the directory structure intact, but calibre doesn't recognize that there are ebooks in the folders.

Is there any way to rebuild the database from the directory and subdirectories that calibre created?

[animedude01]

No, there isn't. You've already asked this in several other threads and I answered in another as well.

Preferences > Advanced > Check Database Integrity checks the db and fixes errors, but there's no way to rebuild from the directory. The only way would be if you'd done a Save to Disk on all your books and then you can re-import them.

That's why it's always a good idea to back up your data, which includes the db file.

Hope this helps.

[apswartz]

Originally Posted by animedude01

No, there isn't. You've already asked this in several other threads and I answered in another as well.

The fact this comes up as often as it does suggests an easy way to do this is needed.

Actually I found a way to do it. Rename the directory with your ebooks and create a new directory with the name calibre saves ebooks to. Then start calibre then go to Add Books, but select the option to import from subdirectories.

As far as backing up, I do. But when the backup is a week old and you have added about a hundred books in the interim it is easy to miss some of the books you added.

[animedude01]

That pulls the books back in, but doesn't recreate the metadata, which is what I assumed you meant by rebuilding the database. The books are the easy part, it's recreating the metadata that's a hassle.

Week old backups are better than starting from scratch! You can start with the week old data and then just re-add and fix the hundred or so new books you've added since then. It's also a good idea to back up again after any decent size addition so if something like this happens, you don't have to do too much work

As to coming up often, I don't actually see it coming up all that often, but maybe I just haven't been looking hard enough.

I've been using Calibre for over a year and have never had the database get corrupt. If it's happening frequently for you, maybe there's a more fundamental problem with your system?

[apswartz]

Originally Posted by animedude01

That pulls the books back in, but doesn't recreate the metadata, which is what I assumed you meant by rebuilding the database. The books are the easy part, it's recreating the metadata that's a hassle.

It pulls the books back in recreating the database in the process. It seems to be the only way to do so at the moment.

Originally Posted by animedude01

Week old backups are better than starting from scratch! You can start with the week old data and then just re-add and fix the hundred or so new books you've added since then. It's also a good idea to back up again after any decent size addition so if something like this happens, you don't have to do too much work

Yes, I should have backed up, but I disagree with the first sentence when you are dealing with the number of lost books I had.

Originally Posted by animedude01

As to coming up often, I don't actually see it coming up all that often, but maybe I just haven't been looking hard enough.

All of the threads I responded to (which you mentioned) were this year. That doesn't make it that unusual.

Originally Posted by animedude01

I've been using Calibre for over a year and have never had the database get corrupt. If it's happening frequently for you, maybe there's a more fundamental problem with your system?

I have been using it for over a year as well, and I support it financially. I believe in it, I'm not sure what you mean. There is nothing wrong with my "system" as you suggest.

[kovidgoyal]

If you want to recreate the index of books from the library folder do the following two things:

1) Copy all the folders to a new location and delete them from the existing library.

2) Click the arrow next to the add books button and select add from subfolders one book per folder and point it to the new location to which you copied the book folders

That should recreate the index. You will lose any metadata you created in the previous database however.

[DoctorOhh]

Originally Posted by apswartz

As far as backing up, I do. But when the backup is a week old and you have added about a hundred books in the interim it is easy to miss some of the books you added.

I find the best way to back up the library is to create a directory called, for example, Calibre_Library_Backup then select all books and select save to disk and choose this backup directory.

Now as long as "save cover separately" and "save metadata in opf file" are checked in the preferences - Add/Save - Saving Books tab you will have a backup that will import back into Calibre flawlessly with all metadata in tact.

To keep this up to date anytime you add or update books just highlight those new adds or changes and save to disk to the backup directory again.

Relatively simple backup and it can be tested easy enough without screwing up your current library.

I do have one question, does this happen often?

[animedude01]

It re-adds the books to the database, but doesn't re-create any missing metadata. That means all your tags, corrections, etc. won't be re-created unless the metadata in the saved books had been updated. That generally happens when the books are loaded onto a reader, the format is shifted, or the books are Saved to Disk. If all you want are the books themselves, then yes, importing from the old Calibre library will pull everything in, but possibly with stale metadata.

Well, that's just my opinion. My thinking was that the week old database still has up-to-date metadata on the books you haven't added and it may be less work to re-create the metadata for 100 books rather than more. You may feel otherwise

Corruption can happen at any time, but I haven't noticed that it's common, especially considering how many people appear to be using it. Of course, I haven't been paying any particular attention so my views my be completely wrong

I didn't mean to imply that anything's wrong with your system. I may have simply misread your posts. When I read your multiple posts, I seemed to pick up some frustration and assumed that this had happened more that once. If so, I thought maybe there's something else wrong with your system, such as at the file system level or maybe a hard drive starting to go bad. Again, I didn't mean to imply anything in particular and may simply have misread your posts. If my posts implied anything else, it was completely unintentional.

[pilotbob]

Originally Posted by apswartz

As far as backing up, I do. But when the backup is a week old and you have added about a hundred books in the interim it is easy to miss some of the books you added.

How big is your book library folder? If it is less than 2GB you can back it up for free and in real time using Dropbox.

https://www.dropbox.com/referrals/NTMwMDU2MDk

If you have more than 2GB you can refer people to Dropbox and get up to 3GB more for free... or you can pay for it. Although, I think using Jungledisk is cheaper if you need more than 2GB and are willing to pay.

There are also some other offering. If you use Windows you can use LiveMesh which comes with 5GB of space for free.

BOb

[DoctorOhh]

Originally Posted by pilotbob

How big is your book library folder? If it is less than 2GB you can back it up for free and in real time using Dropbox.

https://www.dropbox.com/referrals/NTMwMDU2MDk

If you have more than 2GB you can refer people to Dropbox and get up to 3GB more for free...

I currently have the max free space offered 5.25gb.

Originally Posted by pilotbob

There are also some other offering. If you use Windows you can use LiveMesh which comes with 5GB of space for free.

I actually run Dropbox and Livemesh on my home computer.

I use dropbox to sync my files between 4 computers, 2 at home and 2 at work.

I setup Livemesh to backup my wife's folders so we both have 5gb of online storage automatically backing up our selected folders. She likes being able, when needed, to access her livemesh backup from the Churches computer.

They work fine together handling different data.

[pilotbob]

Originally Posted by dwanthny

I currently have the max free space offered 5.25gb.

How did you get the extra .25GB? I have the following on my referal page:

So far you've earned XXX of the 3.0GB maximum bonus space possible from referrals.

So, 2GB free plus 3GB for referral seems to add up to a max of 5GB free space?

BOb

[DoctorOhh]

Originally Posted by pilotbob

How did you get the extra .25GB?

You can earn 250mg by completing 5 out of the 6 items on the getting started tab(this tab is turned on and off in account settings).

[pilotbob]

Originally Posted by dwanthny

You can earn 250mg by completing 5 out of the 6 items on the getting started tab(this tab is turned on and off in account settings).

Oh... cool... I didn't even realize or remember I had earned that.

BOb