Library crash - unable to rebuild - Page 3

cybmole · 06-12-2015, 06:46 AM

i have FFS sync set to do two tasks.
1. back up the calibre library folder
2. back up the calibre app data folder ( where the preferences & plug-ins live)

I don't worry about closing calibre before running.[ I hardly ever close calibre, except for updates ]
There is an option in FFS to say carry on regardless if it cannot lock a specific file/folder while it copies it. ( I need that ticked anyway when I do backup across LAN & there is no remote write access on the PCfolder that I am syncing from )

Ellachanted · 06-12-2015, 10:32 AM

Quote:

Originally Posted by cybmole

something like a large fan fiction library could be v hard to rebuild from scratch - so put an extra copy into cloud storage also, to be extra safe,

just pick one ( or more) of google drive, dropbox, one drive, amazon cloud drive... they all give you a few Gb for free.... and copy your entire calibre folder to there ( then don't forget the password!

then even if your entire PC goes up in flames, you still have that library !

Google drive is not a good backup for Calibre. That said, it can be rebuild from. (spoken from experience.)

Has anyone had luck with One Drive? My Copy.com backup is pretty full. I know Dropbox is the best for Calibre, but frankly, it is way too expensive.

The rebuild ran for about 2 days. But I got most of it back and was fortunate that a lot of the ones where the metadata didn't return were still on FFnet or wherever I got them from.

I've used SyncToy at work, I didn't realize it was a free program. It seems to work well. But it says it doesn't always work on Windows 8, so I've downloaded FreeFileSync. I will give it a go. I normally just back up by date, this will be much easier.

cybmole · 06-12-2015, 11:48 AM

they are all good and safe AS BACKUPS even google drive. so long as all you do it copy your live library to the cloud drive, and copy if back again if ever you need to,. the only danger with google drive is that it can corrupt live operations, so never use it for the live-in-calibre- program library.

FFS works fine in windows 8

i paid nothing for my 10Gb of dropbox. it is easy to earn extra space for referrals and to pick up other bonuses. e.g at one time they were giving away silly amounts of extra space just for enabling camera upload, so you turn that on & upload 1 picture to qualify....

google drive were giving an extra 2.5Gb for life just for running their security check....

one drive is only going to work on win 10 if you use PCs microsoft account logins. you won't be able to use it like you can in win 7/8 because MS want to tie it to your portable microsoft account.

BookJunkieLI · 06-12-2015, 12:08 PM

Quote:

Originally Posted by cybmole

they are all good and safe AS BACKUPS even google drive. so long as all you do it copy your live library to the cloud drive, and copy if back again if ever you need to,. the only danger with google drive is that it can corrupt live operations, so never use it for the live-in-calibre- program library.

This is true of Dropbox too. It's completely safe as a backup but it can muck things up if you're using it live and don't remember to pause syncing while working. After the mumblemumble time I forgot I finally switched to a portable drive and use Dropbox for backups.

Quote:

Originally Posted by cybmole

i paid nothing for my 10Gb of dropbox. it is easy to earn extra space for referrals and to pick up other bonuses. e.g at one time they were giving away silly amounts of extra space just for enabling camera upload, so you turn that on & upload 1 picture to qualify....

This was still true as of early spring when I setup my mother with a free account. I didn't bother going through all the offers because she doesn't need that much space but it is easy to increase your allotment with them before you need to go to a paid account.

chaley · 06-12-2015, 03:39 PM

Quote:

Originally Posted by cybmole

they are all good and safe AS BACKUPS even google drive.

Re Google Drive: what you say is true as long as there is absolutely no possibility that two computers might access the "drive", and even then we can't be sure. The problem: Google Drive can replace the caibre ID in the path with its own "revision number", thereby destroying the library. I speak from experience: this happened to me. I would never recommend using Google Drive to backup a calibre library because you are simply asking for trouble. Murphy says that if you need to recover the library then it has been corrupted, simply because it is possible. YMMV.

eschwartz · 06-12-2015, 03:52 PM

I suppose Google Drive could also be used with a mirror-style backup -- assuming you don't mind reimporting all books when you need to restore.

Personally, I prefer my backups to include a working metadata.db -- which renders GDrive useless.

theducks · 06-12-2015, 07:22 PM

Quote:

Originally Posted by eschwartz

I suppose Google Drive could also be used with a mirror-style backup -- assuming you don't mind reimporting all books when you need to restore.

Personally, I prefer my backups to include a working metadata.db -- which renders GDrive useless.

Why not Zip/Rar your Library to a GD folder? GD can do anything it wants to that file name

cybmole · 06-13-2015, 01:57 AM

good points. I do use an incremental mirror style backup, via FFS, into google drive. I did inspect various author folders for calibre ID integrity the last time we had this discussion and all was good. But I only have GD on this one PC and its basically a backup of a backup- the logic being it's 15gb of free cloud space why not use it.

eschwartz · 06-14-2015, 10:20 AM

Quote:

Originally Posted by theducks

Why not Zip/Rar your Library to a GD folder? GD can do anything it wants to that file name

Why don't I do my backup-disk incremental backing up via tape archives while I am at it?
Because zipped backups are so 1980s and there are better solutions.

And if GDrive doesn't allow me to use a good solution with it, I have other options so I don't need to bend over backwards to get GDrive to work.

Ellachanted · 06-19-2015, 02:11 AM

I had few problems with Google drive until I tried to sync it to another computer. Totally crashed the database. And somehow messed up the original db. Probably my fault. I forget to rename the libraries and get confused as to which is which.

Currently using Copy as a backup. I have 20 gbs but my Copy copy somehow has about 10K more stories than it should. I backed it up to an external hard drive and am going to delete the Copy library on my computer and copy the good library over. Hopefully this will help lower the size. Copy does work great, though I have to remember to turn on the sync when I update it.

I'm thinking of trying out Mediafire - I think that's what it's called. I think it was giving out something like 100gbs for $25 a year. I tried their free account but just to upload/download files. It's okay. Don't know how it will work with Calibre though.

I would rather not share my entire computer with Microsoft. Trying to set up 8.1 with a non-MSN account was hard. Actually, I had to use my Hotmail account to set up but it it setup with local sign in only. Not a big deal on that one. There's not a lot of personal info on it. But I've been avoiding updating my Win 8 computer to 8.1 just because of things like that.

cybmole · 06-19-2015, 04:21 AM

mega.co.nz still give 50Gb for free. no fancy sync features but if all you want to do is connect and upload/download....

i have a few family home movies & stuff stashed there.

alternatively , a fast 32Gb USB3 flash drive is only £8 in UK. connect it only when needed, sync to it with FFS or similar, lock it in a safe when not in use

buy 2 if you are paranoid

LadyKate · 06-26-2015, 07:45 AM

Quote:

Originally Posted by cybmole

i have FFS sync set to do two tasks.
1. back up the calibre library folder
2. back up the calibre app data folder ( where the preferences & plug-ins live)

I don't worry about closing calibre before running.[ I hardly ever close calibre, except for updates ]
There is an option in FFS to say carry on regardless if it cannot lock a specific file/folder while it copies it. ( I need that ticked anyway when I do backup across LAN & there is no remote write access on the PCfolder that I am syncing from )

Isn't there a problem with the metadata.db backing up?

HarryT · 06-26-2015, 08:00 AM

Quote:

Originally Posted by LadyKate

Isn't there a problem with the metadata.db backing up?

As discussed earlier in the thread, backing up "metadata.db" while Calibre is running is an accident waiting to happen. Sooner or later, it will get corrupted.

eschwartz · 06-26-2015, 09:40 AM

If you are taking a snapshot of it using Windows' Volume Shadow Copy service, or a linux LVM/btrfs snapshot, that shouldn't be a problem. Of course, this requires knowing what you are doing and would be the exception not the rule...

I believe FFS includes support for Volume Shadow Copy.

mbovenka · 06-26-2015, 09:45 AM

Quote:

Originally Posted by eschwartz

I believe FFS includes support for Volume Shadow Copy.

Yes, it does.

06-19-2015, 02:11 AM	#40
Ellachanted Connoisseur Posts: 50 Karma: 10 Join Date: Oct 2013 Device: Kindle touch	drives I had few problems with Google drive until I tried to sync it to another computer. Totally crashed the database. And somehow messed up the original db. Probably my fault. I forget to rename the libraries and get confused as to which is which. Currently using Copy as a backup. I have 20 gbs but my Copy copy somehow has about 10K more stories than it should. I backed it up to an external hard drive and am going to delete the Copy library on my computer and copy the good library over. Hopefully this will help lower the size. Copy does work great, though I have to remember to turn on the sync when I update it. I'm thinking of trying out Mediafire - I think that's what it's called. I think it was giving out something like 100gbs for $25 a year. I tried their free account but just to upload/download files. It's okay. Don't know how it will work with Calibre though. I would rather not share my entire computer with Microsoft. Trying to set up 8.1 with a non-MSN account was hard. Actually, I had to use my Hotmail account to set up but it it setup with local sign in only. Not a big deal on that one. There's not a lot of personal info on it. But I've been avoiding updating my Win 8 computer to 8.1 just because of things like that.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Personal experience with restoring library after crash	MiniMouse	Calibre	16	08-26-2014 04:57 AM
Rebuild library on PC from device?	CrankYanker	Library Management	2	04-22-2013 12:27 AM
Any way to rebuild meta data for library?	JeremyR	Library Management	2	07-08-2011 05:35 AM
Thumbnails, how to rebuild Library and get PDF thumbs for old uploads?	rakista	enTourage Archive	2	12-21-2010 10:29 AM
Retrieve Calibre library after Windows crash?	BikinOn	Calibre	3	05-25-2009 02:45 PM

06-12-2015, 06:46 AM	#31
cybmole Wizard Posts: 3,720 Karma: 1759970 Join Date: Sep 2010 Device: none	i have FFS sync set to do two tasks. 1. back up the calibre library folder 2. back up the calibre app data folder ( where the preferences & plug-ins live) I don't worry about closing calibre before running.[ I hardly ever close calibre, except for updates ] There is an option in FFS to say carry on regardless if it cannot lock a specific file/folder while it copies it. ( I need that ticked anyway when I do backup across LAN & there is no remote write access on the PCfolder that I am syncing from )

06-12-2015, 11:48 AM	#33
cybmole Wizard Posts: 3,720 Karma: 1759970 Join Date: Sep 2010 Device: none	they are all good and safe AS BACKUPS even google drive. so long as all you do it copy your live library to the cloud drive, and copy if back again if ever you need to,. the only danger with google drive is that it can corrupt live operations, so never use it for the live-in-calibre- program library. FFS works fine in windows 8 i paid nothing for my 10Gb of dropbox. it is easy to earn extra space for referrals and to pick up other bonuses. e.g at one time they were giving away silly amounts of extra space just for enabling camera upload, so you turn that on & upload 1 picture to qualify.... google drive were giving an extra 2.5Gb for life just for running their security check.... one drive is only going to work on win 10 if you use PCs microsoft account logins. you won't be able to use it like you can in win 7/8 because MS want to tie it to your portable microsoft account.

06-12-2015, 03:52 PM	#36
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85400180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	I suppose Google Drive could also be used with a mirror-style backup -- assuming you don't mind reimporting all books when you need to restore. Personally, I prefer my backups to include a working metadata.db -- which renders GDrive useless.

06-13-2015, 01:57 AM	#38
cybmole Wizard Posts: 3,720 Karma: 1759970 Join Date: Sep 2010 Device: none	good points. I do use an incremental mirror style backup, via FFS, into google drive. I did inspect various author folders for calibre ID integrity the last time we had this discussion and all was good. But I only have GD on this one PC and its basically a backup of a backup- the logic being it's 15gb of free cloud space why not use it.

06-19-2015, 04:21 AM	#41
cybmole Wizard Posts: 3,720 Karma: 1759970 Join Date: Sep 2010 Device: none	mega.co.nz still give 50Gb for free. no fancy sync features but if all you want to do is connect and upload/download.... i have a few family home movies & stuff stashed there. alternatively , a fast 32Gb USB3 flash drive is only £8 in UK. connect it only when needed, sync to it with FFS or similar, lock it in a safe when not in use buy 2 if you are paranoid

06-26-2015, 09:40 AM	#44
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85400180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	If you are taking a snapshot of it using Windows' Volume Shadow Copy service, or a linux LVM/btrfs snapshot, that shouldn't be a problem. Of course, this requires knowing what you are doing and would be the exception not the rule... I believe FFS includes support for Volume Shadow Copy.

Advert

Advert