A lesson on the importance of backups...

[shalym]

Originally Posted by HarryT

But you presumably do your database backups using the backup facilities of MS SQL Server itself (just as I do), not by simply copying the underlying file system files. If you use the backup facility of the database itself, then yes, it does it properly, no argument about that, and no need to shut anything down. But that's not what we're talking about in this situation: it is assuredly not safe to simply copy the files of an open database. You might get away with it 9 times out of 10, or even 99 times out of a hundred, but sooner or later you'll hit problems when you have pending transactions that haven't been committed to the database.

It depends on the solution. For some, yes, that is the procedure. For others, the database backup script is included in the main backup script for the server. I'm just saying that making a blanket statement of "backing up an open database will cause corruption" isn't necessarily true. If it were, then most mail servers wouldn't be backed up, and neither would most websites--these days most websites have at least one component that is completely dependent on a database, and also need 24/7 uptime.

Shari

[HarryT]

Originally Posted by shalym

I'm just saying that making a blanket statement of "backing up an open database will cause corruption" isn't necessarily true.

I agree, with you, and if you look back at my posts you'll see that I didn't make any such generic statement. What I said specifically was that copying a database such as the one that Calibre uses (ie a file-based, not server-based, database) while an application has the database open was an unsafe practice, and that's a statement that I 100% stand by. You'll probably get away with it the majority of the time, but it is nonetheless not a good idea.

[AnotherCat]

FreeFileSync, for example, gives the option of using the Windows Shadow Copy Service which should be quite safe for backups (i.e. a one way update or one way mirror to an archive target, not a two way sync) of Calibre when open (and email and other databases too); in any event I have never struck a problem with it.

I am not promoting FreefileSync in saying that, it is just the one I have used for a number of years now, I have no doubt there are other alternatives. However, as far as I recall SyncToy does not use the Shadow Service (and there will be others too, so I am not picking on that; again it just the one I used to use) and the warning given by others applies to updating live databases (SyncToy's original purpose was for simple backups of the likes of directories of photographic images and its functionality never really developed beyond that type of purpose).

[eschwartz]

Originally Posted by Katsunami

RSync is free, and much more powerful than both. However, it requires Cygwin on Windows, and it's hard to use if you're not comfortable on the command line.

Just use what works for you. I have had reliability problems with getting all files copied with SyncToy, couldn't figure out why, and finally ended up with a custom rsync / bash script.... on Windows.

+111 to rsync for power and flexibility (if you know how to use it).

[eschwartz]

Originally Posted by soulfuldog

@ eschwartz, did you post instructions on how to use task scheduler in Windows to automatically save any changes to a certain folder (like your Dropbox folder) at one point? I'm sure I had something like this set up but just checked and my local back-up hasn't been updated since October, so something has went wrong!

I have mentioned setting cron jobs under linux, which is more or less the same concept.
All you do is setup rsync with the right options and exclusions and stuff, make a bash script to run those commands, and run said bash script at regular intervals.



You can do the same thing with FreeFileSync: http://www.trustyetc.com/trustyblog/...-freefilesync/
You setup FreeFileSync to do what you want, then go to File ==> Save and you can save the settings to run FreeFileSync with that setup via a script. Then go to Task Scheduler and schedule FreeFileSync to run regularly, with the saved settings file as a parameter.

[Tarana]

A friend of mine had their laptop stolen as well as the external hard drive they used for backups. Years of irreplaceable family photos lost.

This is so true to have off site backup. Heat and smoke will destroy data disks as was discovered by my cousin in a fire last March. An acquaintance had all her HD and data disks taken by the police because a family member was being investigated and a number of them were 'lost.' In her case though, she had her photos backed up to Walgreens cloud and ebooks in Dropbox. The most common though is as Harry describes, a failed HD.

[Mivo]

I have been using DirSync Pro (entirely free and open source) for many years and swear by it. What I love most about it is that it manages obsolete files and only backups files that are new or have been modified, so there is little overhead.

I back up my laptop's drive to an external USB drive roughly once a week. Important files, such as a ZIPed backup of my Calibre library, additionally live in cloud space (and on my tablet).

[HarryT]

Originally Posted by Mivo

I have been using DirSync Pro (entirely free and open source) for many years and swear by it. What I love most about it is that it manages obsolete files and only backups files that are new or have been modified, so there is little overhead.

Pretty much any backup tool will only back up modified or new files.

[DiapDealer]

I still think not having any data worth backing up is the way to go.

If the data's analog equivalent sat in piles gathering dust, I'm not going to stress over losing the digital version either. There's several rooms in my house I wish something would accidentally erase the contents of.

[HarryT]

Originally Posted by DiapDealer

I still think not having any data worth backing up is the way to go.

If you're able to live your life unburdened by anything that's of any personal value to you, good for you; it must be a very relaxing way to live. Unfortunately, not everyone is able to do likewise. I personally re-read a great many of my books, and hence place a high value on my ebook library. I also run a business for which I am obliged by law to keep financial records securely.

[DiapDealer]

Originally Posted by HarryT

If you're able to live your life unburdened by anything that's of any personal value to you, good for you; it must be a very relaxing way to live.

How you determined (from what I said), that I have nothing in my life that's of any personal value to me, I'll never know.

[soulfuldog]

Originally Posted by eschwartz

I have mentioned setting cron jobs under linux, which is more or less the same concept.
All you do is setup rsync with the right options and exclusions and stuff, make a bash script to run those commands, and run said bash script at regular intervals.



You can do the same thing with FreeFileSync: http://www.trustyetc.com/trustyblog/...-freefilesync/
You setup FreeFileSync to do what you want, then go to File ==> Save and you can save the settings to run FreeFileSync with that setup via a script. Then go to Task Scheduler and schedule FreeFileSync to run regularly, with the saved settings file as a parameter.

Thanks eschwartz, I eventually rediscovered I'd set up FreeFileSync and my PC just hadn't been on for a while during the time set in Task Scheduler. I've such a bad memory I couldn't remember it was FreeFileSync I was using, the only thing I remembered was that it was one of your posts that I followed to set something up in the first place....

[AnotherCat]

Originally Posted by DiapDealer

...If the data's analog equivalent sat in piles gathering dust, I'm not going to stress over losing the digital version either. There's several rooms in my house I wish something would accidentally erase the contents of.

Which perhaps brings us to the position that we only do backups because we can, in which case they must be redundant.

Before we could do backups hardly anyone kept backup copies of paper books, photographs, mail, business records, etc., etc. off site and we thought nothing of it except just being a little bit careful not to burn the house down or to let burglars in.

[wawasteele]

Originally Posted by AnotherCat

Which perhaps brings us to the position that we only do backups because we can, in which case they must be redundant.

Before we could do backups hardly anyone kept backup copies of paper books, photographs, mail, business records, etc., etc. off site and we thought nothing of it except just being a little bit careful not to burn the house down or to let burglars in.

Having had a house fire many years ago, the first things we grabbed to get out while we could still get things, were all our photo albums. I'm so glad that nowadays all photos for the last 15 yrs or so are digital and are backed up to the cloud in addition to a local backup.

[grumbles]

With cheap DVD (and not so cheap Blu-ray) writers and really cheap media (DVD blanks are well under a dollar and Blu-ray blanks are under two) there is no reason not to backup. My Calibre library fits on a dual layer DVD (but a rewriteable Blu-ray is more convenient.) If the data is important, I keep backups of my photographs, parity files add a degree of security. Note that parity files like par2 only protect data sectors but not the file system. For the truely paranoid, a utility like DVDisaster creates parity of the entire disc including the file system. But then those parity files need to be protected.

There are many things that don't need to be protected or saved but what does need to be saved has to be saved well.

It's also essential to label the discs. Unlike printed material, you just can't look at it to see what it is. I keep finding far too many discs lying around that I have no idea what is on them.