calibre library/ dropbox / system image questions

[cybmole]

I am trying to work thru a what-if & even with google help I am not finding a trustworthy explanation.

I keep my calibre library on C drive, & that library is daily copied to my dropbox folder, on the same C drive. ( I do also have a copy on another local drive but that's not part of the question)

occasionally , I make a system image of the whole C drive, so that I can recover from a hard drive failure

my what-if question is: what if the C drive fails or I decide to swap it out & rebuild from the system image. Clearly the system image will put back a much older version of my calibre library & will also put back an older version of my local dropbox folder.

I've been assuming that the cloud dropbox will still have the latest library copy and it will update any newly restored local version on the restored drive as soon as I go online ...
then I'd copy FROM local drop box to calibre library to get back to where i need to be ......BUT

..is that really what dropbox would do when PC goes online - will it realise that the (just restored) local files are actually older than it's cloud copies, or might it sync in the wrong direction? I know that dropbox claim to keep a 30 day backup of changes but I don't do system images that often.

The obvious answer is to rely not on drop box but on local external drives as my main backup, but I would like to know how dropbox handles the scenario of a C: drive being rebuilt from a system image ( which will rebuild the local dropbox folder and all of the local dropbox app data records )

[BetterRed]

@cybmole - My suggestion is to get all your data off the C drive/partition onto a separate partition so that the only thing on C is Windows, 3rd Party software and 'dont care about' data. You can then use different backup strategies for System and Data - eg image copies for System and file system copies for Data.

BR

[cybmole]

Originally Posted by BetterRed

@cybmole - My suggestion is to get all your data off the C drive/partition onto a separate partition so that the only thing on C is Windows, 3rd Party software and 'dont care about' data. You can then use different backup strategies for System and Data - eg image copies for System and file system copies for Data.

BR

I've considered that, in fact I used to have the dropbox folder on an external drive, but then I think those drives can't ever sleep, as dropbox will keep checking for syncs-to-do ?

Actually windows system image has to snapshot the whole physical drive, you can't image only a partition. So to implement your idea needs 2 or more physical drives.
For all data sets that would be a pain to have to replace, I use free file sync to maintain backup copies on a secondary drive anyway. My C drive is 4 + years old now so I keep thinking I ought to swap it out anyway, but it's such a hassle

FWIW I've send a query to dropbox help, asking how they handle synching after a restore from system image.. I guess similar questions would arise for users of acronis or other whole disc backup solutions

I do also have the same dropbox account on a 2nd PC & when I switch that on I see it downloading any recent calibre library changes to the dropbox folder on that PC, so I know that sync works well across 2 PCs. If I had a primary drive failure, I think I'd go to that other PC & take a full backup of my dropbox folder on there, before bringing the 1st one back online, then if stuff gets wrongly synced , I'd know that i can fix it.

[BetterRed]

Originally Posted by cybmole

I've considered that, in fact I used to have the dropbox folder on an external drive, but then I think those drives can't ever sleep, as dropbox will keep checking for syncs-to-do ?

Actually windows system image has to snapshot the whole physical drive, you can't image only a partition. So to implement your idea needs 2 or more physical drives.
For all data sets that would be a pain to have to replace, I use free file sync to maintain backup copies on a secondary drive anyway. My C drive is 4 + years old now so I keep thinking I ought to swap it out anyway, but it's such a hassle

FWIW I've send a query to dropbox help, asking how they handle synching after a restore from system image.. I guess similar questions would arise for users of acronis or other whole disc backup solutions

I do also have the same dropbox account on a 2nd PC & when I switch that on I see it downloading any recent calibre library changes to the dropbox folder on that PC, so I know that sync works well across 2 PCs. If I had a primary drive failure, I think I'd go to that other PC & take a full backup of my dropbox folder on there, before bringing the 1st one back online, then if stuff gets wrongly synced , I'd know that i can fix it.

@cybmole - My externals sleep, although they're normally off when I don't want them.

I believe the EaseUS Partition Master product will image partitions - I've not used it.

The problem with image backups is that you're backing up the accumulated detritus since you last did a fresh Windows install. For that reason I've given up on taking image copies, I think I only restored from an image once to good effect, on several occasions I've reinstalled from image only to find that 'the problem' I'm trying to fix predates the oldest image.

There's a 'feature' for Windows Vista/7/8 called in-place upgrade install, essentially it gives you a pristine Windows install without the need to reinstall applications. You might have to fix a few things like reinstall odd drivers (eg I have an ASUS BT dongle), do a bit of repair work in browsers and reapply OS tweaks, but other than that everything remains as was.

It takes longer but its a nice warm feeling to have a pristine Windows.

BR

[Adoby]

A backup system that hasn't been tested can't be assumed to work. If you want to know what will happen in different circumstances, test it. It will ensure that you know what will work, and what won't. And how to restore backups.

Make a backup to an external drive (or two) and fake the scenario you described. And see what happens.

I have a script that take a full backup of my calibre library automatically every time I exit calibre, using rsync. And I store them 30 days on a separate internal disk and I also replicate the backups daily over to my NAS. Some other stuff I automatically backup the same way every time I start the computer. And I also have weekly backups on external drives. Unfortunately my Internet up speed is too slow to use for backups.

[eschwartz]

I would imagine it acts the same as if you use a regular computer installation that simply hasn't synced in a really long time. I believe Dropbox caches checksums of your files locally in order to check if files have change on disk, and if the client reports that no files have been changed from what Dropbox expected, then there are no local changes. Then it consults the server and sees changes, and syncs them.

So, as long as the system image is of a point when Dropbox was up to date, you should be fine.

That being said, why waste time imaging your data when only the OS needs imaging? There are alternatives which can do single-partition backups.

[cybmole]

Originally Posted by eschwartz

So, as long as the system image is of a point when Dropbox was up to date, you should be fine.

but that is the crunch issue.
it could be up to date I was doing a planned upgrade, but not if I was responding to an unplanned drive failure. - i had a reply from dropbox which did not answer that clearly.
if this hard drive failed in the next few minutes, I'd be rebuilding from a system image taken a few weeks ago, so the calibre library within local dropbox within that system image would be well out of date.
I need to know if dropbox woud see its cloud copy as " newer" than that restored local copy, or vice versa

[eschwartz]

And as I said -- based on my understanding of how it works when a PC is in use but Dropbox was not started in a while, the client will record the state of the local files and the sync status. If the files have not changed locally since Dropbox ran, it will assume the cloud changes are authoritative.

If restoring, there is no doubt in my mind that it is the equivalent of syncing calibre, turning off the PC for a year, updating file on other computers, then coming back to the old computer and starting up the Dropbox client. It will download changes from the Dropbox server, overwriting the stale data saved locally.

If you hd changed files on the PC since the last sync, the checksums would mismatch and the dropbox client would upload the changes. This would be reported on the web interface with an option to revert. Like always. There is a good chance Dropbox would sense a conflicted copy situation, too.

[cybmole]

hopefully you are correct, but would you bet your calibre library on that theory ?

I see no simple way to test, except for having a spare PC & doing that whole what-if for real.

My concern is that windows could stupidly somehow put today's date on all the restored image files, - it shouldn't, but but I'd be happier if dropbox had better documentation or gave a clearer reply.

From what I can puzzle out dropbox replies on hidden database files in app data to record local states & it's not forthcoming as to exactly how they work

PS tell me more please about ways to image only a windows installation+ all of its installed programs, so that all of that can be restored to a replacement drive without needing an actual fresh install of windows & a fresh install of each program.

[BetterRed]

Originally Posted by cybmole

PS tell me more please about ways to image only a windows installation+ all of its installed programs, so that all of that can be restored to a replacement drive without needing an actual fresh install of windows & a fresh install of each program.

Not sure if you can image selected folders or exclude folders from an image.

What a lot people do is to have a partition for OS & programs usually C: and another partition for data usually E: (D: being the CD ROM). IMO that's 'best practice'.

You can then image copy C: with something like ==>> Create a complete partition clone with EaseUS Partition Master

I am NOT recommending that product, and I doubt its the only one. Search for 'partition image' and 'partition clone"

To re-install Windows and keep applications you can do this Repair Install - Windows 7 Help Forums I have fixed up about a dozen broken Windows installs following that process. Obviously this only works on a functioning disk.

BR

[cybmole]

i had a bad experience with one of the free cloning products - pretty sure it was easeus- , so I've reverted to only using windows tools fro hard drive swap outs.
What happened was that i cloned an existing C drive onto a fresh partition of a quieter + larger drive , using easeus, & it seemed ok at first - the initial boot failed but windows offered to fix that by re-writing some boot sector, & it then booted up fine - but then the PC could not recognize mouse, keyboard , external drives etc when they were connected to different USB ports
The clone process had somehow completely messed up the USB drivers. I spent some time partially fixing it by comparing & retrieving drivers from another PC, but then decided I did not really want to trust the clone, & put the original noisier drive back in.

I am taking an interest in all of this because I had a data drive die unexpectedly recently. A WD caviar green 1TB just went from 100% OK to 100% dead.-
Luckily I had enough redundancy to recover from that as it mostly held backups. My local google drive was on there, so it was lost but that restored OK from cloud.
It only take one such "surprise" to get you thinking - how safe are my other drives & data!

[BetterRed]

@cybmole - I do recall reading a number of -ve reports about free Easus a few years ago. EaseUS Partition Master happened to come up first when I googled 'image copy partition'. I know a number of people who 'swear by' Easus Partitioning products - although in Win 7/8 I don't know what they do that they couldn't do Windows.

My experience with Windows backup is contrary to yours, for me it doesn't work and it has never worked.

What I neglected to say previously is that I have Macrium Reflect to image partitions. The pay-4-me version can also do incremental and differential image backups - so no long waits. I've never had any problems with using Macrium to do a recovery. I know a few people who use its Free version and on a couple of occasions I've used its backup media to build an as-was system partition on a new drive.

BR

[chaley]

I have successfully cloned drives on three occasions in the last two months using MiniTool Partition Wizard Free (Home). What I needed was to clone drives while reducing the partition size (in two instances) and increasing the partition size (the remaining instance), which it did without hassle or problem. One instance was a data drive, the other two were system (C) drives. One of the system drives had developed a bad patch which was happily in the free list and one file. Neither windows backup nor Acronis TrueImage Home (2013) could handle that. Partition Wizard sailed through it, noting the damaged file.

[rwmcafee]

Originally Posted by chaley

I have successfully cloned drives on three occasions in the last two months using MiniTool Partition Wizard Free (Home).

I use this software too and like it.

FYI - BitsDuJour dot com has the MiniTool Partition Wizard Professional Edition for free today.

[cybmole]

Originally Posted by eschwartz

I would imagine it acts the same as if you use a regular computer installation that simply hasn't synced in a really long time.....

...

today's reply from dropbox ( an answer to " what happens if I restore a 6 week old system image" ) says the opposite - I'm not sure the help guy is believable though
he says:
"Thanks for your reply.
If you restore your computer from a system image, you will first need to verify that the Dropbox programme has been reinstalled and is syncing correctly.
If it is not then you would need to download and install the programme again.
If the system image correctly restores Dropbox and it functions correctly then it would overwrite the existing files in your account when it syncs.
This is because it would reset those files to how they were 6 weeks previously.
"
ADDITION:
there is much more ( confusing) debate here:
http://superuser.com/questions/69096...rwrite-the-clo

there's a statement part way down that page which seems to back up the above reply that I posted from dropbox

don't know the technicalities of this, but I do have the situation where I and my assistant operate a common dropbox with two different computers going on and off at different times.

The answer to your question is that when you connect with the Cloud, Dropbox knows (somehow) that the version currently on the Cloud was originally synced from your computer. So if your computer version is different, that is the version that will be synced to the Cloud.

all of which is making me think that I don't want dropbox to be any part of my calibre library backup strategy. I'm happy to use it as a go-between for getting books to tablet when PC is off ( via calibre cloud app). But i'm now thinking multiple local library backup copies ( and one dumped into google drive also for good measure)