The Technology Vent and Rant Thread

[binaryhermit]

I'm guessing that the Airpods Max would be decent $200 or $300 headphones that you pay an extra several hundred dollars in Apple tax so you can have a case that looks like boobs or a butt

[Hitch]

Hey, gang:

My rant is at the end, so...feel free to ignore this next bit. Question? Does anyone here know of a service, like Carbonite, let's say, that allows for cloud-to-cloud backups (think MultCloud), rather than direct uploads from the specific computer/server, and also has harddrive/physical media disaster recovery?

The reason I ask is, I have nearly 2TB of data to back up. I live rurally, which means that the nearest lockbox or bank box location is a 35mi RT. Making a backup and keeping it in my home is all well and good, but it won't save my ass in case of a Fire. We have the vast bulk of the company data on Dropbox (for this reason) but I just replaced my 2T RAIDS with 6T Raids and had to reinstall DB (didn't have a slot in which I could simply copy the drive over, unfortunately) and it took 12 DAYS to redownload/synch the DB data. And that's the FAST direction, down. For upload? I mean, FUHGEDDABOUDIT.

I use Multcloud, which means I could put a copy of the DB data on Amazon S-3, or GDrive, etc., but in case of a loss, we're back to...yup, 12 days for recovery.

I tried looking at Backblaze, but they are ixnay on the C2C. They do mirrors and again, no matter how "mirror-y" you get, that's a lotta data to put through a slow pipe.

Ideas? Other than the dreaded, backup to my 6TB external drives and drive that sucker to the bank? I figured if anybody in the Nerd Herd here would know, it would be one of youse guys. :-)

(Rant: is there some reason the Backblaze guy had to act as though I was thinking of sacrificing babies, when I asked about C2C instead of mirroring? I mean, really????)

Hitch

[pdurrant]

Originally Posted by Hitch

The reason I ask is, I have nearly 2TB of data to back up. I live rurally, which means that the nearest lockbox or bank box location is a 35mi RT. Making a backup and keeping it in my home is all well and good, but it won't save my ass in case of a Fire. We have the vast bulk of the company data on Dropbox (for this reason) but I just replaced my 2T RAIDS with 6T Raids and had to reinstall DB (didn't have a slot in which I could simply copy the drive over, unfortunately) and it took 12 DAYS to redownload/synch the DB data. And that's the FAST direction, down. For upload? I mean, FUHGEDDABOUDIT.

Is your RAID a mirrored raid? How about replacing one of the disks with a fresh disk, letting it rebuild in the background, while you post the removed disk to a trusted location. Then do that again in whatever time frame seems good to you. When the trusted location receives the latest disk, they ship the earlier disk back to you to be reused. You'd need four disks instead of two. Two would always be in use, one would be in the trusted location, and one would be in transit, or waiting with you to be reused.

Disaster recovery is just the time to get replacement hardware, and the off-site disk, and rebuild the raid from it. probably a lot less than the two week needed to download from from the internet!

With incremental backups to the cloud storage, you could also recover the work done in the time since the last offsite backup was sent, in a lot less time than getting all the data down from the cloud.

[PeterT]

Years ago I seem to remember where I worked using the remove a drive from a raid1 array as a means of backup on a production environment.

[Hitch]

Originally Posted by pdurrant

Is your RAID a mirrored raid?

Yes, (mirrored) we are considering that and we don't see a viable alternative. We've thought about truning a computer into a file server, but I need more levels of complexity like I need a hole in my head.

How about replacing one of the disks with a fresh disk, letting it rebuild in the background, while you post the removed disk to a trusted location. Then do that again in whatever time frame seems good to you. When the trusted location receives the latest disk, they ship the earlier disk back to you to be reused. You'd need four disks instead of two. Two would always be in use, one would be in the trusted location, and one would be in transit, or waiting with you to be reused.

The problem with that is, it's a snapshot. There are no incremental backups. It's all or nothing and if you crash, and your snapshot is from a week ago...you're hosed. At least, I would be.

While it's true that I can restore files from Dropbox, as I have a commercial account with 180-day restoration capabilities--here's the thing. I'd have to KNOW that I needed to restore it, because it was updated after the snapshot. I mean...I have just around 1M files in the DB. I wouldn't know what was updated versus what wasn't to save my damn life. Keep reading, it gets murkier....

Disaster recovery is just the time to get replacement hardware, and the off-site disk, and rebuild the raid from it. probably a lot less than the two week needed to download from from the internet!

I agree with you there.

With incremental backups to the cloud storage, you could also recover the work done in the time since the last offsite backup was sent, in a lot less time than getting all the data down from the cloud.

Yes, but...if I pop in a new RAID drive--remember, this is Dropbox we're talking about here--the older dates will OVERWRITE the data on the DB.com website, which is meant to be the incremental backups.

I just ran into DB's limitations, in a very unpleasant way, finding out that if you uninstall DB and reinstall it, despite representations from DB support to the contrary, that it overwrites the goddamned dates for the folders. So, everything I have now, folder-wise, is date-stamped with the time and date it re-installed, NOT the time and date that the file or folder was actually created or modified.

So, my fear is...and think this through with me--I have a crash. I have the RAID in storage. I pull it out and let's say, for S&G's, that it's 5 days old. I reinstall the RAID and it overwrites all the dates on DB. That in turn synchs up to the Cloud, which in turn ruins all the incremental data.

Right? Isn't that what happens? I mean, DB's real down-n-dirty is heavily hidden. It's nearly impossible to get the real skinny on what it does or how or the sequence, but it's not doing what they say it should do and you can only--get this!--restore files with their original dates through their new (utterly useless) desktop app. Which is a NIGHTMARE to use. I mean..seriously? I'm meant to restore a million files, through their desktop app, instead of simply reinstalling the DB *software* and re-synching the files? Why the hell would the folders re-date? It makes NO sense.

Originally Posted by PeterT

Years ago I seem to remember where I worked using the remove a drive from a raid1 array as a means of backup on a production environment.

Right and that's how we thought about it, but I keep running into this problem, trying to think around HOW not to utterly screw the pooch on the dates and how DB works.

Hitch

[pdurrant]

It sounds like Dropbox isn't designed for this kind of backup system. But I'm afraid I don't know what is.

[badgoodDeb]

Hitch, you aren't Unix based, are you? (I think you aren't) Or even Mac (which is Unix at heart).

I manage Unix backups, and could talk you through a backup to disk system .... and then you post those disks to someone, as described above, for your offsite backup. Unix can do incremental backups nicely .... unless all your data looks like a single item (a Virtual PC, for instance, looks like a single file to Unix, I believe).

[binaryhermit]

Actually, Macs are running the only operating system in widespread use that's actually *technically* Unix.

Linux and BSD are close enough for practical purposes.

[Hemmi]

Originally Posted by Hitch

The reason I ask is, I have nearly 2TB of data to back up. I live rurally, which means that the nearest lockbox or bank box location is a 35mi RT. Making a backup and keeping it in my home is all well and good, but it won't save my ass in case of a Fire.

How about a low tech solution of one of those fireproof safes to keep your backup in?

[Tarana]

My Echos are driving me crazy. I should be able to go from room to room with a synced audiobook. Even the SAME Echo will sometimes start 2 chapters in despite being on chapter 28. I have to bluetooth my Fires to play my audiobooks on the Echoes and it won't even recognize half of them. Grr..

[Hitch]

Originally Posted by pdurrant

It sounds like Dropbox isn't designed for this kind of backup system. But I'm afraid I don't know what is.

Yes, well...it's changed, quite a bit, since I first implemented DB back in...IDK, '08 or '09? I first went to DB when my first bookmaker came to work part-time for me, a guy across a given pond. It was at the time the best and easiest file-sharing method, plus it had the added benefit of allowing me to access my working files and travel, too.

But: they have changed the synching protocols. No two ways about it. I've "hotswapped" RAIDS before. (Yes, I know, that's misspeaking, but you get my drift.) I did it to go up to the 2.0TB drive and I didn't spend weeks reinstalling the data and I most certainly didn't get corrupted, fiddled modification dates, which is pissing me off no end. They've changed something and that's significantly altered how it works.

We have ~14 bookmakers and 3 admin employees, working around the globe. We handle MASSIVE files, I mean, it's not uncommon for us to do intake on 3gig source files. It's simply...too b*ggering unwieldy for me (or anyone in admin) to a) download that sort of source file from a customer; b) share it in a different way with my Queen o'Quotes (my person who does all our qutoing); then c) move it yet again to production, whereupon d) the bookmaker then has to download it to his/her computer, e) work on it locally and then f) reupload the result to the Project Management System (but wait, there's more!) from whence g) our Doyenne of Customer Delight h) ships it to the customer (via 128-bit secured, encrypted links) who then i) downloads it/them to his/her computer. (Oh, I forget our QA, our Princess of Perfection, who has to download the files, check them, and then upload her comments to the production teams and when applicable, customers, too.)

I mean...I'd turn into a human file server. I'd need Shiva's arms. I mean, for me, instead to file serve those around, on an as-needed basis, to the crew members, in a more controlled way, rather than relying on the cloud-sharing capabilities. That described ONE project and we have 135-150 books running at any given time. Granted, they're not all 3gig. We have many that are just word processing files, 1mb tops, or the like. But the bottom line is, the fastest, easiest way to do that is some sort of Cloud-based sharing. One download by admin to the system; then sharing the file links as required, across our internal "network" of comptuers (not a real network), and it saves untold hours of up/downloading.

This is a real pisser.

Originally Posted by badgoodDeb

Hitch, you aren't Unix based, are you? (I think you aren't) Or even Mac (which is Unix at heart).

I manage Unix backups, and could talk you through a backup to disk system .... and then you post those disks to someone, as described above, for your offsite backup. Unix can do incremental backups nicely .... unless all your data looks like a single item (a Virtual PC, for instance, looks like a single file to Unix, I believe).

Badgood, I appreciate that enormously, but while we have two Mac users in admin and a crew (bookmaking) that use Macs, the "big" computer and main..."terminal" for everything, the Charing Cross Station (for NYers, Port Authority) of the business is me and that's all PC, all the time. Plus, realistically, although they've tried to do this and that to be more inclusive, KDP is really fashioned around PCs, not Macs. (Not that this is relevant, but Apple pissed me off very badly once and from that day forward, I won't give them a penny of my money. Won't even buy music through them any longer.)

We did discuss lighting the fires under our older Linux Server that we have around here some damned place, and using that for the incrementals. GMTA, as we say at my shop (GIM-tah!), or Great Minds Think Alike. :-) But, I think I'm probably going to end up with Hemmi's solution, below:

Originally Posted by Hemmi

How about a low tech solution of one of those fireproof safes to keep your backup in?

Yup and that's what we're thinking. I can double-bang the Dropbox, by MultClouding it to Amazon S3, "just in case," and we've laid in some more 6TB drives and I guess that's it. I'll need to ensure it's theft-proof too, though. I know, it sounds stupid--I mean, REALLY, who the hell wants to rip off a computer that's really nothing but full of books?--but the two hard-disk recovery concerns are fire and theft. If I can ensure that the Fireproof can't just be picked up and snagged, I don't see an alternative.

Then we can simply do a backup, and then incrementals for however long at a time, (and our backup is ALSO a RAID, by the way--can you say "paranoid person," anyone?), and slap 'em in the Fireproof.

Honestly, it kinda boggles me how I got to this point. I've kludged together this entire system, using 4 different SAAS systems (Salesforce for admin/email; Freshbooks for accounting; TeamworkPM for project management and Dropbox for "file sharing") and mostly, it works. To replace it with something more robust, something smoother and more commercial would run me so many thousands, if not tens of, that I might as well pack up my ball and bat and go home.

Sigh. Oh, well. Fireproof safes, here I come. If anyone here has any recommendations, kindly do let me know.

I appreciate all the thinking caps, folks. I was hoping that somebody had a miraculous cure, but I was already fairly sure, when I asked, that we'd exhausted the options. Bummer!

Thanks again, seriously. I genuinely appreciate your collective time. guys!

Hitch

[binaryhermit]

I don't necessarily think S3 is the best choice given that AWS tends to be $$$. I think there are providers who use the S3 API who charge way less.

[Hitch]

Originally Posted by binaryhermit

I don't necessarily think S3 is the best choice given that AWS tends to be $$$. I think there are providers who use the S3 API who charge way less.

I was simply contemplating the avaiable cloud or pseudo-cloud options available through Multcloud. The Usual Suspects are there--Dropbox, which of course, is my damn problem in the first place, GDrive, GPhotos, GSuite, OwnCloud, OneDrive, etc. and the like.

MC does, recently, say that you can link MC to an FTP and thence to BackBlaze...Hmph. I might see if I can backdoor to BackBlaze and see if I can do it that way. I've used MultCloud to painlessly xfer large numbers of videos to GDrive, for a hobby group, and that's been lovely. If I can use MC-->FTP-->Backblaze, that might solve my "just in case want a disk option" problem.

Hmph. Thanks for your comment, @binaryhermit; it made me look at something that I otherwise would not.

Hitch

[binaryhermit]

The one I was thinking of is (sponsor or former sponsor of some podcasts I listen to) Wasabi, but I've never used them and can't comment.

I think AWS is kinda like IBM was in the day AKA "Nobody ever got fired for buying IBM"

[Hitch]

Originally Posted by binaryhermit

The one I was thinking of is (sponsor or former sponsor of some podcasts I listen to) Wasabi, but I've never used them and can't comment.

I think AWS is kinda like IBM was in the day AKA "Nobody ever got fired for buying IBM"

Yes, I concur. That's probably the case. To be fair, though, whether it's me or someone else, when you're thinking about where to put the lifeblood of your business...it's hard to overlook the fact that in all likelihood, Amazon (or IBM!) is going to still be there when you need them.

When you look at businesses like, say, BackBlaze or Alfresco, etc., you have to have a fair degree of faith to think oh, sure, they'll still be there in the morning. :-)

Hitch