When disaster strikes...

[Quoth]

What's a "major" file type and what stupid program was that so everyone can avoid it?

[Mel]

I've long thrown that drive in the trash so I can't tell you the brand. Pretty much the included backup software only picked up the very standard file types such as .jpeg, .doc, .pdf, etc. that you'd find in a typical office environment. Because it picked up the cover graphics Calibre looked normal until I tried to open a book.

[JSWolf]

Disaster is when your Kobo Libra 2 breaks and you cannot get another and are stuck trying to find a decent Reader but can't because Kobo discontinued it and did not come out with an updated version. Libra Colour is not a Libra 2 replacement.

[JSWolf]

Originally Posted by Mel

My worst backup disaster was restoring my calibre library after loss of a laptop and finding that the backup drive I had been using has been designed to only backup major file types. So the library initially looked fine, because the cover graphics has been saved along with the few pdfs. But the majority were empty. Luckily the previous backup drive was still on the shelf with 75% of the library so only the newer items had to be tracked down. I still curse whoever designed that thing when I think about it.

Why would the drive be at fault? To me, it sounds like it's the fault of the backup software.

[theducks]

I suspect they fell for a '1-click backup' app (better than NO backup, but only slightly).
Freefilesync will back up any file, but you do need to set up which folders to do, which is why the 1-click type exists.

[JSWolf]

Originally Posted by theducks

I suspect they fell for a '1-click backup' app (better than NO backup, but only slightly).
Freefilesync will back up any file, but you do need to set up which folders to do, which is why the 1-click type exists.

A backup app should (IMHO) be able to compress the files being backed up if they do compress to be smaller. Then you get to use less disk space for the backup.

[haertig]

Critical to me is a backup plan/app that does not use proprietary formats. Because backup apps come and go all the time. And your compressed/encrypted/proprietary backup may not be restorable when you need it. Because they stopped updating their software around Windows 7 and it won't run on newer OS'es and the company has long since gone out of business. I also don't like that many backup apps try to be smart and not backup "uneeded stuff". Like, they'll skip all temporary files. Which to THEM, might mean anything that ends in .tmp. But for ME, because a file ends in .tmp does not mean that I don't want to back it up. Backup software making assumptions like this is not good IMHO.

Also critical to me is to not have just an "image" backup. Where you have to restore the whole thing. When you only want to restore a single file or two. Images are OK when you're doing a bare metal restore because your computer caught on fire. The problem there is that you buy a new computer, do the restore, but Windows won't run because it now sees it's on different hardware and assumes you pirated it. So while I do image backups of my wife's Windows computers, I also do file backups separately. On my Linux computers, this is not an issue. But despite images working fine on new hardware under Linux, I typically only do file backups. Because it's quicker for me to reinstall Linux from scratch than it is to restore an image of a pre-existing installation.

[JSWolf]

What is a good backup program for Windows that will compress files only if they will reduce in size?

[Quoth]

My OS is on SSD and the HDD is partitioned. Small partition mounted /var and big one /home.

I use rsync for user files daily to my server outside the house (over 1 Gbps ethernet!). I use Linux Timeshift to backup system to a 3.5" SATA in a USB 3.0 case kept in a small flight case near the door (was made for CDs). I've tested restore to a different PC and laptop.

[Quoth]

I had terrible problems with Windows backups for decades. The total path can exceed what the backup SW can do, or even Explorer copy.

Also tape drives that claimed success and were later unreadable. Customers hated being told that every tape backup had to be verified, or it might not be a backup.
Zip drives, CDs and DVDs were all rubbish for backups.

USB sticks / SD cards can fade in a drawer, a regular HDD won't. An always connected drive, RAID 5, Mirror or Cluster isn't a backup, but high availability.

Amazon download filenames can have ":" in them as often they are the web page title. This is totally illegal in Windows, but "works" on Linux, till you try to copy via SMB (even to linux server). So now on EVERY download from ANYWHERE I check/edit the filename and usually shorten it if long.

[JSWolf]

Originally Posted by Quoth

I had terrible problems with Windows backups for decades. The total path can exceed what the backup SW can do, or even Explorer copy.

Also tape drives that claimed success and were later unreadable. Customers hated being told that every tape backup had to be verified, or it might not be a backup.
Zip drives, CDs and DVDs were all rubbish for backups.

USB sticks / SD cards can fade in a drawer, a regular HDD won't. An always connected drive, RAID 5, Mirror or Cluster isn't a backup, but high availability.

Amazon download filenames can have ":" in them as often they are the web page title. This is totally illegal in Windows, but "works" on Linux, till you try to copy via SMB (even to linux server). So now on EVERY download from ANYWHERE I check/edit the filename and usually shorten it if long.

Back in the old days, I did use a tape backup. It worked well. Now it would be too small (need way too many tapes).

There has to be a good backup program for Windows besides what comes with Windows.

[PeterT]

I was very happy with Macrium Reflection and will probably reinstall it when I get a new PC and NAS

[rkomar]

Originally Posted by JSWolf

What is a good backup program for Windows that will compress files only if they will reduce in size?

I'm not sure that an application can know the answer to whether it will reduce or not without going ahead, compressing a file and comparing the sizes. Are there existing programs that do it another way?

[haertig]

Originally Posted by Quoth

USB sticks / SD cards can fade in a drawer, a regular HDD won't.

As I understand it, SSD's will fade too. I haven't researched this deeply, and I don't know if simply keeping them powered up or powering them up for XXX time every YYY interval will do it. Do you actually have to re-write every bit that you want to persist to "refresh" it?

Many systems these days come only with an SSD - no HDD installed. It's easy enough to follow a "power up" protocol if this indeed will keep all the bits refreshed. But if you actually have to read, then re-write each bit to refresh it ... that's a whole different ballgame. That eBook you saved to your SSD years ago but never got around to reading it - will it still be 100% intact in five years? Hmmm, that's kind of scary thinking about that if simply keeping your computer powered up is not sufficient to refresh things.

For example, my main Calibre database is on my desktop computer. That is total SSD. I've got tons of books that were written to the SSD years ago and haven't been touched since. And this could be a big problem for many people. I do backup my desktop every 24 hours to a remote computer ("remote" in the sense that's it's a separate computer, but it's also located in my house, so not remote from my immediate property). And that backup is written to an HDD, not an SSD. In addition, every 24 hours I copy my Calibre library over to another server for web presentation, and that copy is also to an HDD. So even if the main Calibre database on my desktop dies due to its SSD not being totally refreshed simply by being powered on, I have a reliable restore path from those other HDD copies.

However, I do have a different SSD-only computer that has been powered down for maybe 1-1/2 years now. When it is powered up, backups to a different servers HDD occur automatically. But the sorry fact is that I've left it powered down for 1-1/2 years. It's SSD could be dead or dying AFAIK. Luckily nothing has changed on it during those 1-1/2 years (duh - because it's powered down!) so the last backup it ran before it's extended sleep is still an up to date backup (it's not one backup either - it does both image backup and a file-based backup, since it's a Windows computer).

Still, it would be good to know FOR SURE if simply keeping an SSD powered up is sufficient to keep all bits refreshed. I'm already starting to rethink my idea stated a few posts back of keeping my "write once" stuff copied to thumbdrives stored in a Safe Deposit Box. Maybe that idea should be updated to store that data on an HDD rather than flash memory. True, this "write once" data of mine is movies, books, music, etc. so not the end of the world if it is lost. But if you're going to do something, might as well do it right in the first place. I'm glad @Quoth brought this flash memory degradation point up, even if just in passing. I had apparently forgotten/ignored it when I was speaking above. My apologies for giving bad or questionable advice/suggestions about thumbdrive/SD card use.

[haertig]

Originally Posted by rkomar

I'm not sure that an application can know the answer to whether it will reduce or not without going ahead, compressing a file and comparing the sizes. Are there existing programs that do it another way?

It would be darn near impossible to predict if a file will shrink when compressed or not. It's much easier to predict that a file WON'T shrink. Files that are already compressed generally won't compress any/much more on a second attempt, and many will actually increase in size. So one might be able to predict that a ZIP file, a PNG file, etc. will not benefit from additional compression attempts (much, if any). You could look at the contents of a file and bounce that against the planned compression algorithm and maybe predict shrinkage or not - but that's pretty much the same thing as just doing the compression in the first place and then looking at the results. Some applications compress their files as a normal course of business. That's just what the developers programmed them to do. An outside separate compression program is not going to know that, and will not be able to predict that those application's files shouldn't be touched.