Is your stuff backed up? Recovering from a hardware failure
I had a massive hardware failure over the holiday weekend. I can only tell you what I think happened. I'm not an electrical engineer so if you know more (and I'm sure you do), do feel free to share your thoughts in the comments.
For my main machine in my home office I'm running the Ultimate PC that Jeff Atwood was so kind to build for me a few years back. It's not the Ultimate anymore, but I will say it's still VERY competent. I've upgraded the memory, video card, and SSD (more on the SSD in a moment) and it's a great machine and very very fast.
The tower was connected to an APC 850VA UPS that had worked nicely for a while. I replace just the batteries in my UPS's every 18-24 months. This last week the UPS started a low "scream" so I went over to check it out. It had turned off suddenly, so the PC and all the accessories were also off. I turned it back on and there was a series of snaps and pops, another screen, followed by a non-trivial amount of smoke and a burning electrical smell.
When it was all over, the main fuse of the house had popped, the UPS was dead and smoking, and the power supply in my computer was dead and smoking. Ugh.
I headed down to the local electronics shop and bought a new 1000W power supply, a new APC BR1000G, and went to town, rebuilding my machine. After redoing all the cables and stuff...it didn't boot. I didn't even see the hard drive (SSD). The drive is a Crucial c300. I loved this drive and it worked great for like 6 years...and now it's dead. Turns out that these Crucial drives are known to die when they lose power quickly. I tried to bring it back to live using all the various forums and whitepapers about this known issue, but nope. It's dead.
OK, so have I lost data? What now? Fortunately I backup my systems. And I hope you, Dear Reader, also backup as well.
Stop reading this now and please, think about your backups. Do you have one? Have you tested to see if you can restore from your backups?
Backups always succeed. It's restores that fail. Test your backups by restoring from them.
I've got a number of backups because I practice the Backup Rule of Three.
- 3 copies of anything you care about - Two isn't enough if it's important.
- 2 different formats - Example: Dropbox+DVDs or Hard Drive+Memory Stick or CD+Crash Plan, or more
- 1 off-site backup - If the house burns down, how will you get your memories back?
Here's what my backup situation is/was and how I restored.
While you can use Imaging Software and restore an entire image of Windows or Mac, I find that reinstalling Windows takes less than an hour. I keep a bootable USB key of Windows 10 around. You can also download an ISO and make a USB key quickly. You don't usually need an activation key if you're reinstalling Windows. In my case, I installed the new drive, booted off the USB, signed into Windows with my Live ID (Microsoft Account) and it picked up my Windows license already.
Windows File History
I have a 4TB external drive on my desk that uses Windows 10 File History. This is like the Mac Time Machine feature. It's one of the best little "hidden" features of Windows 10 and everyone should use it. It's actually been around for years. My Documents, Desktop, and any other folders I want are automatically backed up as often as I want. I have a backup going every 30 min and I never think about it. It just works, and I don't notice any performance issues.
In this case, I *did* have crap on my desktop that wasn't in Dropbox and wasn't yet backed up to the cloud. I just hooked up the drive and restored from File History. I literally lost nothing. All my desktop crap was restored in place. If you have an external drive that you always have hooked up but it's not really getting use, setup File History in just minutes.
Multiple Cloud-based Backups
I have a number of clouds in my backup rotation:
- GitHub - I have github repos, both private and public for code.
- DropBox - My primary cloud files backup
- OneDrive for Business - My work cloud files backup
- Synology - I love my Synology. It's a complete home NAS Server with massive storage, RAID, VPN, Docker, and so much more. A daily joy and a local cloud.
- CrashPlan - I keep TBs up there and pay them happily for the service.
Related Links
Here's some additional reading on ways to back up your system. Please do also help non-technical relative back up their stuff as well. Every week I hear about someone working on their PhD thesis losing their whole life's work an instant. Backup is a system and it CAN be automatic.
- The Computer Backup Rule of Three
- A basic non-cloud-based personal backup strategy
- On Losing Data and a Family Backup Strategy
What do you do for backup?
Sponsor: Many thanks to my friends at Stackify for sponsoring the feed this week! it’s what being a developer is all about so do it the best you can. That’s why Stackify built Prefix. No .NET profiler is easier or more powerful. You’re 2 clicks and $0 away, so build on! Prefix.io.
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
David Parvin
I've been looking into the Backblaze solution that now has a built in interface for the synology, but it is more of a sync than a true back up. Way easier to setup and uses way less resources on the synology.
In any case, my backup strategy is to backup to a local hard disk, to keep my important files (nearly all my files!) on Dropbox and to use Backblaze for formal off site continuous backup of my disk.
Thanks
jesse
I have another Synology stored in my garage (about 20 meters from my house)
Syno1 backs up to Syno2 with smart recycle (1 per hour for a day, 1 per day for a month, older than a month it will keep a weekly version)
Both devices run on BTRFS so I am able to create a snapshot at filesystem level as well.
This is scheduled once a day and is really fast.
This should reduce the risk for a cryptolocker to lock your files (and backups!)
This way it's already pretty safe. The backup Synology also backs up irreplaceable files, like photos, to another Synology of a friend.
Buy a bigger drive for the mirror and you'll have something nicer to migrate too after failure as well.
2) Time machine to a local drive
3) Dropbox - in theory Backblaze should take care of it - but by using Dropbox as well I get the really important stuff sync'd across my desktop/laptop and online
It is cheap to 3 bucks a month for 100 GB.
I currently have a free Office 365 sub so I'm paying nothing for 1TB one drive and getting Office too. I was paying £1 a month for 50GB on top of my grandfathered free space giving me a total of 90GB previously.
I intended to re-sub to the family plan at the end of that since even with just me and my hubby using it, split the bill 50-50 that's only £3.33 a month for 1TB + Office on 3 devices each, that's a no brainer.
I'd add in the list syncthing (with its companion synctrayzor). I've a low cost computer running in my office (with a simple ubuntu server on it), that allows me to remotely clone my important folders.
The advantage of syncthing is that it's 100% open source and does not rely on 3rd party backup tool or undocumented format.
hth
I'm really disappointed about File History, because of three main problems:
1) I've added my sencond partition D: to the backup - and the configuration suddenly removed my partition - I'm checking this from time to time, but it's not really acceptable
So File History just makes sence, if you have all your files in your users dirs.
2) File History is NOT telling you, if something goes wrong - it just writes a log in the eventlog, but is that ok for a not techy user?!
3) File History is not using VSS, so there are some files that don't get backuped
For me the best built-in file based backup tool for Windows 7 and 10(!) is "Backup and Restore (Windows 7)". Not to be confused with Windows Image Backup!
"Backup and Restore (Windows 7)" is a file based backup tool. This is the backup tool known from Win7. MS had removed it in Win8 and silently reintroduced it with Win10. You find it via searching the control panel for "backup".
You can configure it to backup any files on your machine. I contrast to File History it does support EFS encrypted files. You can schedule it to run at night and it can reliably wake your computer to execute. It does not have the nicest UI but in my experience it is way more robust than all the commercial tools (especially Acronis) I tried.
My 'master' backup is a 4TB drive in my home server. My desktop is backed up there, as is anything else personal. I have three 4TB external drives (LUKS encrypted) that I swap every week or two between home, office, and parent's house (less often, 150 miles away).
For work, I have a 1TB laptop drive in a USB3 enclosure (Bitlocker encrypted) that has File History going to it. The fear of the unreliable SSD made me do it.
Mr. Hanselman: As of right now, your story finally made me figure out how to use BTRFS scrub, so I'm scrubbing said 'master' backup, and watching it closely.
sudo btrfs scrub start partition
sudo btrfs scrub status partition
Another wrinkle to consider is mobile devices - I'll use dropbox as a central source of things accessible on my phone, and Google Photos works I think enough as a backup of photos/videos. Photos sent via text (Textra!) are auto saved to a folder that is also picked up by Google Photos. And I'm relying on the Android/Google restore system to catch main things and figure I'd deal with anything missing by rebuilding. But there's no real cohesive backup plan of sorts that I'm running on my phone. Nevermind that my wife is on iOS so I'm assuming that is relying on the iTunes backup that may or may not be active.
I still use the 2 original Intel X25s I got when they were first released. I have killed exactly one SSD (I have a dozen) and more than a few spinning rust platters in the last decade so don't be frightened of your silent partner. Don't be complacent either, they give you no warnings, no "click of death".
File hierarchy. Does each backup location have the same file hierarchy? How do you organise what gets backed up where? Is there one location that's the granddaddy of all others, that has a copy of absolutely everything?
De-duplication and discovery. Inevitably you copy files around. How do you find a file that you know you had written, but didn't store in the right place when you did or moved it afterwards? Do you rely on the backup stores automatically de-duplicating for you? Do you ever do spring cleaning?
Versioning. Windows File History provides a time versioned store. What happens when you sync this with a flat file backup that's not time versioned, e.g., the Synology? Do these locations have disjoint contents? Do you ever have sync problems, where a newer version of a file is erroneously automatically overwritten by an older file from another backup source?
E-mail. Do your backups contain copies of (Outlook) .PST files? How do you reconcile them to maintain a complete e-mail history over time, e.g., you may have deleted some e-mails from newer .PST files, so you have to keep the older .PST files for those e-mails, but that means duplication as well.
...
I think we've passed the point where the term 'backup' could have been rescued and kept to mean 'Can go back to any point in time', so perhaps it should be best left meaning 'copy, intended for archival purposes but no guarantees'.
Can we invent a new term for keeping copies of data with the ability to go back to any point in time?
"We can't leave for the plane just yet. I'm bunkering my repositories"
"Did you remember to add the incriminating photos to the datardis?"
"Is it saved securely?" ... "Yes, bastioned!" ... "That wasn't called for!"
I suppose you could keep the vm directory in the list of directories backed up but what does an incremental backup of a vm look like? I also am wondering how you deal with multiple vm's on a single machine? Do you have an SDD for each vm?
My vm's range from 60GB to 200GB. What am I missing for backing up vm's or do you have an external drive dedicated to each vm?
But how do you handle the files? Are they in the server just as back up? Do you still have all of them in your main PC?
My laptops drive is getting full and pics and videos are to blame for. Should I just send them all to the Synology and keep the relevant ones? But then, I'd need something to sync them right? Like only keep last month's or last year's pics in the laptop.
Comments are closed.