A recovery story
For nearly 10 years, I used tape backup and archival systems
At first introduction to using tape, it was uncomfortable. It was clunky and complicated. It felt like there had to be something better. I suggested moving to a more efficient disaster recovery (DR) system, but, at the time, all the options were far too expensive and my budget requests were denied. Eventually, I grew to accept that tape backups (and time-consuming recoveries) was just the way IT worked.
In fact, after the first year, I became dependent on the tape system as my main responsibility, my precious. Every day was consumed by the same monotony—remove, label, store, replace. My entire value to IT had been defined by my tape use and management. At first, the pitfalls of tape affected my work, but everyone seemed to painfully accept that this was the way IT worked. Then, the tape addiction began intruding on my personal life—late hours, broken plans, sleepless nights—but I reassured myself that tapes weren’t the problem. That is, until I lost my job. I hit rock bottom. I sought help. As soon as I got tape-free, my life turned around—almost immediately.
Life on Tapes
If you’re like me, responsible for managing your tape backup system, this is what a typical week looks like:
- Monday . This is the official tape drive cleaning and offsite day.You grab your favorite cup of coffee and hope that you have green lights from the backup that ran on Friday night, so all you have to do is take it out and put in the cleaner tape before beginning your day.
This is done to clear out dust and keep your tape drive hardware running at its 20% reliability rate. This isn’t very hard, you’re only away from your desk for an hour or so. You return to your desk by mid-morning to begin going through unanswered requests while checking vitals on the rest of your systems.
- Monday – Friday. Just about every day, a coworker, manager or boss calls fretting about a lost file, typically an email—the sky is falling and you’re supposed to drop-everything to recover it.Your first, unintentional, response is to let out a long, drawn-out sigh. Coworkers love this. How important is it?You spend the next 3-4 hours in the server room, recovering the file. You miss lunch and settle for some empty calories found in a candy bar you keep at your desk for such occasions. Meanwhile, other, more pressing requests are piling up at your desk.
- Beginning of Each Day. When the business day starts, it’s time to verify last night’s backup, remove and order it into your labeled tape area, then replace it with a tape next set to be overwritten. We used a tray stored in a cabinet, near the tape drives in the server room. If you’re lucky, you convince the office manager to help you out by doing the tape switch so you can respond to other requests. Hopefully, they will preserve the organization you’ve established as part of the tape routine.
- Friday. Friday is full-backup day, when you switch the tapes as usual but, this time you’re going to change the backup to run a full instead of an incremental. This also means you need to take the last full backup offsite. We stored tapes in a safety deposit box. Because Fridays are always busier, you have to leave early to beat the traffic, again, leaving important requests unanswered. Sometimes, there’s just no time in the schedule to take the tape offsite, so it sits safely in your car until Saturday.
- Occasional Saturday Night Panic. You get a distress “9-1-1” call from the boss saying that the everything is down. There’s been a hardware failure on the company’s email and SQL server. You ditch whatever personal plans you had and drive to the safety deposit box to pick-up the last full backup tape, then drive onsite to get to work on the system recovery. You arrive in an hour or two.If you dodge the near 40% failure rate of tape recoveries, and have spare hardware available, you just might be done by midday Sunday. Throughout the night you update your friends or family that you won’t be home tonight and aren’t sure when you’ll be finished. If they’re good friends, maybe they’ll come visit you with a coffee and snack in the morning to keep you company?
The glaring problem here is that the whole system blocks productivity—for IT, for coworkers, for the business. From the slow response and recovery times, to the manual offsite storage and retrieval, to the requirement to be away from your desk just to check statuses. Your system’s limitations have nothing to do with you. You’re just working with what you have.
How IT Ends
The final straw comes when that Saturday night panic call happens, you go through the motions—but there’s nothing you can do except send the tapes to an expensive and slow, physical data recovery firm.
Of course, you help users with other problems like troubleshooting systems, deploying patches, running updates, managing the network—but this single, critical system is what defines your entire value; the backups, tape backups.
When IT gets better
At some point, you realize that this lifestyle and reliance on yesterday’s technology is not smart, efficient, or soul-enriching. It shouldn’t take 4 hours to restore a file — it should take 4 seconds. At this point, you start hearing some noise about the cloud and more automated, efficient approaches to DR.
All the experts are marching in the same direction and recommending modern, DR solutions. Many suggest leveraging the cloud to make backups and DR more affordable, more reliable and easier to use.
When you first start working with this new paradigm, your life changes immediately and dramatically:
- Monday. A day like any other. You check the status of all your backup systems from your laptop. You don’t even need to be onsite. Your focus is on productive activities…
- Monday-Friday. This part doesn’t change. Every day, coworkers still request email and file recoveries. Except this time, they’re not as stressed because they know you can easily recover their file(s) in a few clicks. You don’t even need to put down your morning cup of jo’ .
- End of Each Day. Every day, your backups are replicated offsite on a schedule of your choosing. You don’t need to do anything. Instead, you focus on productive, meaningful work.
- Saturday Night Panics. Oh, these still happen. But, they don’t cause the same night sweats. You just login from your laptop, see that your backups are there, boot up your machines in less than 15 minutes or recover the necessary files to the appropriate machines. You feel good, and so does your boss. Now, go enjoy the rest of your night, Mr. Hero.
Since finding the right tool for the job, you spend more time being an asset to the company than ever before. You have more time to help those around you. You have more time to find other areas of the business that can be improved. Your “wish list” of things to get done, starts getting done. You’re breaking less personal engagements and sleeping better. Suddenly, you’re called into business meetings to talk about efficiency—new tools, new training, new productivity. Your voice is heard and respected. You are not simply a cost center. You’re happier.
Who would have thought that being dependent on tapes would have such a profound effect? I mean, everyone was doing tape, and tape was normal. IT was the way IT had always been, until IT wasn’t.
Here’s to the new normal.