Revamping Mason’s Computing Disaster Recovery Plan
July 21, 2011Print-Friendly Version
By Lea Lubag
Being able to surf the web, get directions, listen to music and check your email makes most people say they couldn’t survive a day without their smartphone. But a damaged or missing phone can easily be replaced within the same day with pay-as-you-go smartphone options available on the market.
Now imagine an entire business trying to run with all of its computers, servers and external hardware wiped out. The organization would come to a screeching halt, creating a real emergency. But is there a quick-fix in this case?
Initially a Pet Project
Some 25 years ago, James Madison University was Mason’s sister school for support in the event of a computing disaster. It was agreed that both universities would assist each other during an emergency. However, each school had different system configurations and lacked capacity to support another institution in addition to their own computing needs.
“While it sounded like a good plan, it wasn’t very realistic,” says Ron Secrest, Mason director of enterprise servers and messaging.
In 2005, Secrest first began what he calls his “pet project” of resolving the disaster recovery problem. He gathered a team of staff members and led them in developing a system that would really work in an emergency.
Secrest says the determining factor in the development of the university’s current disaster recovery plan is based on the advancements in technology.
“With the rise of virtual servers and more affordable storage area networks, it’s now possible to build and manage a university disaster recovery site at a remote campus,” Secrest says.
Prior to this, equipment manufacturers planned on delivering new servers and computing equipment within 48 hours of an emergency. However, it was difficult to test this scenario. The next option was to pay a fee to service providers to house a duplicate version of an organization’s system. But this was too expensive and difficult to keep the system updated.
Secrest took advantage of the technological opportunity at hand to initiate the building of Mason’s own remote disaster recovery site, completely revamping the university’s disaster recovery stratagem. Mason’s disaster recovery site is currently located at the Prince William Campus in Manassas.
This past January, Secrest, along with Barry Freese, senior computer systems engineer and architect, and Mark Craft, senior technology advisor of ESM, presented on Mason’s disaster recovery technical infrastructure at the Educause 2011 Mid-Atlantic Regional Conference.
An editor for EdTech magazine attended the conference and urged Secrest to write an article on disaster recovery best practices. Secrest’s article, “Better Safe than Sorry,” which details his five best practices for managing a university disaster recovery site, was published in the June/July 2011 edition of EdTech magazine.
The five best practices are:
1. Include disaster recovery funding in new IT project budgets.
2. Phased deployment is the right approach, but plan for replacement of the technical infrastructure periodically.
3. Choose your configuration carefully, with an eye to needs and costs.
4. Test your systems frequently and rigorously.
5. Use risk mitigation techniques to guard against preventable situations.
Secrest says the three most important things in disaster recovery are “testing, testing, testing.”
“You’re constantly modifying configurations, so if you don’t test, you don’t know if it’s going to work in an emergency situation,” he says. “What we developed here is a real disaster recovery plan. We know it works here because we’ve tested it.”
He adds, “Detailed disaster recovery information becomes outdated and hard to maintain. So what I recommend is to keep basic documentation and to do rigorous testing over and over again. If you had some 300-page document and you never tested it, then people would have to sit down and read it. And that’s not going to work.
“Although we’ve come a long way from 25 years ago, the process will still continue to evolve from here,” Secrest says.
“As technology changes, equipment becomes obsolete and needs to be replaced. Versions need to be updated. It’s a constant effort. The business processes of our university change, so this is a project that’s never really going to end,” he says.