Member Spotlight

Finding a Way: How Creative Collaboration Produced a Disaster Recovery Solution

What happens when two innovative IT leaders connect, exchange ideas, and envision the full possibilities of PennREN, KINBER’s high performance network?

A cost-saving collaboration that produces a disaster recovery solution that meets the needs of two institutions of higher learning and the students and faculty they serve.

In this instance, the leaders and institutions are Carrie Rampp, associate vice president and CIO, Information Technology Services, Franklin & Marshall (F&M) College and Mark Huber, CIO at Susquehanna University.

Disaster Recovery

According to Rampp, F&M was paying $40,000 annually for a vendor disaster recovery service that also required a dedicated link up to the site in Reading, PA, which replicated data in near-real time with all campus systems.

Susquehanna handled its disaster recovery in a different manner. The university had an on-campus colocation site so if an incident occurred and they lost data in one data center the university could continue to operate from the other. Susquehanna’s offsite solution was to backup to tape and store those tapes with nearby Bucknell University.

As Susquehanna was planning for the future, getting out of the tape business was a major goal. The university wanted to have an off-site disaster recovery capability and took steps to budget for a commercial provider. They also joined KINBER to access PennREN, desiring to leverage its high-performance member exchange capacity.

“When Susquehanna joined KINBER, it was one of the first off-net campuses to connect via a local broadband provider for the last mile,” Huber said.

The Creative Solution

The two institutions of higher education had already established a collaborative relationship when they hired a shared consortium Chief Information Security Officer (CISO), and becoming participants in KINBER’s programming provided additional common ground. As F&M and Susquehanna worked together to hire a replacement CISO last year the discussion of exploring additional collaborations evolved with the realization that both organizations could leverage KINBER’s on-net capability and KINBER member exchange capacity to handle disaster recovery at each other’s locations.

As it considered its disaster planning, Susquehanna approached F&M about collaboration.

“We felt like we had a solid plan,” said Huber. Both organizations executed a formal MOU that covers the basic topics including expectations as well as making sure both organizations had the right insurance coverage. Each is responsible for its own encryption site to site and they do not share resources except the routers.

“All contracts must have a certain amount of trust with partners,” said Rampp.
“This was not the first time the teams have worked together, as they built a relationship over years from working with the joint CISO on campus. It helps because we know where they are and how they live. We have been to each other’s campuses, which was helpful for laying the ground work for this project.”

Because F&M’s campus tech architecture is very different than Susquehanna’s mousetrap, how they deployed solutions were completely different. Under the collaboration the F&M architecture was about the same as the colocation center. Because Susquehanna was moving from tape to disc, the team had to test and build up an enterprise data vaulting solution to support the off-site archiving.

Huber noted that the data protection design provides real time backup across Susquehanna’s campus locations and in case of total disaster Susquehanna has something to go back to with the vaulted data stored at F&M.

For disaster recovery planning, the rule of thumb is to plan for at least 50 miles between the primary and backup sites. For F&M it had been just under 50 miles to Reading. Now the backup is at Susquehanna University, 80 miles away from F&M’s Lancaster campus.

Benefits of Collaborative Partnerships

“Throughout the process we had minor tech challenges to work through, for example, after we initially went live we accidentally hogged up most of Susquehanna’s bandwidth. So some immediate fine tuning was required,” said Rampp. She noted the team continued to monitor the traffic to make sure they were not using all bandwidth.

“We were able to work it out, and even with cost of equipment we came out ahead in year one,” she said.

Another benefit of the collaboration is that there are “boots on the ground” at the recovery site, as each staff has authorized access to the recovery areas and can enter and help each other if needed.

Many organizations talk about collaborating, but these two organizations made it happen.

“The nature of our relationship plus KINBER having the structure made it possible for us to do this collaboration, which is a point of pride for all of us,” Huber said.