Marriott International Goes Underground to Mitigate Risk

By Edward Goldman, Vice President, Information Resources Technology and Operations Planning, Marriott International, Inc.

For six months now, Marriott International has been operating a disaster recovery facility that is energy-efficient and cost-effective. The facility not only reduces the overall risk of our business recovery position, but also serves a second data center and is aligned with our corporate philosophy of environmental awareness. More importantly, it puts control of mitigating risk into our own hands.

It also happens to be located 220 feet underground.

The 10,000 square foot data center, which primarily houses Marriott's development and test environments, is:

  • Part of a 145-acre "underground city" leased to the company by Iron Mountain.
  • Naturally cool (consistently 55 degrees).
  • Housed in a former limestone mine.
  • Meeting all of Marriott's criteria for a disaster recovery facility and provides the added benefit of also being a second data center.

Faster, More Efficient Recovery Process

In our former contracts we tested twice a year and we had to test it all at once; this required that we build our environment from the ground up. In a recovery situation we would be in the same position of building from the ground up. Today, because of our relationship with Iron Mountain, we have an entirely new data center and all the benefits that go with it.

Used for day-to-day development and quality assurance (QA) operations, this data center can be quickly converted, in the event of a disaster, from the development environment into the production environment. This step alone has cut the time to begin recovery from more than 24 hours to 4 hours, and has significantly improved our time to fully recover.

Being in complete control of our operations has also provided us with the ability to recover any subscribed system on an as-needed basis. We've also built out the computing infrastructure so that it is identical to our primary facility, eliminating equipment or software learning curves. In our former scenario the equipment that would have been provided may not have been in the same configuration as our primary data center. In the event of a disaster we could have been confronted with significant learning curves during a very critical time.

Virtualization Benefits

We are also using virtualization extensively, which plays a big role in our ability to meet growing business demands. Through virtualization we are able to keep systems and data at both data centers up to date. And, because anything that is on a virtual server is easier to bring up in another location, we significantly decrease the time that an application is unavailable. However, we still have critical nonvirtualized systems that we will need to recover traditionally.

Being heavily virtualized in the primary data center has allowed us to easily extend virtualization to the new site. Eventually, we hope to be able to have both sites fully virtualized so we can easily move applications from one site to another.

Better Testing Environment

Because the secondary site is connected into our existing environment it gives us control over application recovery response time, as well as how we manage our tests. We are no longer subjected to the rigid time frames imposed in our third-party agreements. Another big advantage is that we can test individual systems or applications on our schedule, saving time and resources.

We're also able to test throughout the year when it makes sense for our application portfolio. With the outsourced solution, we could only test two to four applications two to three times a year. Today, in most cases, we schedule a weekend and shut down the application at one site and bring it up at another at no additional cost to Marriott.

Proving ROI

The business case for the data center was completed several years ago. But even in this economy it would not be a tough sell. Risk mitigation is very important for most companies, and if you can lower the risk and prove a strong ROI then you have a very strong business case. Sometimes in a tough economy you have to spend money that will have longer-term value for the company.

A multiyear cost analysis of operating at Iron Mountain's underground facility determined that it would be cost-neutral compared to our previous disaster recovery setup, even before considering more applications to be added to our current business recovery position. Each additional application only makes our business case stronger.

On the whole, this secondary site significantly ups the value proposition for Marriott. We have increased computing diversity, can respond to global growth more quickly and efficiently, and have lowered risk vulnerability.

Our analysis and planning took two years from conception to business sign-off. We went live in August of last year, but still had the backup capabilities from our third-party vendor. We were officially on our own January 2009.