Cisco on Cisco

Cisco Data Center Lays Foundation for Greater Agility and Resiliency

Standards and governance are central to getting the best return on IT investment in your new service-oriented data center.

A greenfield opportunity... If you could create whatever you wanted in a data center, what would you build? Here’s what Cisco IT envisioned: an entirely new business data center, one that bolsters Cisco’s ability to move into new markets, can easily support US$8 billion in business revenue generated quarterly, and that increases recoverability of the data center in a natural or other disaster.

This new Cisco data center is now under construction in Richardson, Texas, and will support all North American operations. It will be Cisco’s first fully operational data center that adopts the Cisco IT service-oriented data center model and test runs the impact such a model can have on its business (see article, "Roadmap for a Cisco IT Service-Oriented Data Center").

To make the service-oriented data center model work, Cisco IT has focused considerable effort on operations, specifically governance-the policies that ensure compatibility and security of applications as well as their ability to minimize use of resources. The focus on governance arose in part from a realization that Cisco needed to align its IT operations to 21st century demands and to improve them overall, says Brett Colbert, senior director of IT at Cisco. Basic blocking and tackling in fundamental areas such as incidents, change releases, and configuration management drew early attention, as the IT group developed policies and standards to which applications must adhere.

One of the first steps the group took was to ask existing production data center users and partners what single thing they would change about operations if they could. From their responses, the IT group developed a list of 40 proposed policies and submitted them to Cisco’s Enterprise Architecture Council, which has so far approved more than 30 of them.

One of these policies, for example, states that all applications must pass a security vulnerability assessment to be served by the new data center. Another requires all databases to run on a platform supported by the data center. And another policy states that an application must be able to run in a service-oriented environment to migrate to the new data center or provide a compelling business reason why it should be maintained on a standalone server.

Resiliency also factors into the policies, even down to the application level. To be served by the new data center, every application must have a disaster recovery plan stating, at a minimum, who should be called if data is corrupted, how the data will be restored, and how long the application can be down without affecting customers or other critical business processes.

Simplicity = Flexibility

“We’re forcing simplification,” Colbert says. “We don’t want ten different applications running on ten different architectures if they can run on the same one. The simpler the architecture, the more structure imposed, the faster IT can respond to the needs of the business.”

This heavy emphasis on structure poses a challenge for Colbert and his team. “It isn’t necessarily obvious to our IT clients that more structure will make things better,” he says. “Many of them hear ‘standards’ and ‘regulations’ and think ‘bureaucracy’ and ‘slowness.’”

So, the IT staff has worked diligently to educate clients about the advantages of a service-oriented data center environment, identifying risks and ways to mitigate them in the process. Several clients have experienced benefits through early-stage service-oriented architecture operations at Cisco’s production data centers in San Jose, California, and Research Triangle Park, North Carolina.

“Many of our IT clients have already seen a significant drop in high-priority trouble calls, the ones that get them out of bed at night or interrupt their weekends,” Colbert says. “We’re starting to get a fair amount of the emotional buy-in that’s crucial to success in the new data center.”

At the same time, Cisco IT is standardizing its own activities. People in IT operations don’t necessarily speak the same language across the company, notes Colbert, leading to misunderstanding and confusion. For example, he says, “People use ‘incident,’ ‘problem,’ and ‘issue’ to describe the same thing. Some will do a root cause analysis; others won’t. Some will document fixes; others don’t. We’re setting uniform policies for our own operations as well.”

A Peek Inside the Richardson Data Center

Cisco’s new data center in Richardson, Texas, will contain 29,000 square feet of raised floor space, divided into four “halls” that support the business needs of not only Cisco IT, but also of Linksys, Scientific Atlanta, and Cisco’s government services group. Halls will be divided into pods, each containing all the processing, storage, and networking resources needed to support a number of applications. Resources will be virtualized and dynamically reassigned as needed. These are among the essential ingredients for Cisco’s service-oriented data center (see Table 1).

Table 1. Evolution toward a Cisco IT Service-Oriented Data Center
Today Future
Heterogeneous Environment Standards-Based Environment
Rigid User Environment Flexible User Environment
Application-Specific Security Self-Defending Data Center
Multiservices Network Intelligent Network Services
SAN and NAS Storage Network Virtualized Storage
Dedicated Compute Resources Dynamic Compute Resources
Application-Specific Management Policy-Based Management

Richardson was chosen after an extensive search. One of the key site criterions Richardson satisfies: It is relatively free from potential natural disasters such as earthquakes, unlike the sites on which approximately 70 percent of Cisco’s present production data center capacity reside.

“At a minimum, the data center will function at Tier III availability, up from Tier II. This is the terminology used by the Uptime Institute, an industry organization, to describe the redundancy and recoverability of a data center,” Colbert explains. “You can go up to a Tier IV.” The new data center will be served by two communications carriers and two power suppliers.

Generally, Tier III data centers demonstrate 99.98 percent availability. The new data center will be served by two communications carriers and two power suppliers.

Cisco IT supports just under 10,000 servers, roughly 3,800 applications, and about 800 databases. So far, more than 1,000 of those applications have been evaluated and approved to migrate to the Richardson facility, which is expected to employ approximately 4,500 servers.

“You don’t need more server and storage capacity to do more work in our service-oriented data center model. Because of virtualization, you can get a great deal more work out of what you have,” emphasizes Todd Glenn, IT manager at Cisco.

In addition, the Richardson data center will be as “green” as Cisco can make it. This is an especially high-priority goal given that 80 percent of data center challenges are related to power, cooling, or space, according to Gartner estimates. For more on green efforts in the new data center, see article, "The 21st Century Data Center".

Says Glenn, “We’re building a new rack design, for example, that is hooded so heat is funneled and exhausted from the room. We alternate racks of equipment so we have hot and cool rows for the most efficient air flow, and fans are located to get the maximum benefit from cold air.” Power consumption in the Richardson data center will total 5 megawatts.

The data center will make use of a number of Cisco tools for application load balancing and server and storage management. Applications and databases themselves will also be virtualized whenever possible. For example, customer relationship management (CRM) and order fulfillment software can share the same database of customer information.

Based on the latest construction schedules, Cisco’s Workplace Resources Group expects to turn the physical plant over to IT in August 2007. The next step, which should stretch through the end of 2007, is “Wave 0,” in which all of the processing, storage, network, infrastructure services, and middleware needed are moved into the center and proved in. At the beginning of 2008, “Wave 1” will see the first movement of applications into the new data center, followed by successive waves over the next few years.

Return on Investment

The biggest return on investment (ROI) in the Richardson data center will come from its ability to adapt to Cisco’s changing business needs.

“IT systems can be a significant barrier to a company’s ability to move into new markets,” according to Colbert. “If an application and its supporting resources were designed to handle enterprise sales, they may not support the company’s move into consumer or small business markets. We want our IT systems to enable every move the business wants to make.”

For example, consumers purchasing online don’t want to struggle with page after page of configuration options when they are looking for a simple home router. So, the software application should be capable of hiding the configuration pages intended for enterprise purchasers.

“A big reason so many companies are looking at service-oriented architectures is to better support their business strategies. They need a flexible infrastructure and services that can be easily reconfigured when they want to go after a new type of business,” Colbert says.

Specifically, they want an infrastructure and services that will support applications that consist of loosely coupled modules, rather than monolithic ones. This is among the new business capabilities Cisco will derive from the Richardson data center—in effect, the ability to mix and match modules, perhaps even from different applications, to suit the customer, the product, and the market segment.

Cisco IT will try out such a loosely coupled application architecture in a new e-commerce platform at the Richardson data center, in which ordering, tracking, pricing, discounting, and related applications will be designed within modules that can be pulled up to satisfy specific customer needs. “We want to make it much easier for people to buy from us,” says Colbert.

Cisco IT is examining specific ROI and performance metrics in new ways, as well. For example, can operational metrics, such as system availability, problem resolution within guaranteed service-level agreements (SLAs), and the usage percentages for servers and storage networks, be tied to business agility and resiliency goals? This is the type of business-benefits metric the IT group wants to cultivate.

“This new data center is a transformational opportunity,” notes Colbert. “We will build in and demonstrate architectural and operational excellence, and we will prove how much IT affects—and can support—business goals in the 21st century.”