Cisco on Cisco

Roadmap for a Cisco IT Service-Oriented Data Center


Next-generation data center offers service-level agreements instead of hardware, and IT clients get the resources their applications and business operations need.

Service delivery must be simple—unencumbered by different architectures, types of systems, protocols, operating systems, policies, and boundaries. In Cisco IT’s service-oriented data center environment, processing power, storage, and communications can be drawn from one big pool of resources only when needed. Hundreds or thousands of applications, handling business processes for an entire enterprise, run from this central data center.

The Cisco IT service-oriented data center model is a huge leap away from a data center environment in which every application or department has its own dedicated group of servers, dedicated storage, and dedicated linkages, often comprising different hardware and software. In this environment, systems are usually labor intensive, and, as in Cisco’s case, one system administrator may be needed for every 80 servers or so. Servers sit in silos, chronically underused. Provisioning a new application can take months, as data center personnel identify, perhaps purchase, and configure available servers and storage, set policies, test the setup, and finally pronounce it ready to use.

Not so in the service-oriented data center model. Resources that previously were scattered and dedicated are housed in a single location. Data center administrators have the ability to orchestrate all of the resources required by an application automatically, so the application can be provisioned in a day or less. Resources are managed for optimal levels, reducing costs and ensuring guaranteed levels of availability.

In Cisco IT’s service-oriented design, pods are established for manageability within the data center, each with a certain amount of server, storage, and communications capacity. When a request comes in, the data center “dynamically constructs a virtual pod to serve the application,” explains Sidney Morgan, manager of IT engineering systems at Cisco. “It follows the just-in-time model.”

Cisco will soon have the opportunity to reap the full benefits of the service-oriented data center architecture when it completes the final stages of building a new data center in Richardson, Texas (see article, "Cisco Data Center Lays Foundation for Greater Business Agility and Resiliency”). Meanwhile, Cisco has been operating several of its production data centers as “service-oriented data centers in the making," says Morgan.

Cisco IT’s roadmap for deploying a service-oriented data center consists of four phases that typically will overlap—consolidation, virtualization, integration, and automation. In the consolidation phase, the focus is on standardization of compute resources and networking of shared storage resources. This includes combining storage resources into single storage area networks (SANs) using virtual SANs (VSANs).

Cisco’s goal with the virtualization phase is to extract the application from the underlying server, storage, and network infrastructure so that resources can be dynamically partitioned, provisioned, and assigned to different applications with ease. Virtual machines, network-hosted storage virtualization, and virtualized network service technologies are key ingredients in this phase.

In the integration phase, common shared infrastructure services are deployed to address needs across all server platforms and storage systems. Where it makes sense, these services should be integrated into the network. For example, network-hosted data replication, global file sharing, and data security can serve all networked server and storage resources transparently and reliably.

Finally, automation of many of the repetitive, time-consuming operational tasks is the ultimate goal for most IT organizations. In many cases, automation will allow the infrastructure to rapidly self-correct to changing application demands, disruptions, and security threats without heavy manual intervention. All aspects of the infrastructure, including virtualized server, storage, and network resources, integrated services, and management systems, need to be architected to achieve the goal of automation.

At its production data centers in San Jose, California, and Research Triangle Park, North Carolina, Cisco has already embarked on consolidation and virtualization. The new data center in Richardson will include integration and automation.

Executive Buy-in Crucial

Long before an organization is ready to open the doors at its new service-oriented data center, executive buy-in is mandatory. This crucial preliminary step includes convincing upper-level managers that the operations for which they are responsible will indeed get the resources they need in the new data center. After all, the bottom line for these executives is their business unit’s performance, and they are accustomed to having resources under their own control. Now they can be assured with a promise—a service-level agreement, or SLA—rather than actual hardware.

“No one believed two years ago that what we envisioned would be possible,” says Morgan. “Lots of folks didn’t believe virtualization would give them acceptable access to what they needed.” So, Cisco IT designed four levels of SLAs: platinum, gold, silver, and bronze, which differ in the amount of redundancy built into the virtual data center, the time to repair, the length of data retention, and other factors. “This approach guarantees what the various business units need, and it increases IT’s return on investment by reducing the amount of capacity and redundancy that we actually need,” Morgan says.

Another important task: mapping all the interdependencies among applications and databases, and all the databases and other applications they interface with. This is a complex and sometimes brutal process, notes Morgan, but not doing it will doom the implementation of a successful service-oriented data center model.

Also crucial is setting policies for applications that will populate the data center, a set of criteria that should be adhered to. Applications must be able to run from a centralized data center with virtualized resources and use certain common interfaces. With the policies set, only those applications that meet the standards or those that can be reengineered to fit should be selected.

Of the roughly 3,800 applications used across Cisco, the company has thus far identified more than 1,000 that have moved, or will move, to the new service-oriented data center in Richardson.

After these tasks are done, an enterprise can consolidate and virtualize its data center resources and move applications onto them, as Cisco has begun in San Jose and Research Triangle Park. An example of one of these virtualized applications, says Morgan, is the Cisco Networking Academy Program, a global e-learning curricula that provides students with Internet technology skills. The program is delivered through web-based content and online instructor-led training and hands-on lab exercises.

Cisco has consolidated isolated SANs into VSANs and is now integrating these with network-attached storage (NAS), so that users have access to both block-level (from SANs) and file-level (from NAS) storage within a consolidated shared storage pool. Cisco is also well down the road to integrating its server capacity into processing pools.

Full Automation Is the Grail

Consolidation and virtualization get closer to their full potential when all resources are integrated into one unified network fabric, drawing on unified network services. In the case of a vital service such as security, Morgan envisions the service-oriented data center as servers, storage, and management software completely encircled and pervaded by security that is moderated by an intelligent network.

Integration also relies on technologies such as Wide Area Application Sharing (WAAS), Application-Oriented Networking (AON), content switching and products such as the Cisco Server Fabric Switch and Infiniband. The latter two create a network fabric that aggregates input and output among systems within the service-oriented data center.

Integration gives IT staff a “golden image”—a complete image as near perfect as possible with which they can configure one or 10,000 servers. Given identical configurations, all 10,000 can be managed as easily as one.

With the final phase of automation implemented, the network, working with a configuration management database, rapidly and automatically detects requests for, and changing needs of, applications. It then provisions servers and storage as needed. If an application initially can be served by three servers, the management software detects when their capacity exceeds a predetermined level and adds a fourth server, or when it falls below a specified level and reassigns one of the group. These dynamic responses occur in minutes or seconds.

Eventually, the service-oriented data center model should actually consist of two separate data centers, working in tandem in an active:active architecture. If one data center goes down, the other, which is already running with half the load, simply takes over all of it.

Repurposing SysAdmins

Cisco IT currently operates just under 10,000 servers. While this number could double within the next two years, the company doesn’t plan to hire additional system administrators to care for the servers, most of them virtualized. In the Cisco IT service-oriented data center, SysAdmins can each manage orders of magnitude more systems.

In a virtualized, automated environment, IT staff can focus their attention on other important areas. “We want to spend more time with our clients and learn more about their business and what they need from the IT infrastructure,” says Morgan. “It’s another way we can help the business become more efficient and profitable.”

The overriding benefits of Cisco’s service-oriented data center model—operational and cost efficiencies, increased business agility, and resiliency—are too strong to ignore. As Morgan comments, “Cisco IT wasted nine months coming up with our service-oriented data center model trying to find another way. Hopefully, sharing our experience will help other IT organizations develop their model faster and with less pain.”

PDF Document Cisco IT Checklist for a Service-Oriented Data Center Model (PDF -  32 KB)