Cisco on Cisco
Design for a Productive Data Center
A distributed physical design boosts productivity and enhances your company’s resiliency.
By Doug Alger
For the past ten years, Cisco’s Data Center Infrastructure Team has been designing the company’s server environments using a distributed physical approach. Cables run from a network row to a network substation located at the end of each server row deployed across the data center (see Figure 1). Networking devices for a given server row are installed in the corresponding substation, and from there data center users can plug patch cords into a patching field to connect to individual server cabinets.
In the traditional data center design, or direct-connect model, cables run from all server cabinet locations directly back to a main network row. While this design can work in small server environments, it is difficult to maintain and doesn’t scale in larger ones.
Once considered an uncommon approach to data center design, variations on the distributed model are increasingly being deployed in enterprise server environments across the industry. No surprise, because the distributed model provides several compelling business advantages over the direct-connect approach.
Figure 1. Distributed design used in a Cisco data center
Less cable congestion. Routing data cabling to multiple networking substations creates several paths for the infrastructure to follow, which means fewer potential choke points where cables cross. Where cable bundles do intersect, the overlap is much less than if all of the infrastructure was going to the same destination point.
Shorter cable runs. Because most of a server row’s connectivity needs are addressed by an associated network substation, only a fraction of the structured data cabling in the data center needs to continue from the substation to the room’s main networking row. This distributed design results in fewer long cable runs than in a direct-connect design (see Figure 2), and translates into lower cabling installation costs and fewer chances of a cable being exposed to damage.
Greater cooling efficiency. In data centers where cabling is routed in an underfloor plenum, the reduced cable congestion and fewer long cable runs make it easier for air to flow through the space.
Simpler cable management and troubleshooting. In a distributed cabling design, the port density of any given network substation is much less than in a design where the cables all terminate exclusively within a main networking row. The smaller volume of data cabling, in turn, makes it easier to manage the cables as well as locate and troubleshoot suspected problems with physical connections.
Greater modularity and scalability. Each network substation across the data center supports similar quantities of servers and data connections. It is, therefore, relatively straightforward to add or remove physical infrastructure or expand or shrink the data center, if the need arises.
At Cisco, the distributed design is now the default standard for new data centers and the prevalent design among the approximately 150,000 square feet (14,000 square meters) of existing hosting space.
Figure 2. Direct-Connect versus Distributed Cabling Design
The benefits of a distributed design are also being applied to data center electrical infrastructure as well. Remote power panels installed at the end of each server row perform the equivalent function as the network substations. Electrical conduits in a row connect to a nearby power panel instead of criss-crossing other conduits; connection is made with a source power distribution unit in a different part of the room.
This distributed design - for both cabling and power - is being applied even further in Cisco’s new data center under construction in Richardson, Texas, and scheduled to open in August 2007. The network substations will be divided in half and positioned at the end of different server rows to provide physical separation between them. Each half of the substation will house one of a pair of redundant network devices.
Designing server environments using a distributed model not only increases productivity among IT staff but goes a long way toward ensuring a flexible, resilient physical infrastructure developed with your company’s long-term needs in mind.