Cisco Adapter Fabric Extender

Solution Overview

The Cisco^® Fabric Extender architecture provides a highly scalable, unified server-access platform across a range of 100 Megabit Ethernet, Gigabit Ethernet, 10 Gigabit Ethernet, unified fabric, copper and fiber connectivity, rack and blade server, single OS, and virtual embedded bridge environments and directly connected virtual machines. The Cisco Nexus^® 5000 Series Switches is well suited to support today's traditional Gigabit Ethernet while allowing transparent migration to 10 Gigabit Ethernet and unified fabric technologies. It also allows flexible and scalable deployment options for virtualized environments that use virtual embedded bridges and highly optimized direct virtual machine connectivity.

The Cisco Nexus 2000 Series Fabric Extenders are the first products in the Cisco Fabric Extender architecture. With more than 3000 customers and more than 3 million ports sold, the Cisco Nexus 2000 Series has proven its exceptional business value and operational simplicity. The Cisco Nexus 2000 Series Fabric Extenders act like remote line cards for a parent Cisco Nexus switch. Together, the Nexus 2000 Series Fabric Extenders and the parent Cisco Nexus switch form a distributed modular system.

The Cisco Fabric Extender architecture enables:

●   Architecture flexibility: The common, scalable, and adaptive architecture across data center racks and points of delivery (PoDs) is server and server adapter agnostic and supports several connectivity options, physical topologies, and evolving needs.

●   Highly scalable server access: Scalable Gigabit and 10 Gigabit Ethernet server access is offered, with no reliance on Spanning Tree Protocol.

●   Simplified operations: One single point of management and policy enforcement using upstream Cisco Nexus switches eases the commissioning and decommissioning of server racks through zero-touch installation and automatic configuration of fabric extenders.

●   Increased business benefits: The extremely cost-effective in-rack cabling solution offers consolidation, rack-space reduction, reduced power and cooling, investment protection through feature inheritance from the parent switch, and the capability to add functions without the need for a major equipment upgrade of server-attached infrastructure. All these factors contribute to reduced operating expenses (OpEx) and capital expenditures (CapEx).

●   Open standards-based implementation: The Cisco Fabric Extender architecture uses IEEE 802.1Qbh standard. The introduction of the Cisco Adapter Fabric Extender (Adapter FEX) provides the same architectural benefits for the network interface card (NIC) as are provided for the physical access layer. Adapter FEX is logically an extension of the parent switch inside the server.

Interfaces of Adapter FEX are local logical ports on the parent switch. Adapter FEX uses innovative server connectivity (I/O connectivity) technology that enables on-demand creation of virtual NICs (vNICs) or virtual host bus adapters (vHBAs) on a single NIC. With Adapter FEX, a single physical adapter port is presented as multiple logical adapter ports to the server OS and the network, as if it were multiple physical adapter ports. A dual-port 10GE Adapter FEX can support hundreds of Peripheral Component Interconnect Express (PCIe) standards-compliant virtual interfaces that can be configured by the server administrator.

Each vNIC and vHBA created on the adapter automatically corresponds to a virtual Ethernet (vethernet) port on the parent switch to which the Adapter FEX is connected. Network properties are then assigned to each of the logical interfaces by the network administrator to help guarantee advanced quality of service (QoS) and granular bandwidth allocation.

Adapter FEX technology extends the current benefits of the Cisco Nexus 2000 Series Fabric Extender architecture to the server NICs, providing architecture flexibility, high scalability with 4000 logical interfaces, and one single point of management and policy enforcement, which result in increased business benefits.

In the Cisco UCS B-Series Blade Servers, the Cisco UCS M81KR Virtual Interface Card (VIC) is the first product that implements Cisco Adapter FEX technology.

Expanding this capability outside the Cisco UCS B-Series, the Cisco Nexus 5500 platform can support adapters implementing the Adapter FEX technology. An ecosystem of adapter vendors is now about to support this technology using the IEEE 802.1Qbh standard, the first vendor being Cisco itself, with the Cisco UCS P81E VIC (Figure 2), designed for use with Cisco UCS C-Series Rack-Mount Servers. Other adapter vendors will soon follow, providing adapters that support this capability. Both Cisco Nexus 5500 platform and the Cisco Nexus 2000 Series Fabric Extenders support the Adapter FEX technology across a variety of adapter platforms. Therefore, the offering can now be expanded outside a Cisco UCS environment to third-party server vendors that support IEEE 802.1Qbh-capable adapters.

Adapter FEX technology provides exceptional scalability and flexibility for both virtualized and non-virtualized environments, enabling on-demand, cost-effective solutions for data center server connectivity.

Features and Benefits

The Cisco Adapter FEX technology addresses these main challenges:

●   Organizations are deploying virtualized workloads to meet the strong needs to save costs and reduce physical equipment. Virtualization technologies and the increased number of CPU cores, however, require servers to be equipped with a large number of network connections. For example, a typical VMware server has six 1 Gigabit Ethernet ports and two 4-Gbps Fibre Channel ports corresponding to the eight cables needed to connect to the eight upstream network ports. This increase has a tremendous impact on CapEx and OpEx because of the large number of adapters, cables, and switch ports, which directly affects power, cooling, and cable management costs.

●   Network administrators struggle to link the virtualized network infrastructure and virtualized server platforms to the physical network infrastructure without degrading performance and with a consistent feature set and operational model.

●   With a consolidated infrastructure, it is challenging to provide guaranteed bandwidth, latency, and isolation of traffic across multiple cores and virtual machines.

●   Data center designs call for efficient cabling and reduced power and cooling because of stringent budgetary constraints.

●   In virtualized environments, network administrators experience lack of visibility into the traffic that is exchanged among virtual machines belonging to the same host. Administrators also face challenges in establishing and enforcing policies and maintaining configurations and policies consistently across mobility events. They often also see a dramatic increase in the number of management points; disparate provisioning, management, and operational models; and inconsistency between the physical and virtual access layers.

Adapter FEX provides these benefits:

●   Flexible and efficient deployment for non virtualized environments

●   Scalability for traditional virtualized environments using virtual embedded bridges

●   Highly optimized virtual machine connectivity for virtualized environments

Adapter FEX technology can be used in virtualized server environments to make the network infrastructure virtual machine aware (future). In this context, it is referred to as the Cisco Virtual Machine Fabric Extender (VM-FEX). This fabric extender integrates with the server virtualization management tool, allowing the user to bind a virtual machine to a vNIC carved out of Adapter FEX. This binding makes it possible to use an external hardware switch for switching the virtual machines traffic, having a single point of management and policy enforcement on the switch and enabling virtual machine migration support with port profile consistency. It provides tools with the same level of visibility, security, and troubleshooting for virtual machines as customers are accustomed to using for physical devices.

The portfolio of Cisco virtual machine networking products provides a variety of options that meet a range of customer needs:

●   Hypervisor switching with the Cisco Nexus 1000V Series Switches

●   Hardware switching with the Cisco VM-FEX and Cisco Nexus 5500 platform

●   Hardware switching with the Cisco VM-FEX and Cisco Unified Computing System

Across all these solutions, virtual machine networking enables:

●   Policy-based virtual machine networking

●   Transparent network and security policy mobility with virtual machine migration

●   Non-disruptive operational model, with the network administrator managing both virtual and physical networking resources with a consistent set of tools

Adapter FEX technology provides these main business benefits to IT departments:

●   On-demand design and deployment of data center applications to enable cloud deployments with reuse of existing equipment: The wire-once model enables subsequent deployment of unified fabric and virtualization technologies.

●   Exceptional scalability: 4000 logical host-facing ports are managed through a single point of configuration.

●   Infrastructure efficiency through consolidation: The network is simplified by reducing the number of adapters, cables, and network ports, also reducing the number of network devices and the management overhead and thus lowering CapEx and OpEx.

●   Ease and consistency of management: The server administrator can independently configure the adapter to a certain number of logical NICs, repurposing the physical NIC in real time as application needs evolve with little impact on the network and storage teams. At the same time, the network team can preconfigure the advanced network configuration, lowering the overall management overhead. Both teams continue to use traditional management tools with BIOS- or OS-based management tools on the server side and the command-line interface (CLI) on the network side.

Roles and Provisioning

Adapter FEX technology introduces outstanding flexibility and detailed control for both the network and server administrators. For each logical NIC that the server administrator defines on the Adapter FEX, the network administrator is responsible for the definition of the necessary network configuration.

On each Cisco Nexus switch connected to the server hosting the Adapter FEX, the network administrator creates port profiles (type vethernet) to be associated with the vNICs of the adapter. For example, if four vNICs (two for data, one for management, and one for backup) are required on the server, the network administrator creates one port profile for each type of vNIC (user_data, user_management, and user_backup) and configures relevant properties and policies (VLAN, bandwidth, QoS, application control lists [ACLs], etc.) in the port profile. Following is an example of a port-profile configuration:

port-profile type vethernet user_data

    switchport trunk allowed vlan 2-100

    switchport trunk native vlan 2

    switchport mode trunk

    state enabled

 

port-profile type vethernet user_management

    switchport access vlan 1

    state enabled

 

port-profile type vethernet user_backup

    switchport mode trunk

    switchport trunk allowed vlan 2-100

    switchport trunk native vlan 2

    mac port access-group mac_acl1

    ip port access-group ip_acl1 in

    ipv6 port traffic-filter ipv6_acl1 in

    state enabled

 

On the Adapter FEX, the server administrator now creates all the necessary vNICs and applies the relevant port profile as defined by the network administrator. To do this, the server administrator accesses the adapter configuration utility on the server and creates the desired number of vNICs with the desired properties (unique channel numbers, MAC addresses, and port-profile names). Names of port profiles (type vethernet) configured on the switch are pushed down to the server adapter as soon as connectivity is established. These port-profile names will be available in a drop-down list in the adapter configuration utility.

Figure 3 shows the configuration using the Cisco UCS P81E managed through the Cisco Integrated Management Controller (CIMC) tool.

Platform Support and Compatibility

Adapter FEX technology is supported on the new Cisco Nexus 5500 platform, which extends the industry-leading versatility of the Cisco Nexus 5000 Series of purpose-built 10 Gigabit Ethernet data center-class switches and provides innovative advances toward higher density, lower latency, and multilayer services. The Cisco Nexus 5500 platform is well suited for enterprise-class data center access-layer deployments and smaller-scale midmarket data center aggregation deployments across a diverse set of physical, virtual, storage-access, and high-performance-computing (HPC) data center environments. Adapter FEX connectivity is also supported by the Cisco Nexus 2000 Series Fabric Extenders, allowing cascading of the Fabric Extenders.

The Cisco Nexus 5548UP Switch (Figure 4) is a 1RU 10 Gigabit Ethernet and Fibre Channel over Ethernet (FCoE) switch offering up to 960-Gbps throughput and up to 48 ports. The switch has 32 1- and 10-Gbps fixed Enhanced Small Form-Factor Pluggable (SFP+) Ethernet and FCoE ports and one expansion slot.

The Cisco Nexus 5596UP Switch (Figure 5) is a 2RU 10 Gigabit Ethernet and FCoE switch offering up to 1.92-Tbps throughput and up to 96 ports. The switch has 48 1/10-Gbps fixed SFP+ Ethernet and FCoE ports and three expansion slots.

The Cisco Nexus 2232PP 10GE Fabric Extender (Figure 6) is a fabric extender controlled by the upstream parent switch. It operates as a remote line card, using the Port Extension technology described by prestandard IEEE 802.1Qbh. It provides 32 10 Gigabit Ethernet and FCoE SFP+ server ports and eight 10 Gigabit Ethernet and FCoE SFP+ uplink ports in a compact 1RU form factor.

Adapter FEX technology can also be supported when servers are connected to a Cisco Nexus 5500 switch through the Cisco Nexus 2232PP 10GE Fabric Extender. This support is possible because of the flexibility of the IEEE 802.1Qbh standard, which allows cascading of Port Extenders (Figure 7).

A Cisco innovation, the Cisco UCS P81E VIC (see Figure 2) is a virtualization-optimized FCoE PCIe 2.0 x8 10-Gbps adapter designed for use with Cisco UCS C-Series Rack-Mount Servers. The VIC is a dual-port 10 Gigabit Ethernet PCIe adapter that can support up to 128 PCIe standards-compliant virtual interfaces, which can be dynamically configured so that both the interface type (NIC or host bus adapter [HBA]) and identity (MAC address and worldwide name [WWN]) are established using just-in-time provisioning. In addition, the Cisco UCS P81E supports the Adapter FEX capability in bare-metal servers as well as virtualized environments.

System Requirements

Any server that supports PCIe network adapters will be compatible with the Cisco Adapter -FEX technology. The solution also requires a Cisco Nexus 5500 switch and, optionally, a Cisco Nexus 2000 Series Fabric Extender.

Cisco UCS P81E, when functioning as an AdapterFEX, supports up to 16 vNICs. Each vNIC can connect to the network using one of the two Cisco UCS P81E 10 Gigabit Ethernet uplink ports as an active uplink and the other as a standby uplink.

A Cisco Nexus 5500 switch will support 128 vethernet interfaces, and up to 256 VLANs can be configured on each of these interfaces.

For More Information

For more information, please visit http://www.cisco.com/go/nexus2000.