Overview

Introduction to Cisco UCS Manager

Cisco UCS Manager is embedded software that resides on the fabric interconnects, providing complete configuration and management capabilities for all of the components in the Cisco UCS system. This configuration information is replicated between the two fabric interconnects, providing a highly available solution for this critical function. The most common way to access Cisco UCS Manager for simple tasks is to use a Web browser. A command-line interface (CLI) and an XML API are also included for command-line or programmatic operations.

The Cisco UCS Manager GUI provides role-based access control (RBAC) to allow multiple levels of users administrative rights to system objects. Users can be restricted to certain portions of the system based on locale, which corresponds to an optional organizational structure that can be created. Users can also be classified based on their access levels or areas of expertise, such as Storage Administrator, Server Equipment Administrator, or Read-Only.

Cisco UCS Manager provides unified, embedded management of all software and hardware components. Every instance of Cisco UCS Manager and all of the components managed by it form a domain. For organizations that deploy multiple Cisco UCS domains, Cisco UCS Central software provides a centralized user interface that allows you to manage multiple, globally distributed Cisco UCS domains with thousands of servers. Cisco UCS Central integrates with Cisco UCS Manager and utilizes it to provide global configuration capabilities for pools, policies, and firmware.

Cisco UCS Manager User Documentation

Cisco UCS Manager offers you a new set of smaller, use-case based documentation described in the following table:

Guide

Description

Cisco UCS Manager Getting Started Guide

Discusses Cisco UCS architecture and Day 0 operations, including Cisco UCS Manager initial configuration and configuration best practices.

Cisco UCS Manager Administration Guide

Discusses password management, role-based access configuration, remote authentication, communication services, CIMC session management, organizations, backup and restore, scheduling options, BIOS tokens, and deferred deployments.

Cisco UCS Manager Infrastructure Management Guide

Discusses physical and virtual infrastructure components used and managed by Cisco UCS Manager.

Cisco UCS Manager Firmware Management Guide

Discusses downloading and managing firmware, upgrading through Auto Install, upgrading through service profiles, directly upgrading at endpoints using firmware auto sync, managing the capability catalog, deployment scenarios, and troubleshooting.

Cisco UCS Manager Server Management Guide

Discusses the new licenses, registering Cisco UCS domain with Cisco UCS Central, power capping, server boot, server profiles, and server-related policies.

Cisco UCS Manager Storage Management Guide

Discusses all aspects of storage management, such as SAN and VSAN in Cisco UCS Manager.

Cisco UCS Manager Network Management Guide

Discusses all aspects of network management, such as LAN and VLAN connectivity in Cisco UCS Manager.

Cisco UCS Manager System Monitoring Guide

Discusses all aspects of system and health monitoring, including system statistics in Cisco UCS Manager.

Fundamentals of Cisco Unified Computing System

Cisco Unified Computing System Overview

Cisco UCS has a unique architecture that integrates compute, data network access, and storage network access into a common set of components under a single-pane-of-glass management interface.

Cisco UCS fuses access layer networking and servers. This high-performance, next-generation server system provides a data center with a high degree of workload agility and scalability. The hardware and software components support Cisco's unified fabric, which runs multiple types of data center traffic over a single converged network adapter.

Figure 1. Cisco Unified Computing System Architecture

Architectural Simplification

The simplified architecture of Cisco UCS reduces the number of required devices and centralizes switching resources. By eliminating switching inside a chassis, network access-layer fragmentation is significantly reduced. Cisco UCS implements Cisco unified fabric within racks and groups of racks, supporting Ethernet and Fibre Channel protocols over 10/25/40 Gigabit Cisco Data Center Ethernet and Fibre Channel over Ethernet (FCoE) links. This radical simplification reduces the number of switches, cables, adapters, and management points by up to two-thirds. All devices in a Cisco UCS domain remain under a single management domain, which remains highly available through the use of redundant components.

High Availability

The management and data plane of Cisco UCS is designed for high availability and redundant access layer fabric interconnects. In addition, Cisco UCS supports existing high availability and disaster recovery solutions for the data center, such as data replication and application-level clustering technologies.

Scalability

A single Cisco UCS domain supports multiple chassis and their servers, all of which are administered through one Cisco UCS Manager. For more detailed information about the scalability, speak to your Cisco representative.

Flexibility

A Cisco UCS domain allows you to quickly align computing resources in the data center with rapidly changing business requirements. This built-in flexibility is determined by whether you choose to fully implement the stateless computing feature. Pools of servers and other system resources can be applied as necessary to respond to workload fluctuations, support new applications, scale existing software and business services, and accommodate both scheduled and unscheduled downtime. Server identity can be abstracted into a mobile service profile that can be moved from server to server with minimal downtime and no need for additional network configuration.

With this level of flexibility, you can quickly and easily scale server capacity without having to change the server identity or reconfigure the server, LAN, or SAN. During a maintenance window, you can quickly do the following:

  • Deploy new servers to meet unexpected workload demand and rebalance resources and traffic.

  • Shut down an application, such as a database management system, on one server and then boot it up again on another server with increased I/O capacity and memory resources.

Optimized for Server Virtualization

Cisco UCS has been optimized to implement VM-FEX technology. This technology provides improved support for server virtualization, including better policy-based configuration and security, conformance with a company's operational model, and accommodation for VMware's VMotion.

IOMs and Fabric Interconnects Connectivity

Each chassis is equipped with two IOMs: IOM 1 should be connected to Fabric Interconnect A. IOM 2 should be connected to Fabric Interconnect B. This configuration provides redundant paths, ensuring uninterrupted operation of the Cisco UCS system even in the event of a failure in one of the Fabric Interconnects or IOMs. Additionally, this configuration enables traffic load distribution across both Fabric Interconnects, enhancing load balancing and increasing throughput. As a result, the Cisco UCS system achieves high availability, reliability, and optimal performance, making it ideal for data center environments.

Uplink Connectivity

Use fabric interconnect ports configured as uplink ports to connect to uplink upstream network switches. Connect these uplink ports to upstream switch ports as individual links, or as links configured as port channels. Port channel configurations provide bandwidth aggregation as well as link redundancy.

You can achieve northbound connectivity from the fabric interconnect through a standard uplink, a port channel, or a virtual port channel configuration. The port channel name and ID configured on fabric interconnect should match the name and ID configuration on the upstream Ethernet switch.

It is also possible to configure a port channel as a vPC, where port channel uplink ports from a fabric interconnect are connected to different upstream switches. After all uplink ports are configured, create a port channel for these ports.

Downlink Connectivity

Beginning with release 4.3(2a), Cisco UCS Manager supports Cisco UCS X9508 server chassis with Cisco UCS X-Series servers. Cisco UCS X-Series servers support Intelligent Fabric Modules (IFM), which function similarly to the Input/Output Module (IOM) in Cisco UCS B-Series servers. This guide uses the term IOM to refer both IOM and IFM.

Each fabric interconnect is connected to IOMs in the UCS chassis, which provides connectivity to each blade server. Internal connectivity from blade servers to IOMs is transparently provided by Cisco UCS Manager using 10BASE-KR Ethernet standard for backplane implementations, and no additional configuration is required. You must configure the connectivity between the fabric interconnect server ports and IOMs. Each IOM, when connected with the fabric interconnect server port, behaves as a line card to fabric interconnect, hence IOMs should never be cross-connected to the fabric interconnect. Each IOM is connected directly to a single fabric interconnect.

The Fabric Extender (also referred to as the IOM, or FEX) logically extends the fabric interconnects to the blade server. The best analogy is to think of it as a remote line card that’s embedded in the blade server chassis, allowing connectivity to the external world. IOM settings are pushed via Cisco UCS Manager and are not managed directly. The primary functions of this module are to facilitate blade server I/O connectivity (internal and external), multiplex all I/O traffic up to the fabric interconnects, and help monitor and manage the Cisco UCS infrastructure.

Configure Fabric interconnect ports that should be connected to downlink IOM cards as server ports. Make sure there is physical connectivity between the fabric interconnect and IOMs. You must also configure the IOM ports and the global chassis discovery policy.


Note


For UCS 2200 I/O modules, you can also select the Port Channel option and all I/O module-connected server ports will be automatically added to a port channel.


Unified Fabric

With unified fabric, multiple types of data center traffic can run over a single Data Center Ethernet (DCE) network. Instead of having a series of different host bus adapters (HBAs) and network interface cards (NICs) present in a server, unified fabric uses a single converged network adapter. This type of adapter can carry LAN and SAN traffic on the same cable.

Cisco UCS uses Fibre Channel over Ethernet (FCoE) to carry Fibre Channel and Ethernet traffic on the same physical Ethernet connection between the fabric interconnect and the server. This connection terminates at a converged network adapter on the server, and the unified fabric terminates on the uplink ports of the fabric interconnect. On the core network, the LAN and SAN traffic remains separated. Cisco UCS does not require that you implement unified fabric across the data center.

The converged network adapter presents an Ethernet interface and Fibre Channel interface to the operating system. At the server, the operating system is not aware of the FCoE encapsulation because it sees a standard Fibre Channel HBA.

At the fabric interconnect, the server-facing Ethernet port receives the Ethernet and Fibre Channel traffic. The fabric interconnect (using Ethertype to differentiate the frames) separates the two traffic types. Ethernet frames and Fibre Channel frames are switched to their respective uplink interfaces.

Fibre Channel over Ethernet

Cisco UCS leverages Fibre Channel over Ethernet (FCoE) standard protocol to deliver Fibre Channel. The upper Fibre Channel layers are unchanged, so the Fibre Channel operational model is maintained. FCoE network management and configuration is similar to a native Fibre Channel network.

FCoE encapsulates Fibre Channel traffic over a physical Ethernet link. FCoE is encapsulated over Ethernet with the use of a dedicated Ethertype, 0x8906, so that FCoE traffic and standard Ethernet traffic can be carried on the same link. FCoE has been standardized by the ANSI T11 Standards Committee.

Fibre Channel traffic requires a lossless transport layer. Instead of the buffer-to-buffer credit system used by native Fibre Channel, FCoE depends upon the Ethernet link to implement lossless service.

Ethernet links on the fabric interconnect provide two mechanisms to ensure lossless transport for FCoE traffic:

  • Link-level flow control

  • Priority flow control

Link-Level Flow Control

IEEE 802.3x link-level flow control allows a congested receiver to signal the endpoint to pause data transmission for a short time. This link-level flow control pauses all traffic on the link.

The transmit and receive directions are separately configurable. By default, link-level flow control is disabled for both directions.

On each Ethernet interface, the fabric interconnect can enable either priority flow control or link-level flow control (but not both).

When an interface on a Cisco UCS 6600 Series Fabric Interconnect, Cisco UCS Fabric Interconnects 9108 100G, Cisco UCS 6500 Series Fabric Interconnect, Cisco UCS 6454 Fabric Interconnect or has Priority Flow Control (PFC) admin configured as auto and Link-Level Flow Control (LLFC) admin configured as on, the PFC operation mode will be off and the LLFC operation mode will be on.

Priority Flow Control

The priority flow control (PFC) feature applies pause functionality to specific classes of traffic on the Ethernet link. For example, PFC can provide lossless service for the FCoE traffic, and best-effort service for the standard Ethernet traffic. PFC can provide different levels of service to specific classes of Ethernet traffic (using IEEE 802.1p traffic classes).

PFC decides whether to apply pause based on the IEEE 802.1p CoS value. When the fabric interconnect enables PFC, it configures the connected adapter to apply the pause functionality to packets with specific CoS values.

By default, the fabric interconnect negotiates to enable the PFC capability. If the negotiation succeeds, PFC is enabled and link-level flow control remains disabled (regardless of its configuration settings). If the PFC negotiation fails, you can either force PFC to be enabled on the interface or you can enable IEEE 802.x link-level flow control.

Cisco UCS Building Blocks and Connectivity

Figure 2. Cisco UCS Building Blocks and Connectivity

The primary components included within Cisco UCS Manager are as follows:

  • Cisco UCS Manager—Cisco UCS Manager is the centralized management interface for Cisco UCS. For more information on Cisco UCS Manager, see Introduction to Cisco UCS Manager in Cisco UCS Manager Getting Started Guide

  • Cisco UCS Fabric Interconnects—The Cisco UCS Fabric Interconnect is the core component of Cisco UCS deployments, providing both network connectivity and management capabilities for the Cisco UCS system. The Cisco UCS Fabric Interconnects run the Cisco UCS Manager control software and consist of the following components:

    • Cisco UCS fabric interconnects:

      • Cisco UCS 6664 Fabric Interconnect

      • Cisco UCS Fabric Interconnects 9108 100G (Cisco UCS X-Series Direct)

      • Cisco UCS 6536 Fabric Interconnect

      • Cisco UCS 6400 Series Fabric Interconnect

    • Transceivers for network and storage connectivity

    • Expansion modules for various Fabric Interconnects

    • Cisco UCS Manager software

    For more information on Cisco UCS Fabric Interconnects, see Cisco UCS Fabric Infrastructure Portfolio.

  • Cisco UCS I/O Modules and Cisco UCS Fabric Extender—IO modules are also known as Cisco FEX or simply FEX modules. These modules serve as line cards to the FIs in the same way that Cisco Nexus Series switches can have remote line cards. IO modules also provide interface connections to blade servers. They multiplex data from blade servers and provide this data to FIs and do the same in the reverse direction. In production environments, IO modules are always used in pairs to provide redundancy and failover.


    Important


    The 40G backplane setting is not applicable for 22xx IOMs.


  • Cisco UCS Blade Server Chassis—The Cisco UCS 5100 Series Blade Server Chassis is a crucial building block of Cisco UCS, delivering a scalable and flexible architecture for current and future data center needs, while helping reduce total cost of ownership.

  • Cisco UCS Blade and Rack Servers—Cisco UCS Blade servers are at the heart of the UCS solution. They come in various system resource configurations in terms of CPU, memory, and hard disk capacity. The Cisco UCS rack-mount servers are standalone servers that can be installed and controlled individually. Cisco provides Fabric Extenders (FEXs) for the rack-mount servers. FEXs can be used to connect and manage rack-mount servers from FIs. Rack-mount servers can also be directly attached to the fabric interconnect.

    Small and Medium Businesses (SMBs) can choose from different blade configurations as per business needs.

  • Cisco UCS I/O Adapters—Cisco UCS B-Series Blade Servers are designed to support up to two network adapters. This design can reduce the number of adapters, cables, and access-layer switches by as much as half because it eliminates the need for multiple parallel infrastructure for both LAN and SAN at the server, chassis, and rack levels.

Cisco UCS Fabric Infrastructure Portfolio

The Cisco UCS fabric interconnects are top-of-rack devices and provide unified access to the Cisco UCS domain. The following fabric interconnects are available in the Cisco UCS fabric interconnects product family:

  • Cisco UCS 6600 Series Fabric Interconnect

    • Cisco UCS 6664 Fabric Interconnect introduced with Cisco UCS Manager Release 6.0(1b)

  • Cisco UCS Fabric Interconnects 9108 100G (Cisco UCS X-Series Direct introduced with Cisco UCS Manager Release 4.3(4b)

  • Cisco UCS 6500 Series Fabric Interconnects

    • Cisco UCS 6536 Fabric Interconnect)

  • Cisco UCS 6400 Series Fabric Interconnects

    • Cisco UCS 64108 Fabric Interconnect introduced in Cisco UCS Manager Release 4.1(1a)

    • Cisco UCS 6454 Fabric Interconnect introduced in Cisco UCS Manager Release 4.0(1a)

Cisco UCS Manager Fabric Interconnects

Cisco UCS 6600 Serie Fabric Interconnects

Overview of Cisco UCS 6664 Fabric Interconnect

The Cisco UCS 6664 Fabric Interconnect is a 2-rack unit (RU), fixed-port system designed for Top-of-Rack deployment in data centers. The fabric interconnect has both Ethernet and unified ports. Unified ports provide Fibre Channel over Ethernet (FCoE), Fibre Channel, NVMe over Fabric, and Ethernet. By supporting these different protocols, you can use a single multi-protocol Virtual Interface Card (VIC) in your servers.

The UCS 6664 Fabric Interconnect supports an array of Gigabit Ethernet (GbE), Fibre Channel (FC), and Fibre Channel over Ethernet (FCoE) ports to offer connectivity to peer data center devices. This device is also ideal for high-performance, scalable, and secure networking in modern data centers.

The Cisco UCS 6664 Fabric Interconnect includes:

  • 64 total ports

    • 48 ports of 40/100 Gbps

    • 16 unified ports

      • Supports 10/25 Gbps for Ethernet and Fibre Channel over Ethernet (FCoE)

      • Supports 10/25 Gbps or 16/32/64 Gbps Fibre Channel for maximum flexibility

  • 2RU fixed form factor for dense 100 Gbps connectivity

  • Fibre Channel end-host and switch-mode support

Cisco UCS 6664 Fabric Interconnect

The Cisco UCS 6664 Fabric Interconnect (UCSC-FI-6664) is a 2-rack unit (2RU), fixed-port system designed for Top-of-Rack (TOR) deployment in data centers. It serves as a central component in the Cisco UCS system, providing both network connectivity and management capabilities. It connects servers to the LAN or SAN, facilitating a unified computing environment. Management of the fabric interconnect can be performed through its console interface for initial setup, and subsequently via its management interface within Cisco UCS Manager.

Figure 3. Cisco UCS 6664 Fabric Interconnect Front View

Key features of Cisco UCS 6664 Fabric Interconnect:

  • Ports

    48 QSFP ports: These ports are divided into two groups:

    • QSFP Ports 1-24: Support Gigabit Ethernet links at speeds of 10 Gbps (through QSA transceivers), and 25 Gbps, 40 Gbps, and 100 Gbps (through QSA28 transceivers). FCoE links are also supported, but standard FC links are not.

    • QSFP Ports 41-64: Support only Gigabit Ethernet or Fibre Channel over Ethernet (FCoE) links at speeds of 10 Gbps (through QSA transceivers), and 25 Gbps, 40 Gbps, or 100 Gbps (through QSA28 transceivers). Standard Fibre Channel links are not supported on these ports.

  • 16 Unified Ports (SFP ports 25-40): These ports, indicated by yellow silk screening, are flexible and support:

    • Fibre Channel: 16 Gbps, 32 Gbps, or 64 Gbps. These are the only ports on the Fabric Interconnect that support standard FC traffic.

    • Gigabit Ethernet or Fibre Channel over Ethernet (FCoE): 10 Gbps or 25 Gbps.

  • MACsec-capable Ports: QSFP ports 49-64 are recommended as uplinks and support Media Access Control Security (MACsec) on Gigabit Ethernet links, but not FCoE links.

  • Breakout Support: Breakout port support for this fabric interconnect currently not available.

  • Secure Boot Support

  • Fan modules (4): These are port-side exhaust fan modules (NXASFAN-160CFM2-PE) with blue coloring, providing redundant and load-sharing cooling.

  • Power supply modules (2): Two 1400-W AC power supply modules (NXA-PAC-1400W-PE) are provided for 1+1 redundancy. These also feature port-side exhaust airflow with blue coloring.

Figure 4. Cisco UCS 6664 Fabric Interconnect Front Panel View

1

Beacon (BCN), Status (STS), and Environment (ENV) LEDs

2

QSFP Ports 1, 2, 5, 6, 9,10, 13, 14, 17, 18, 21, and 22. Ports are arranged vertically in pairs and support QSA or QSA28 transceivers.

3

SFP Ports 25, 26, 29, 30, 33, 34, 37, and 38 are Unified Ports

4

QSFP Ports 41, 42, 45, 46, 49, 50, 53, 54, 57,58, 61, and 62.

5

MACsec-capable QSFP ports 49, 50, 53, 54, 57, 58, 61, and 62.

6

MACsec-capable QSFP ports 51, 52, 55, 56, 59, 60, 63, and 64.

7

QSFP Ports 43, 44,47, 48, 51, 52, 55, 56, 59, 60, 63, and 64.

8

SFP Ports 27, 28, 31, 32, 35, 36, 39, and 40 are Unified Ports

9

QSFP Ports 3, 4, 7, 8, 11,12, 15, 16, 19, 20, 23, and 24. Ports are arranged vertically in pairs and support QSA or QSA28 transceivers.

Figure 5. Cisco UCS 6664 Fabric Interconnect Rear Panel View

1

Management Module

2

Fan Modules, numbered one through four with fan 1 on the left and fan four on the right.

3

Power Supply 1

4

Power Supply 2

Cisco UCS 6664 Fabric Interconnect Architecture

The Cisco UCS 6664 Fabric Interconnect is a high-density, line-rate, low-latency 100 Gbps solution specifically designed for the Cisco UCS X-Series Modular System and Cisco UCS C-Series Rack servers. It serves as a foundational component within Cisco UCS, unifying computing, networking, management, storage access, and virtualization resources into a single, cohesive system. This integrated approach is engineered to significantly reduce the Total Cost of Ownership (TCO) for data center deployments.

Architecture:

  • Front Panel: Houses all the network ports and system status lights (LEDs). These ports are highly flexible, supporting various connection types like Ethernet, Fibre Channel, and Fibre Channel over Ethernet (FCoE) through "Unified Ports."

  • Rear Panel: Contains the management connections (for setup and control), and the power supplies and fan modules.

  • Redundancy and Reliability: The design includes redundant power supplies and multiple fan modules to ensure continuous operation and efficient cooling.

This design allows the fabric interconnect to serve as a central hub, connecting servers to both the local network and external storage systems.

Key Features and Capabilities

  • Unified Fabric:The Cisco UCS 6664 Fabric Interconnect features 64 ports, including 48 QSFP ports (10/25 Gbps with Ethernet/FCoE support) and 16 Unified SFP ports (10/25 Gbps or 16/32/64 Gbps Fibre Channel). Ports 49-64 are optimized for secure uplinks with Media Access Control Security (MACsec).

  • Server Connectivity:

    • For Cisco UCS C-Series Rack Servers, direct connection to the 6664 Fabric Interconnect is supported. Alternatively, the Cisco Nexus 93180YC-FX3 switch can be deployed as a Fabric Extender (FEX) to provide the benefits of networking cabling consolidation for rack servers, similar to a modular server system.

    • For Cisco UCS X-Series Modular Systems, network cable consolidation is achieved through Intelligent Fabric Modules (IFMs). These IFMs are available in both 25 Gbps and 100 Gbps form factors, facilitating up to 200 Gbps of aggregate bandwidth per compute node.


Note


Port breakout functionality is not available on Cisco UCS 6664 Fabric Interconnects.


Port Functionality on Cisco UCS 6664 Fabric Interconnect

The Cisco UCS 6664 Fabric Interconnect is a 2-rack unit (RU) fixed-port system designed for flexible and high-performance networking. It features 64 front panel ports that support a variety of connectivity options.

Front Panel Port Configuration and Types

The UCS 6664 Fabric Interconnect supports the following possible configurations or port types for each front panel port:

Port Number Port Hardware Admin Port Speed Port Type Port Role
1-24 QSFP 28 40 Gbps/100 Gbps

Gigabit Ethernet

  • Server Port

  • Ethernet/FCoE Uplink Port

  • FCoE Storage Port

  • Appliance Port (EHM only)

  • Monitor Port

25-40 (Unified Ports) SFP28 16 Gbps/32 Gbps/64 Gbps Fibre Channel (FC)
  • FC Uplink Port

  • FC Storage Port

10 Gbps/25 Gbps Gigabit Ethernet
  • Server Port

  • Ethernet/FcoE Uplink Port

  • Appliance Port (EHM only)

  • Monitor Port

41-64

Note: Ports 49–64 are MAC Security (MACsec)-capable

QSFP 28 40 Gbps/100 Gbps Gigabit Ethernet
  • Server Port

  • Ethernet/FCoE Uplink Port

  • FCoE Storage Port

  • Appliance Port (EHM only)

  • Monitor Port


Note


Breakout port functionality is not supported on Cisco UCS 6600 Series Fabric Interconnects.


Cisco UCS X-Series Direct

Overview of Cisco UCS Fabric Interconnects 9108 100G (Cisco UCS X-Series Direct)

The Cisco UCS X-Series Direct is identified by the product ID UCSX-S9108-100G, and the product description Cisco UCS Fabric Interconnects 9108 100G.

Components of Cisco UCS Fabric Interconnects 9108 100G:

  • Two Cisco UCS X9508 Chassis

  • A pair of Cisco UCS Fabric Interconnects 9108 100G

  • One or more of the following servers:

    • Up to eight two-socket Cisco UCS X215c M8 Compute Nodes

    • Up to eight two-socket Cisco UCS X210c M6/M7/M8 Compute Nodes

    • Up to four four-socket Cisco UCS X410c M7 Compute Nodes

    • Up to four Cisco UCS C220 M7/M8 Servers

    • Up to four Cisco UCS C240 M7/M8 Servers

    • Up to four Cisco UCS C225 M7/M8 Servers

    • Up to four Cisco UCS C245 M7/M8 Servers

  • Optional components:

    • Cisco UCS 9416 X-Fabric Modules

    • Cisco UCS X440p PCIe Node with up to four GPUs used in conjunction with the 9416 X-Fabric Modules

The Cisco UCS Fabric Interconnects 9108 100G platform streamlines data center architecture by eliminating the need for separate Fabric Interconnects (FIs), integrating essential networking and management functionality directly within the chassis. The Cisco UCS Fabric Interconnects 9108 100G platform is designed for deployments in smaller settings, where the compute server requirements are less extensive than those of a traditional data center. This solution is centered around a single-chassis system, the Cisco UCS X9508 Chassis, which incorporates Cisco UCS Fabric Interconnects 9108 100G directly into the chassis for a consolidated and efficient infrastructure. To ensure high availability, each chassis houses two Cisco UCS Fabric Interconnects 9108 100G that establish direct downlink connections to servers and provide uplink connections to facilitate seamless integration with both Local Area Network (LAN) and Storage Area Network (SAN) systems. The Fabric Interconnects (FIs) are adeptly designed to fit into the Cisco UCS X-Series chassis, presenting as a single module within the NX-OS environment that merges QSFP ports with server backplane ports.

The hardware configuration of the Cisco UCS Fabric Interconnects 9108 100G platform retains the same form factor as the standard Cisco UCS X-Series chassis, and features 17 MACs, each configurable for 10 Gbps, 25 Gbps, 40 Gbps, or 100 Gbps connectivity. It is equipped with an CPU, for operating NX-OS, Cisco UCS Manager for management and Chassis Management Controller (CMC) software. The Cisco UCS Fabric Interconnects 9108 100G includes an onboard Ethernet switch with multiple 10G links dedicated to out-of-band communication between blade components such as the Baseboard Management Controller (BMC), CMC. A dedicated 1G link facilitates IFM-to-IFM clustering and high availability synchronization. Within the Cisco UCS Fabric Interconnects 9108 100G, Ethernet ports 1-8, backplane ports 9-16, and the Baseboard Interface (BIF) port 17 coexist on a singular switch card. Ports 1-2 are unified to manage all SAN features and configurations. The 100G Ethernet ports [1-8] can also be configured as 25Gx4 SFP28 compatible breakout ports or 4x10G ports, offering flexible networking solutions to accommodate a range of data center needs.

Cisco UCS Fabric Interconnects 9108 100G (Cisco UCS X-Series Direct) Architecture

The Cisco UCS X-Series Direct architecture is engineered to support a diverse range of workloads, from traditional applications to cloud-native services, by offering a composable and disaggregated approach to computing resources. Key components of the Cisco UCS X-Series Direct architecture include:

  • Cisco UCSX-9508 Chassis—A modular and future-proof chassis that can accommodate various types of compute nodes, providing the flexibility to adapt to different workload requirements without the need for a complete hardware overhaul.

    Cisco UCS X-Series Direct supports the addition of a secondary Cisco UCS X9508 Chassis, enabling scalability of up to 20 servers. This includes both Cisco UCS C-Series servers and Cisco UCS X-Series Compute Nodes, providing flexible expansion options for diverse workloads.

  • Cisco UCS Fabric Interconnects 9108 100G—This solution is centered around a single-chassis system, the Cisco UCS X9508 Chassis, which incorporates Cisco UCS Fabric Interconnects 9108 100G directly into the chassis for a consolidated and efficient infrastructure. To ensure high availability, each chassis houses two Cisco UCS Fabric Interconnects 9108 100G that establish direct downlink connections to servers and provide uplink connections to facilitate seamless integration with both Local Area Network (LAN) and Storage Area Network (SAN) systems.

  • Software Architecture—In terms of the startup and operational model, the management, Cisco UCS Manager aligns with the approach taken in the Cisco UCS 6500 and 6400 Series Fabric Interconnects. In this model, Cisco UCS Manager is encapsulated within a container and is initiated by the underlying NX-OS, depending on the selected management mode.

Port Breakout Functionality on Cisco UCS Fabric Interconnects 9108 100G (Cisco UCS X-Series Direct)

The Cisco UCS Fabric Interconnects 9108 100G is equipped with advanced port breakout functionality, which allows network administrators to subdivide a single high-bandwidth port into multiple lower-bandwidth ports. This feature is particularly beneficial for optimizing port utilization, managing cabling complexity, and adapting to various bandwidth requirements.

Physical Port

Breakout Options

Logical Ports After Breakout

Supported Speeds through breakout cables

Ethernet 1/1 - Ethernet 1/8

4x25G

Ethernet 1/1/1 to Ethernet 1/8/4

Up to 8x100 Gbps

Fibre Channel 1/1 and 1/2

4x8G, 4x16G, 4x32G

Fibre Channel 1/1/1 to Fibre Channel 1/2/4

Up to 8x32Gbps

Breakout Port Guidelines

Breakout ports are supported as destinations for traffic monitoring. The following are the guidelines for breakout functionality for Cisco UCS Fabric Interconnects 9108 100G:

  • Breakout Availability: Breakout functionality is available for physical ports 1-8.

  • Ethernet Breakout: Ethernet breakout ports can be configured on physical ports 1 through 8, resulting in 32 logical ports.

  • Fibre Channel Breakout: Fibre Channel breakout ports can be configured on unified ports 1/1 and 1/2, resulting in 8 logical ports.

  • Port Configurations: Physical Ports 1-8 can be configured as Uplink Ports, FCoE Uplink Ports, FCoE Storage Ports, and Appliance Ports.

  • Port Conversions: All port conversions support up to 8 standard ports or 8 breakout ports.

  • Server Ports: Configuration of server ports is supported only on ports 1–8, which are 100G ports. However, configuring a server port as a breakout port is not supported.

  • Fibre Channel Direct Ports: Direct ports for Fibre Channel are not supported.

  • Traffic Monitoring: Breakout ports can be utilized as destinations for traffic monitoring.

Cisco UCS 6500 Series Fabric Interconnects

Cisco UCS 6536 Fabric Interconnect

The Cisco UCS 6536 Fabric Interconnect (UCSC-FI-6536) is a One-rack unit (1RU), top of rack (TOR), fixed-port data center platform that provides both network connectivity and management capabilities to the Cisco UCS system.

The fabric interconnect can provide Ethernet and Fibre Channel connectivity to the servers in the system. The servers connect to the fabric interconnect, and then to the LAN or SAN.

Cisco UCS Manager on UCS 6536 Fabric Interconnect supports Cisco UCS X-Series Compute Nodes, B-Series Blade Servers and C-Series Rack Servers, Cisco UCS S-Series Storage Servers, as well as the associated storage resources and networks.

High availability and redundancy can be achieved by connecting a pair of fabric interconnects to each other through L1 or L2 ports in cluster mode configuration.

Each Cisco UCS 6536 Fabric Interconnect offers the following features:

  • Thirty-six QSFP28 ports capable of 100G including 4 unified ports (33-36). Ports also support:

    • Autonegotiating with peer devices to speeds of 100G, 40G, 25G, 10G, and 1G.

    • Port breakout is supported for Ethernet ports (1-32) and Unified ports (33-36).

      • Ethernet breakout is supported on switch ports 1 through 36 when each port is configured with a breakout cable.

        • The Dynamic Ethernet Breakout feature enables converting a standard Ethernet port to a breakout port on-the-fly so that you do not need to reboot the Fabric Interconnect. Dynamic Ethernet Breakout also supports converting breakout ports back to a standard Ethernet port without a reboot.

      • FC breakout is supported on ports 33 through 36 wherein each port is configured with a four-port breakout cable. For example 1/33/1, 1/33/2, 1/33/3, and 1/33/4 are the four FC breakout ports on the physical port 33.


        Note


        Fibre Channel support is only available through the configuration of Unified Ports (33-36) as FC breakout port.


        • FC breakout ports support peer communication at fixed speeds of 8Gbs, 16 Gbps, and 32 Gbps.

        • All four FC breakout ports must be configured with the same speed. Mixed speeds on a QSFP port's FC breakout ports are not supported.

        • Using breakout ports enables the fabric interconnect to support the maximum 16 FC ports supported by Fibre Channel.


      Note


      • Converting from Ethernet to FC breakout ports, or FC breakout ports back to Ethernet, requires a reboot/reload after changing the breakout type.

      • FCoE storage ports are not supported.


  • One management port (one 10/100/1000BASE-T port)

  • Two L1/L2 Ethernet RJ-45 ports for high availability or cluster configurations. Ethernet ports support 10/100/1000Mb speed.

  • One console port (RS-232)

  • One USB 3.0 port

  • CPU: 4 Core, 1.8GHz, Intel 5th-Generation core processor

  • Memory:

    • 32 GB DDR4 DIMMs

    • 128 GB M.2 SSD Flash Drive

    • 32 GB Boot Flash (16 MB primary, and 16 MB standby/golden)

This fabric interconnect includes the following user-replaceable components:

  • Fan modules (6), each is a port-side exhaust fan module with dark grey latch coloring (UCS-FAN-6536).

  • Power supply modules (2). One power supply module (PSU) is the active module for operations, and the second PSU is the standby for redundancy [1+1]) with the following choices:

    • 1100-W AC power supply with dark grey latch coloring (UCS-PSU-6536-AC)


    Note


    All fan modules and power supplies must use the same airflow direction.


The following figure shows the fabric interconnect features on the port side of the chassis.

1

LEDs

3

36 40/100-Gigabit QSFP28 ports

2

Lane Select button

To determine which transceivers, adapters, and cables are support the fabric interconnect, see the Cisco Transceiver Modules Compatibility Information document.

The following figure shows the fabric interconnect features on the power supply side of the chassis.

1

Power supply modules (1 or 2) (AC power supplies shown) with slots numbered 1 (left) and 2 (right).

2

Fan modules (6) with slots numbered from 1 (left) to 6 (right).

3

Layer 2 (L2) Ethernet port, 10/100/100Mb autonegotiating.

Supports high availability (HA) or clustering through an RJ-45 port.

4

Layer 1 (L1) Ethernet port, 10/100/100Mb autonegotiating.

Supports high availability (HA) or clustering through an RJ-45 port.

5

Ethernet network management port (RJ45), 10/100/1000Mb autonegotiating

6

Serial Console port (RJ45), 9600 baud.

7

USB 3.0/2.0 port

Supports booting the system and downloading scripts.

8

Beacon (BCN) LED

9

Status (STS) LED

-

The following figure shows the side of the chassis.

1

Screw holes for mounting brackets

2

Grounding pad

Plan to position the ports in a hot aisle so that fans and power supplies intake air from the cold aisle, blow the cool air through the fabric interconnect, and exhaust the heated air into the hot aisle.

The fan and power supply modules are field replaceable. You can replace one fan module or one power supply module during operations so long as the other modules are installed and operating. If you have only one power supply installed, you can install the replacement power supply in the open slot before removing the original power supply.


Note


All fan and power supply modules must have the same direction of airflow. Otherwise, the fabric interconnect can overheat and shut down.



Caution


Because fans and power supply modules have port-side exhaust airflow (blue coloring for fan modules), you must locate the ports in the hot aisle. If you locate the air intake in a hot aisle, the fabric interconnect can overheat and shut down.


Port Breakout Functionality on Cisco UCS 6536 Fabric Interconnects

The Cisco UCS 6536 36-Port Fabric Interconnect is a One-Rack-Unit (1RU) 1/10/25/40/100 Gigabit Ethernet, FCoE, and Fibre Channel switch offering up to 7.42 Tbps throughput and up to 36 ports.

Cisco UCS 6536 Fabric Interconnect supports splitting a single 40 Gigabit(G)/100G Quad Small Form-factor Pluggable (QSFP) port into four 10G/25G ports using a supported breakout cable. The switch has 32 40/100-Gbps Ethernet ports and four unified ports that can support 40/100-Gbps Ethernet ports or 16 Fiber Channel (FC) ports after breakout at 8/16/32-Gbps FC speeds. The 16 FC ports after breakout can operate as an FC Uplink or FC storage port. The switch also supports two ports (Port 9 and Port 10) at 1-Gbps speed using QSA, and all 36 ports can breakout for 10 or 25 Gbps Ethernet connectivity. All Ethernet ports can support FCoE.

Port breakout is supported for Ethernet ports (1-32) and Unified ports (33-36). These 40/100G ports are numbered in a 2-tuple naming convention. The process of changing the configuration from 40G to 10G, or from 100G to 25G is called breakout, and the process of changing the configuration from [4X]10G to 40G or from [4X]25G to 100G is called unconfigure.

When you break out a 40G port into 10G ports or a 100G port into 25G ports, the resulting ports are numbered using a 3-tuple naming convention. For example, the breakout ports of the second 40-Gigabit Ethernet port are numbered as 1/31/1, 1/31/2, 1/31/3, and 1/31/4.

FC breakout is supported on ports 36 through 33 when each port is configured with a four-port breakout cable. For example: Four FC breakout ports on the physical port 33 are numbered as 1/33/1, 1/33/2, 1/33/3, and 1/33/4.


Note


Fibre Channel support is only available through the configuration of the Unified Ports (36-33) as Fibre Channel breakout port.


The following image shows the rear view of the Cisco UCS 6536 fabric interconnect:

Figure 6. Cisco UCS 6536 Fabric Interconnect Rear View

The following image shows the rear view of the Cisco UCS 6536 fabric interconnect that include Ports and LEDs:

Figure 7. Cisco UCS 6536 Fabric Interconnect Rear View

1

Ports 1-32.

Uplink ports are Ethernet port that can operate with the port speed of 10 Gbps/25 Gbps/40 Gbps/100 Gbps.

When using a breakout cable, each of these ports can operate as: 4 x 10 Gbps/4x 25 Gbps/1 x 40 Gbps/1 x 100 Gbps Ethernet or FCoE ports.

2

Ports 33-36.

Unified Ports can operate with port speed of 10 Gbps/25 Gbps/ 40 Gbps/100 Gbps Ethernet.

or

8 Gbps/16 Gbps/32 Gbps Fibre Channel (FC).

When using a breakout cable, each of these ports can operate as 4 x 10 Gbps/4 x 25 Gbps Ethernet or 4x8Gbps/4x16Gbps/4x32Gbps FC ports.

3

Ports 1-36.

Uplink ports and Unified ports that can be configured as Ethernet Breakout Port and can operate with the port speed of 10 Gbps/25 Gbps/40 Gbps/100 Gbps.

When using a breakout cable, each of these ports can operate as: 4 x 10 Gbps/4x 25 Gbps/1 x 40 Gbps/1 x 100 Gbps Ethernet or FCoE ports.

4

System environment (fan fault) LED

5

System status (STS) LED

6

Beacon (BCN) LED

Breakout Port Guidelines

The following are the guidelines for breakout functionality for Cisco UCS 6536 Fabric Interconnects:

  • The configurable breakout ports are from port 1-36.

  • You can configure the speed for each breakout port. Each breakout port can be configured at the speed of 4 x 8 Gbps/ 4 x 16 Gbps/ 4 x 32 Gbps for Fibre Channel.

    • For Fibre Channel breakout, each breakout port can be configured at the speed of 4 x 8 Gbps/ 4 x 16 Gbps/ 4 x 32 Gbps.

    • For Ethernet breakout, each breakout port can be configured at the speed of 4 x 10 Gbps/4 x 25 Gbps.

  • Fibre Channel breakout ports are supported, and Fiber Channel direct ports are not supported.

  • FC breakout port can be configured from 1/36 through 1/33. FC breakout ports (36-33) cannot be configured unless the previous ports are FC breakout ports. Configuration of a single (individual) FC breakout port is also supported.

  • If the breakout mode for any of the supported Fabric Interconnect ports (1-36) is an Ethernet breakout, the Fabric Interconnect does not lead to a reboot.

  • If the breakout mode for any of the supported Fabric Interconnect ports (36-33) is a Fibre Channel uplink breakout, the Fabric Interconnect leads to a reboot.

  • Breakout ports are supported as destinations for traffic monitoring.

  • Ports 1-36 can be configured as Server Port, FCoE Uplink Port, Appliance Port, and Monitor Port.

  • Port 36-33 can be configured also as FC Uplink Port or FC Storage Port when configured as unified port.

Cisco UCS 6400 Series Fabric Interconnects

Cisco UCS 6400 Series Fabric Interconnect Overview

Cisco UCS 6400 Series Fabric Interconnect provides both network connectivity and management capabilities to the Cisco UCS system. The fabric interconnect provides Ethernet and Fibre Channel to the servers in the system, the servers connect to the fabric interconnect, and then to the LAN or SAN.

Each Cisco UCS 6400 Series Fabric Interconnect runs Cisco UCS Manager to fully manage all Cisco UCS elements. The fabric interconnect supports 10/25 Gigabit ports in the fabric with 40/100 Gigabit uplink ports. High availability can be achieved when a Cisco UCS 6400 Series Fabric Interconnect is connected to another Cisco UCS 6400 Series Fabric Interconnect through the L1 or L2 port on each device.

Cisco UCS 6400 Series Fabric Interconnect consists of:

  • Cisco UCS 6454 Fabric Interconnects

  • Cisco UCS 64108 Fabric Interconnects

The Cisco UCS 6400 Series fabric interconnect supports Cisco UCS B-Series Blade Servers, UCS 5108 B-Series Server Chassis, C-Series Rack Servers, and UCS S-Series Storage Servers.

Cisco UCS 64108 Fabric Interconnect

The Cisco UCS 64108 Fabric Interconnect is a 2 RU top-of-rack (TOR) switch that mounts in a standard 19-inch rack such as the Cisco R Series rack.

The high-density Cisco UCS 64108 Fabric Interconnect has 96 10/25 Gb SFP28 ports and 12 40/100 Gb QSFP28 ports. Each 40/100 Gb port can break out into 4 x 10/25 Gb uplink ports. Ports 1 - 16 are unified ports that support 10/25 GbE or 8/16/32G Fibre Channel speeds. Ports 89-96 support 1Gbps Ethernet speeds.

The Cisco UCS 64108 Fabric Interconnect supports either:

  • Eight FCoE port channels

  • Or Four SAN port channels

  • or Four SAN port channels and four FCoE port channels

The Cisco UCS 64108 Fabric Interconnect also has one network management port, one RS-232 serial console port for setting the initial configuration, and one USB port for saving or loading configurations. The FI also includes L1/L2 ports for connecting two fabric interconnects in a high-availability configuration.

The Cisco UCS 64108 Fabric Interconnect also contains a CPU board that consists of:

  • Intel Xeon Processor, 6 core

  • 64 GB of RAM

  • 8 MB of NVRAM (4 x NVRAM chips)

  • 128 GB SSD (bootflash)

Figure 8. Cisco UCS 64108 Fabric Interconnect Rear View

1

Ports 1-16

Unified ports:

  • 10/25 Gbps Ethernet or FCoE

  • 8/16/32 Gbps Fibre Channel

2

Ports 17-88

(10/25 Gbps Ethernet or FCoE)

3

Ports 89-96

  • 10/25 Gbps Ethernet or FCoE

  • 1 Gbps Ethernet

4

Uplink Ports 97-108 (40/100 Gbps Ethernet or FCoE)

Each of these ports can be 4 x 10/25 Gbps Ethernet or FCoE uplink ports when using a breakout cable.

5

System environment (fan fault) LED

6

System status LED

7

Beacon LED

The Cisco UCS 64108 Fabric Interconnect has two power supplies (redundant as 1+1) and three fans (redundant as 2+1).

Figure 9. Cisco UCS 64108 Fabric Interconnect Front View

1

Cooling fans

(hot swappable, 2+1 redundancy)

2

RS-232 serial console port (RJ-45 connector)

3

Network management port (RJ-45 connector)

4

USB port

5

Grounding pad for two-hole grounding lug (under protective label)

6

Power supplies

Two identical AC or DC PSUs, hot-swappable, 1+1 redundancy)

7

L1/L2 high-availability ports (RJ-45 connector)

8

Beacon LED

9

System status LED

Port Breakout Functionality on Cisco UCS 64108 Fabric Interconnects

About Breakout Ports

Cisco UCS 64108 fabric interconnects support splitting a single 40/100G QSFP port into four 10/25G ports using a supported breakout cable. On the UCS 64108 fabric interconnect, by default, there are 12 ports in the 40/100G mode. These are ports 97 to 108. These 40/100G ports are numbered in a 2-tuple naming convention. For example, the second 40G port is numbered as 1/99. The process of changing the configuration from 40G to 10 G, or from 100G to 25G is called breakout, and the process of changing the configuration from [4X]10G to 40G or from [4X]25G to 100G is called unconfigure. These ports can be used as uplink port, appliance port, server port (using FEX), and FCoE storage port.

When you break out a 40G port into 10G ports or a 100G port into 25G ports, the resulting ports are numbered using a 3-tuple naming convention. For example, the breakout ports of the second 40-Gigabit Ethernet port are numbered as 1/99/1, 1/99/2, 1/99/3, 1/99/4.

Starting with Cisco UCS Manager Release 4.2(3b), configuring the Ethernet breakout ports will not lead to Fabric Interconnect reboot.

The following image shows the rear view of the Cisco UCS 64108 fabric interconnect, and includes the ports that support breakout port functionality:

Figure 10. Cisco UCS 64108 Fabric Interconnect Rear View

1

Ports 1-16. Unified Ports can operate as 10/25 Gbps Ethernet or 8/16/32 Gbps Fibre Channel. FC ports are converted in groups of four.

Unified ports:

  • 10/25 Gbps Ethernet or FCoE

  • 8/16/32 Gbps Fibre Channel

2

Ports 1-96. Each port can operate as either a 10 Gbps or 25 Gbps Ethernet or FCoE SFP28 port.

3

Uplink Ports 97-108. Each port can operate as either a 40 Gbps or 100 Gbps Ethernet or FCoE port. When using a breakout cable, each of these ports can operate as 4 x 10 Gbps or 4 x 25 Gbps Ethernet or FCoE ports.

Ports 97 - 108 can be used to connect to Ethernet or FCoE uplink ports, and not to UCS server ports.

4

Ports 89-96

  • 10/25 Gbps Ethernet or FCoE

  • 1 Gbps Ethernet

5

System environment (fan fault) LED

6

System status LED

7

Beacon LED

Breakout Port Guidelines

The following are the guidelines for breakout functionality for Cisco UCS 64108 fabric interconnects:

  • The breakout configurable ports are ports 97-108.

  • You cannot configure the speed for each breakout port. Each breakout port is in auto mode.

  • Breakout ports are not supported as destinations for traffic monitoring.

  • Ports 97-108 at 40/100G can be configured as uplink, FCoE, or appliance port. Ports 97-108 after breakout to 10/25G can be configured as uplink, appliance, FCoE, or for direct-connect rack server connectivity.

Cisco UCS 6454 Fabric Interconnect

The Cisco UCS 6454 Fabric Interconnect (FI) is a 1-RU top-of-rack switch that mounts in a standard 19-inch rack such as the Cisco R Series rack.

The Cisco UCS 6454 Fabric Interconnect has 48 10/25 Gb SFP28 ports (16 unified ports) and 6 40/100 Gb QSFP28 ports. Each 40/100 Gb port can break out into 4 x 10/25 Gb uplink ports. The sixteen unified ports support 10/25 GbE or 8/16/32G Fibre Channel speeds.


Note


The Cisco UCS 6454 Fabric Interconnect supported 8 unified ports (ports 1 - 8) with Cisco UCS Manager 4.0(1) and 4.0(2), but with release 4.0(4) and later it supports 16 unified ports (ports 1 - 16).


The Cisco UCS 6454 Fabric Interconnect supports:

  • Maximum of 8 FCoE port channels

  • Or 4 SAN port channels

  • Or a maximum of 8 SAN port channels and FCoE port channels (4 each)

The Cisco UCS 6454 Fabric Interconnect also has one network management port, one console port for setting the initial configuration, and one USB port for saving or loading configurations. The FI also includes L1/L2 ports for connecting two fabric interconnects for high availability.

The Cisco UCS 6454 Fabric Interconnect also contains a CPU board that consists of:

  • Intel Xeon D-1528 v4 Processor, 1.6 GHz

  • 64 GB of RAM

  • 8 MB of NVRAM (4 x NVRAM chips)

  • 128 GB SSD (bootflash)

Figure 11. Cisco UCS 6454 Fabric Interconnect Rear View

1

Ports 1-16 (Unified Ports 10/25 Gbps Ethernet or FCoE or 8/16/32 Gbps Fibre Channel)

Note

 

When using Cisco UCS Manager releases earlier than 4.0(4), only ports 1-8 are Unified Ports.

2

Ports 17-44 (10/25 Gbps Ethernet or FCoE)

Note

 

When using Cisco UCS Manager releases earlier than 4.0(4), ports 9-44 are 10/25 Gbps Ethernet or FCoE.

3

Ports 45-48 (1/10/25 Gbps Ethernet or FCoE)

4

Uplink Ports 49-54 (40/100 Gbps Ethernet or FCoE)

Each of these ports can be 4 x 10/25 Gbps Ethernet or FCoE uplink ports when using an appropriate breakout cable.

The Cisco UCS 6454 Fabric Interconnect chassis has two power supplies and four fans. Two of the fans provide front to rear airflow.

Figure 12. Cisco UCS 6454 Fabric Interconnect Front View

1

Power supply and power cord connector

2

Fans 1 through 4, numbered left to right, when facing the front of the chassis.

3

L1 port, L2 port, RJ45, console, USB port, and LEDs

Port Breakout Functionality on Cisco UCS 6454 Fabric Interconnects

About Breakout Ports

Cisco UCS 6454 fabric interconnects support splitting a single 40/100G QSFP port into four 10/25G ports using a supported breakout cable. These ports can be used only as uplink ports connecting to a 10/25G switch. On the UCS 6454 fabric interconnect, by default, there are 6 ports in the 40/100G mode. These are ports 49 to 54. These 40/100G ports are numbered in a 2-tuple naming convention. For example, the second 40G port is numbered as 1/50. The process of changing the configuration from 40G to 10 G, or from 100G to 25G is called breakout, and the process of changing the configuration from [4X]10G to 40G or from [4X]25G to 100G is called unconfigure.

When you break out a 40G port into 10G ports or a 100G port into 25G ports, the resulting ports are numbered using a 3-tuple naming convention. For example, the breakout ports of the second 40-Gigabit Ethernet port are numbered as 1/50/1, 1/50/2, 1/50/3, 1/50/4.

Starting with Cisco UCS Manager Release 4.2(3b), Ethernet breakout ports configuration will not lead to Fabric Interconnect reboot.

The following image shows the rear view of the Cisco UCS 6454 fabric interconnect, and includes the ports that support breakout port functionality:

Figure 13. Cisco UCS 6454 Fabric Interconnect Rear View

1

Ports 1-16 (Unified Ports 10/25 Gbps Ethernet or FCoE or 8/16/32 Gbps Fibre Channel)

2

Ports 17-44 (10/25 Gbps Ethernet or FCoE)

3

Ports 45-48 (1/10/25 Gbps Ethernet or FCoE)

4

Uplink Ports 49-54 (40/100 Gbps Ethernet or FCoE)

Breakout Port Guidelines

The following are the guidelines for breakout functionality for Cisco UCS 6454 fabric interconnects:

  • The breakout configurable ports are ports 49-54.

  • You cannot configure the speed for each breakout port. Each breakout port is in auto mode.

  • In Cisco UCS Manager Release 4.0(2), breakout ports are not supported as destinations for traffic monitoring.

  • Ports 49-54 at 40/100G can be configured as uplink, FCoE, or appliance port. Ports 49-54 after breakout to 10/25G can be configured as uplink, appliance, FCoE, or for direct-connect rack server connectivity.

Ports on the Cisco UCS Fabric Interconnects

Cisco UCS Fabric Interconnect Port Overview

Cisco UCS Fabric Interconnects offer flexible and high-performance networking solutions, supporting various port configurations for Ethernet, Fibre Channel (FC), and Fibre Channel over Ethernet (FCoE) traffic. The specific capabilities vary by model and generation.

Port Details by Fabric Interconnect Model

This section details the unique port configurations and capabilities of each Cisco UCS Fabric Interconnect model.
  • Cisco UCS 6664 Fabric Interconnect

    Cisco UCS 6664 Fabric Interconnect features 64 ports, designed for flexible and high-performance networking:

    • Ports 1-24: supports 40/100 Gbps for Ethernet, FCoE, and storage connectivity.

    • Ports 25-40 (Unified Ports): supports configuration of 10/25Gbps Ethernet or 16/32/64Gbps Fibre Channel (FC).

    • Ports 41-64: supports configuration of 40/100 Gbps Ethernet with MACsec security (ports 49-64).

  • Cisco UCS Fabric Interconnets 9108 100G (Cisco UCS X-Series Direct)

    • Ports 1 & 2: Functions as unified ports to manage all SAN features and configurations.

    • 100G Ethernet ports [1-8]: Can be configured as 25Gx4 SFP28 compatible breakout ports or 4x10G ports, offering flexible networking solutions.

    • 32G Fibre Channel ports [1 & 2] : Can also be configured as 8Gx4 SFP28 compatible breakout ports for enhanced Fibre Channel connectivity.

  • Cisco UCS 6536 Fabric Interconnect

    Ports on the Cisco UCS 6536 Fabric Interconnects can be configured to carry either Ethernet or Fibre Channel traffic. You can configure only ports 33-36 to carry Fibre Channel traffic. The ports cannot be used by a Cisco UCS domain until you configure them.

  • Cisco UCS 6400 Series Fabric Interconnect

    Ports on the Cisco UCS 6400 Series Fabric Interconnects can be configured to carry either Ethernet or Fibre Channel traffic. The Cisco UCS 6400 Series Fabric Interconnect supported 8 unified ports (ports 1 - 8) with Cisco UCS Manager 4.0(1) and 4.0(2), but with Release 4.0(4) and later releases, it supports 16 unified ports (ports 1 - 16).

    Ports 1-16 can be configured to carry Fibre Channel traffic. The ports cannot be used by a Cisco UCS domain until you configure them.

Configuration Notes

When you configure a port on a Fabric Interconnect, the administrative state is automatically set to enabled. If the port is connected to another device, this may cause traffic disruption. The port can be disabled and enabled after it has been configured.

The following table summarizes the port support the Cisco UCS Fabric Interconnects:

Fourth Generation

Fifth Generation

Cisco UCS X-Series Direct

Sixth Generation

Item

Cisco UCS 6454

Cisco UCS 64108

Cisco UCS 6536

Cisco UCS Fabric Interconnect 9108 100G

Cisco UCS 6664 Fabric Interconnect

Description

54-Port Fabric Interconnect

108-Port Fabric Interconnect

36-Port Fabric Interconnect

8 Ports

64 Ports

Form factor

1 RU

2 RU

1 RU

1 RU

2 RU

Number of fixed 10 GB Interfaces

48 10G/25G interfaces

96 10G/25G interfaces

36 10G/25G/40G/100G interfaces

Note

 

144 breakout ports (36x4)

  • 16x 10G/25G ports

  • 48x 100G ports will support 10/25G using QSA/QSA28 modules

Number of Unified Ports

16

This FI supported 8 unified ports (ports 1 - 8) with Cisco UCS Manager 4.0(1) and 4.0(2), but with Release 4.0(4) and later it supports 16 unified ports (ports 1 - 16).

16

ports 1-16

4

Note

 

16 breakout ports (4x4)

Ports 1-2

16

Unified Port Speeds in Gbps

10G/25G or 8G/16G/32G-FC

10G/25G or 8G/16G/32G-FC

10G/25G/40G/100G or 8G/16G/32G-FC

10G/25G/40G/100G or 8G/16G/32G-FC

10G/25G or 16G/32G/64G-FC

Number of 40-Gbps ports

6 40G/100G

12 40G/100G

36

Unified Port Range

Ports 1-16

Ports 1-16

Ports 33-36

Ports 1-2

Ports 25-40

Compatibility with the IOM

UCS 2408

UCS 2408

UCS 2408

UCS 2408

Compatibility with the FEX

Cisco Nexus 93180YC-FX3

Cisco Nexus 93180YC-FX3

Cisco Nexus 93180YC-FX3

Cisco Nexus 93180YC-FX3

Compatibility with IFM

Cisco UCS 9108 25G

Cisco UCS 9108 25G

Cisco UCS 9108 25G

Cisco UCS 9108 100G

Cisco UCS 9108 25G

Cisco UCS 9108 100G

Cisco UCS 9108 25G

Cisco UCS 9108 100G

Expansion Slots

None

None

None

None

Fan Modules

4

3

6

3

4

Power Supplies

2 (AC/DC)

2 (AC/DC)

2 (AC)

Supplied by chassis

2 (AC)

Migration Limitations:

Migration from any generation Cisco UCS Fabric Interconnects (including fourth, fifth, or sixth generation) to Cisco UCS Fabric Interconnects 9108 100G (Cisco UCS X-Series Direct) is currently not supported.

Migration for Cisco UCS Fabric Interconnects

Cisco UCS Manager supports migration between various Fabric Interconnect models to facilitate seamless upgrades and enhanced performance. Below is the list of supported migrations:

  • Cisco UCS 6400 Series to Cisco UCS 6500 Series Fabric Interconnects

    • Cisco UCS 6454 Fabric Interconnect to Cisco UCS 6536 Fabric Interconnect

    • Cisco UCS 64108 Fabric Interconnect to Cisco UCS 6536 Fabric Interconnect

    • Cisco UCS 6454 Fabric Interconnect to Cisco UCS 6536 Fabric Interconnect migration with UCS Central

    • Cisco UCS 64108 Fabric Interconnect to Cisco UCS 6536 Fabric Interconnect migration with UCS Central

    • Cisco UCS 6400 Series to Cisco UCS 6600 Series Fabric Interconnects

      • Cisco UCS 6454 Fabric Interconnect to Cisco UCS 6664 Fabric Interconnect

      • Cisco UCS 64108 Fabric Interconnect to Cisco UCS 6664 Fabric Interconnect

      • Cisco UCS 6454 Fabric Interconnect to Cisco UCS 6664 Fabric Interconnect migration with UCS Central

      • Cisco UCS 64108 Fabric Interconnect to Cisco UCS 6664 Fabric Interconnect migration with UCS Central

    • Cisco UCS 6500 Series to Cisco UCS 6600 Series Fabric Interconnect

    • Cisco UCS 6536 to Fabric Interconnect to Cisco UCS 6664 Fabric Interconnect

    • Cisco UCS 6536 to Fabric Interconnect to Cisco UCS 6664 Fabric Interconnect with UCS Central


Note


  • The Cisco UCS Fabric Interconnects 9108 100G (Cisco UCS X-Series Direct) cannot currently be migrated to any other Fabric Interconnects.

  • For more information, refer to the appropriate Migration Guide for Cisco UCS Fabric Interconnects available on the Cisco UCS Manager Configuration Guides listing page.


Configuration Options

You can configure a Cisco UCS domain in the following ways:

  • As a single fabric interconnect in a standalone configuration

  • As a redundant pair of fabric interconnects in a cluster configuration

A cluster configuration provides high availability. If one fabric interconnect becomes unavailable, the other takes over. Only one management port (Mgmt0) connection is required to support a cluster configuration. However, both Mgmt0 ports should be connected to provide link-level redundancy. In a cluster configuration, the master and slave slots are identified as primary and subordinate.

In addition, a cluster configuration actively enhances failover recovery time for redundant virtual interface (VIF) connections. When an adapter has an active VIF connection to one fabric interconnect and a standby VIF connection to the second, the learned MAC addresses of the active VIF are replicated but not installed on the second fabric interconnect. If the active VIF fails, the second fabric interconnect installs the replicated MAC addresses and broadcasts them to the network through gratuitous ARP messages, shortening the switchover time.


Note


The cluster configuration provides redundancy only for the management plane. Data redundancy is dependent on the user configuration and might require a third-party tool to support data redundancy.