Migrate to Cisco UCS IOM 2408 White Paper

Product overview

The fourth-generation Cisco UCS^® I/O Module (IOM) 2408 provides unified fabric connectivity for the Cisco UCS 5100 Series Blade Server Chassis. IOM 2408 supports eight 25-Gigabit unified Ethernet ports to connect the Cisco UCS 5108 Blade Server Chassis with Cisco UCS Fabric Interconnect (FI) 6400 Series. Each IOM 2408 has four 10-Gigabit backplane Ethernet connectivity through the midplane to each half-width blade serer in the chassis, thus giving a total of thirty-two 10G backplane Ethernet interfaces from an IOM 2408 to the UCS chassis.

Cisco UCS I/O Module 2408, front view

Cisco UCS 5108 Blade Server Chassis with Cisco UCS I/O Module 2408 and 25GbE links to Cisco UCS Fiber Interconnect 6454

Note:      The Figure 2 is applicable for the Cisco UCS Fabric Interconnect 6400 series (6454 and 64108).

Objective

Cisco UCS 2408 I/O Module provide 25GbE connectivity for the Cisco UCS 5108 chassis towards the Fabric Interconnect 6400 series.

This document will provide detailed instructions for migrating the Cisco UCS 5108 Blase Server Chassis having Cisco UCS 2200 Series IOM (10GbE) with the new Cisco UCS 2408 IOM (25GbE).

Planning: prechecks

Table 1.           Supported migration paths

1.     Cisco UCS IOM 2408 connects only to the Fabric Interconnect 6400 series which come in two form factors, the one-rack-unit FI 6454 and the two-rack-unit FI 64108. Please follow the steps described in the Fabric Interconnect 6200 to 6400 migration guide to ensure that the UCS domains with IOM 2200 series have migrated to 6400 fabric interconnects: https://www.cisco.com/c/dam/en/us/products/collateral/servers-unified-computing/ucs-6454-fabric-interconnect.pdf.

2.     Ensure that the right 25-GbE transceivers and cables are used to connect the IOM 2408 with Fiber Interconnect 6400 series. Please refer to the IOM 2408 data sheet for this: https://www.cisco.com/c/en/us/products/collateral/servers-unified-computing/datasheet-c78-742624.html.

3.     IOM 2408 supports only the 4^th generation fabric interconnects and UCS blade servers with Cisco VIC 1300 and 1400 series adapters.

4.     Backup and export the Cisco UCS Manager (UCSM) configuration.

5.     Perform UCSM infrastructure upgrade to supported release – Fabric Interconnect and IOM. The FI 6454 supports IOM 2408 from 4.0(4e) release and above while the FI 64108 supports IOM 2408 from 4.1(1a) release and above.

6.     Before starting the IOM migration, ensure that the UCS domain is fully operational with no faults.

IOM migration steps

Step 1: From UCSM, validate that the fiber interconnect cluster high-availability redundancy state is present and there are no faults, and identify the subordinate Fiber Interconnect 6400 series in the UCS domain.

Fault check on UCS blade server chassis before migration

Cluster high-availability redundancy state

UCSM and firmware version check

Step 2: To have minimal traffic loss during migration, ensure that there are redundant paths from the blade servers over fabrics A and B. Since inflight packet loss is expected during the IOM migration, perform the next set of operations only during a maintenance window.

Step 3: On the subordinate FI-6400 series FI, unconfigure the server ports toward the migrating IOM 2200 series (2204 or 2208). On unconfiguring server ports, the primary fiber interconnect will take over traffic-forwarding for these migrating UCS chassis.

Unconfigure the server port on the fiber interconnect

Step 4: Disconnect the cables connecting the subordinate FI 6400 series and the corresponding IOM 220x’s from each migrating chassis.

Step 5: Remove and replace the migrating IOM 220x’s with IOM 2408s. Connect the IOM 2408 to the subordinate FI 6400 series with 25-gigabit Ethernet cables. At this point, the migrating UCS chassis will have a mix of IOM 220x and 2408.

Step 6: Configure server ports from the UCSM on the subordinate FI 6400 series ports connected to the IOM 2408s, one by one towards each UCS 5108 chassis. Verify that the 25GbE links come up between the IOMs and the fiber interconnects.

Step 7: Wait for the auto-upgrade of IOM 2408 to complete and traffic will resume over the subordinate fabric interconnect fabric after the IOM 2408 goes online.

IOM online check on the fiber interconnect console

Step 8: Ensure that the IOM 2408 has loaded the latest firmware. Verify that there are no faults on the UCS chassis with the new IOM 2408 and traffic has resumed over the migrated IOM 2408.

IOM firmware information on a migrating chassis after replacing IOM towards a subordinate fiber interconnect

High-availability redundancy state after migrating IOMs toward the subordinate fiber interconnect

Step 9: After completing IOM migration towards subordinate fiber interconnect, repeat steps 3 through 7 for replacing the other IOM 220x’s connected to the primary FI 6400 series and complete the IOM 2408 migration for the UCS domain.

Step 10: After completing the IOM 2408 migration on all the UCS 5108 chassis in the domain, verify that cluster high-availability is present on the fabric interconnect, there are no faults on the UCSM, and traffic is forwarding over the new IOM 2408s on both fabric A and B.

IOM firmware snapshot after migrating both IOMs

High-availability redundancy after completing migration

1.     Due to defect CSCvr46327, Fabric Interconnect 6454 have to be rebooted on replacing UCS IOM 2208 with IOM 2408. Without a reboot of FI 6454, traffic won’t forward over the IOM 2408 after migration. Both the subordinate and primary FI have to be reloaded when their respective IOM’s are replaced. Meanwhile, IOM 2204 to 2408 migration doesn’t require FI reload.

●     This issue is not present on 4.1(1) release but only on 4.0(4) release. Future 4.0(4) UCSM releases with the fix won’t require fabric interconnect reload for IOM 2408 migration.

●     To reboot Fabric Interconnect, user needs to SSH to the fabric interconnect virtual-IP, then execute “connect local-mgmt a or b,” and then issue reboot.

2.     RNE for UCSM 4.0(4e) https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/release/notes/CiscoUCSManager-RN-4-0.html#reference_gsv_thd_fjb