Migrating to Cisco UCS 6400 Series Fabric Interconnects

Cisco UCS 6200 Series to Cisco UCS 6400 Series Fabric Interconnect Migration

Software Feature Configuration

Cisco UCS 6400 Series Fabric Interconnects do not support a few software features that were supported on UCS 6200 Series Fabric Interconnects in Cisco UCS Manager 3.2 and earlier releases. The following sections detail each of these features and how they would be reported on the Migration Warnings page.

Ensure that the following features are configured correctly before migration:

Chassis Discovery Policy

UCS 6200 Series Fabric Interconnects support blade server chassis discovery in Port Channel and non-Port Channel modes. Cisco UCS 6400 Series Fabric Interconnects support only Port Channel mode.

During migration, if a UCS 6200 Series Fabric Interconnect has the chassis discovery policy configured as non-Port Channel mode, the Migration Warnings page will report the incompatibility.


Important

You must switch the chassis discovery policy to Port Channel mode before initiating migration and re-acknowledge the chassis after changing the chassis discovery policy.

Chassis Connectivity Policy

UCS 6200 Series Fabric Interconnects support chassis connectivity in Port Channel and non-Port Channel modes. Cisco UCS 6400 Series Fabric Interconnects support only Port Channel mode.

During migration, if a UCS 6200 Series Fabric Interconnect has the chassis connectivity policy configured as non-Port Channel mode, the Migration Warnings page will report the incompatibility.


Important

You must switch the chassis connectivity policy to Port Channel mode before initiating migration and re-acknowledge the chassis after changing the chassis connectivity policy.

Port Configuration Mismatch

On Cisco UCS 6400 Series Fabric Interconnects, the Unified Port capability is restricted to first 16 ports. Only ports 1/1-1/16 can be configured as FC. The FC ports must be contiguous, followed by contiguous Ethernet ports.

On UCS 6200 Series Fabric Interconnects, all ports have the Unified Port capability. All ports can be configured as Ethernet or FC. The Ethernet ports must be contiguous, followed by contiguous FC ports. FC ports appear towards the end of the module.

During cluster addition, the ports that are mismatched will be un-configured.

Multicast Hardware Hash

The Cisco UCS 6400 series Fabric Interconnect supports multicast hardware hashing by default and no configuration is required. However, Cisco UCS 6400 series Fabric Interconnect does not support the provisioning option while enabling multicast hardware hashing. Cisco UCS 6200 series Fabric Interconnects supports the provisioning option while enabling multicast hardware hashing.

VLAN Port Count Optimization

On Cisco UCS 6400 Series Fabric Interconnects, VLAN port count optimization is performed through port VLAN (VP) grouping when the PV count exceeds 16000.

The following table illustrates the PV Count with VLAN port count optimization enabled and disabled on Cisco UCS 6200 and Cisco UCS 6400 Series Fabric Interconnects.

6200 Series FI

 6400 Series FI

PV Count with VLAN Port Count Optimization Disabled

32000

16000

PV Count with VLAN Port Count Optimization Enabled

64000

64000

If the PV count exceeds 16K, the Migration Warnings page will report the PV count with the warning that VP Grouping will be enabled if you choose to proceed with installation.

When the Cisco UCS 6454 Fabric Interconnect is in Ethernet switching mode:

  • The Fabric Interconnect does not support VLAN Port Count Optimization Enabled

  • The Fabric Interconnect supports 16000 PVs, similar to EHM mode, when set to VLAN Port Count Optimization Disabled

Multicast Optimized for QoS

Cisco UCS 6400 Series Fabric Interconnects do not support Multicast Optimized. If Multicast Optimized is enabled, this configuration will appear on the Migration Warning page.

Continuing with the installation despite the warning will cause Cisco UCS Manager to reset the Multicast Optimized field.

NetFlow Configuration

UCS 6200 Series Fabric Interconnects support NetFlow configuration. However, on Cisco UCS 6400 Series Fabric Interconnects, NetFlow is not supported.

During migration, if a UCS 6200 Series Fabric Interconnect has NetFlow enabled, the Migration Warnings page will report the NetFlow configuration. Proceeding with the migration will remove the NetFlow configuration.

VM-FEX Integration

Cisco UCS 6400 Series Fabric Interconnects do not support VM-FEX integration.

During migration, the Migration Warnings page will report all configurations related to port profiles and distributed virtual switches (DVSes). Proceeding with the migration with this configuration will remove these port profiles and DVSes from the Cisco UCS Manager configuration.

Dynamic vNIC Connection Policies

During migration, if a UCS 6200 Series Fabric Interconnect has dynamic vNIC connection policies configured, the Migration Warnings page will report that such connection policies exist. If you proceed with the migration, these policies and dynamic vNICS are automatically deleted.

Reserved VLANs

Cisco UCS 6400 Series Fabric Interconnects reserve more VLANs for internal use than UCS 6200 Series Fabric Interconnects.

During migration, the Migration Warnings page will contain the list of VLANs that could potentially conflict with the default reserved VLAN range. If you proceed with migration, the Reserved VLAN range will be configured but VLANs found in the conflicting range will not be configured.

Cisco UCS 6400 Series Fabric Interconnect Migration Considerations

Beginning with Cisco UCS Manager release versions 4.0 and 4.1, you can migrate from Cisco UCS 6200 Series Fabric Interconnects to Cisco UCS 6454 and 64108 Fabric Interconnects respectively.

To migrate from Cisco UCS 6200 Series Fabric Interconnects to Cisco UCS 6400 Series Fabric Interconnects:

  • Cisco UCS 6200 Series Fabric Interconnects must be on Cisco UCS Manager Release 4.1(1) or a later release to migrate to Cisco UCS 64108 Fabric Interconnects, and Release 4.0(1) or a later release to migrate to Cisco UCS 6454 Fabric Interconnects.

  • Cisco UCS 6400 Series Fabric Interconnects must be loaded with the same infrastructure firmware version as on the Cisco UCS 6200 Series Fabric Interconnect that it will replace.

Prerequisites

Before performing the migration from Cisco UCS 6200 series fabric interconnects to Cisco UCS 6400 series fabric interconnects, ensure that the following prerequisites are met for a successful migration:

  • Back up and export Cisco UCS Manager configuration before initiating the upgrade. Ensure that the backup file from the 6200 Series Fabric Interconnects is compatible with the 6400 Series Fabric Interconnects. You can follow any of the backup methods:

  • Take an inventory of the Cisco UCS domain and remove any unsupported hardware.

  • Ensure to enable the cluster failover.

  • Do not attempt to implement new software features from the new Cisco UCS software version until all required hardware is installed.

  • Validate the software features and configurations between Cisco UCS 6200 series and UCS 6400 series. Ensure that the 6200 series fabric interconnect is reconfigured to only have features that are supported with 6400 series fabric interconnect.


    Note

    This is one of the most common reasons for migration failures.

  • Make sure both Cisco UCS 6200 series and 6400 series fabric Interconnects are on the same UCSM build before migration.

  • Licenses from Cisco UCS 6200 series fabric interconnects are not transferable to Cisco UCS 6400 series fabric interconnects.

  • Standalone installations should expect down time. In a cluster configuration, migrating the fabric interconnects can result in a small traffic disruption when the traffic fails over from one Fabric Interconnect to another. To avoid that there is no permanent traffic loss during migration, ensure that there is redundancy in the UCS domain on both fabric interconnects before migration and test the redundancy before starting the migration.

  • Ensure that the fill pattern is set as IDLE on the corresponding FC switch ports and the direct-attached FC storage array ports when configuring FC Uplink Ports or FC Storage Ports at 8Gbps speed. If the fill pattern is not set as IDLE, FC Uplink ports and FC Storage ports operating at 8 Gbps might go to an errDisabled state, lose SYNC intermittently, or receive errors or bad packets.

    Cisco UCS 6400 series fabric interconnect supports 8 Gbps only with fill-pattern set to IDLE for direct-attached FC connectivity (FC uplink ports or FC storage ports). This limitation is not applicable for Cisco UCS 6400 series fabric interconnects with Fibre Channel (FC) ports at 16 Gbps and 32 Gbps. When migrating to Cisco UCS 6400 series fabric interconnects for direct-attached storage arrays that don’t support IDLE fill-pattern at 8 Gbps do one of the following:

    • Use a SAN switch between the Cisco UCS 6400 series fabric interconnects and the storage array with 8 GB FC connectivity.

    • Upgrade the storage array to 16 GB or 32 GB FC connectivity.

  • Ensure the latest firmware bundle is downloaded and upgraded through GUI or CLI. Incase of attempting to upgrade the firmware bundle using other methods (loader prompt/erase configuration) can result in missing package version.

  • Before migration, make sure that the FC Speed is 8Gbps on Cisco UCS 6200 fabric interconnects or the connected switch supports 8Gbps speed.


    Note

    Post migration, you can configure the FC Port (Scalability Port) speed on Cisco UCS 6400 fabric interconnects.

  • Migrating to different IOM models can result in peer communication issue between IOMs of the primary and secondary fabric interconnects.

  • Make a detailed record of the cabling between FEX and fabric interconnects. You must preserve the physical port mapping to maintain the server pinning already configured and minimize down time.

  • For a cluster configuration, both fabric interconnects must have symmetrical connection topologies between fabric interconnect and FEX.

  • Use the same speed cables on all the adapter ports that are connected to same Fabric Interconnect. Cisco UCS VIC adapter ports connected to Cisco UCS 6400 series fabric interconnect through a mix of 10G and 25G cables can result in UCS rack-mount server discovery failure and ports moving to suspended state.

  • A WWN pool can include only WWNNs or WWPNs in the ranges from 20:00:00:00:00:00:00:00 to 20:FF:00:FF:FF:FF:FF:FF or from 50:00:00:00:00:00:00:00 to 5F:FF:00:FF:FF:FF:FF:FF. All other WWN ranges are reserved. When fibre channel traffic is sent through the UCS infrastructure the source WWPN is converted to a MAC address. You cannot use WWPN pool which can translate to source multicast MAC addresses. To ensure the uniqueness of the Cisco UCS WWNNs and WWPNs in the SAN fabric, Cisco recommends using the following WWN prefix for all blocks in a pool: 20:00:00:25:B5:XX:XX:XX

  • Unconfigure the fibre channel ports on the migrating subordinate Cisco UCS 6200 series Fabric Interconnect and reconfigure on the Cisco UCS 6400 series fabric interconnects.

  • When migrating from Cisco UCS 6296 Fabric Interconnect to Cisco UCS 64108 Fabric Interconnect, the ports on the 64108 Fabric Interconnect that are connected to the extended module on the 6296 Fabric Interconnect must be reacknowledged.

  • Before migrating from Cisco UCS 6200 series fabric interconnects to Cisco UCS 6400 Series fabric interconnects, ensure that you unconfigure the unified ports on the Cisco UCS 6200 Series fabric interconnects.

    After migrating to Cisco UCS 6400 series fabric interconnects, reconfigure the unified ports based on their location on the Cisco UCS 6400 series fabric interconnects, and reacknowledge the newly configured ports. For example, a unified port on a UCS 6248 fabric interconnect should be reconfigured on any port between 1 and 16 on a Cisco UCS 6400 series fabric interconnect.


    Note

    You should login directly to the new Cisco UCS 6454 Fabric Interconnect GUI to configure the unified ports during the migration.

  • Upgrading the fabric interconnect should be done before upgrading to a new FEX or virtual interface card.

Recommendations

Following are the best practices for a successful migration:

  • For minimal disruption during migration, ensure that there is redundancy for Ethernet and FC traffic from the servers in the UCS domain across both 6200 series fabric interconnects before migration.

  • Changes to the topology, such as the number of servers or uplink connections, should be performed after the fabric interconnect migration is complete.

  • During the migration of Fabric Interconnects, ensure the Cluster ID is not changed.

  • During the migration, image synchronization between fabric interconnects is not allowed. This is done to prevent incompatible images from getting synchronized. It is necessary to download B-Series, C-Series, and S-Series server software bundles again after migration is complete.

  • During the migration, ensure that VLAN is not created in the range of 3915 to 4042 which are the reserved VLAN range for Cisco UCS 6400 series fabric interconnects.

  • After migrating from Cisco UCS 6200 to 6400 series fabric interconnects, ensure to assign unique VLAN IDs while reconfiguring the Fibre Channel (FC) ports. It is critical to prevent the virtual SAN's FCoE VLAN IDs from overlapping with the Cisco UCS 6400 fabric interconnect's reserved VLAN IDs, as this overlap can result the FC ports to shut down (Admin down state).

  • The Cisco UCS 6400 series fabric interconnect supports only port-channel mode for chassis-discovery. Hence, change the Chassis/FEX discovery policy on the Cisco UCS 6200 series Fabric Interconnect to port-channel and immediately re-acknowledge the Cisco UCS 5108 chassis.


    Note

    The migration can fail when the chassis is not immediately re-acknowledged.

Important

For more information on migrating Cisco UCS 6200 Series Fabric Interconnects to Cisco UCS 6400 Series Fabric Interconnect with UCS Central, see Considerations for migrating Cisco UCS 6200 Series Fabric Interconnects to Cisco UCS 6400 Series Fabric Interconnects with Cisco UCS Central.

Considerations for migrating Cisco UCS 6200 Series Fabric Interconnects to Cisco UCS 6400 Series Fabric Interconnects with Cisco UCS Central

To successfully migrate from Cisco UCS 6200 Series to Cisco UCS 6400 Series Fabric Interconnects using Cisco UCS Central, ensure the following prerequisites are met:

  • Backup Configurations: Complete a full backup of Cisco UCS Manager and UCS Central configurations before initiating the migration.

  • Chassis/FEX Discovery Policy

    • Set the Link Grouping Preference to PORT CHANNEL .

    • Set the Multicast Hardware Hash to DISABLED.

  • Dynamic vNIC Connection Policy: Ensure the Dynamic vNIC Connection Policy is not applied.

  • QOS System Class: Disable the QOS System Class.

  • Policy Resolution Control: To avoid any configuration issues during migration, ensure these policies are set to Local in Cisco UCS Manager and maintain this setting post-migration. Changing the settings from Local to Global after migration may result in configuration issues.

    • Infrastructure and Catalog Firmware Policy

    • Equipment Policy

    • Port Configuration Policy

  • Reserved VLAN IDs: Ensure the reserved VLAN IDs are not utilized to avoid any issues.


Note

For any additional information, see Cisco UCS 6400 Series Fabric Interconnect Migration Considerations.

Validating Feature Configurations for Cisco UCS 64108 Fabric Interconnect before Upgrade

Table 1. Features that needs special attention prior to upgrading

Feature

Remediation

Chassis and fabric extender I/O port channel

Select a port channel to the I/O module (IOM).

Multicast optimization

Verify that multicast optimization is not enabled under the LAN quality-of-service (QoS) system classes

Fabric forwarding mode for Ethernet

Verify that the Ethernet forwarding mode is set to End Host Mode Only.

Fabric forwarding mode for Fibre Channel

Verify that Fibre Channel forwarding mode is set to End Host Mode or FC Switching Mode.

Cisco NetFlow

Unconfigure NetFlow.

MAC Security

Select Allow for MAC security.

VM-FEX

Remove port profiles and Cisco UCS Manager ESXi or SCVMM related configurations.

Dynamic vNIC connection policies

Set the dynamic vNIC connection policy in the vNIC profile to Not set.

Cisco Switched Port Analyzer (SPAN)

Use receive (RX) direction only. The installer will change SPAN to the RX direction and send an alert indicating that this setting is being changed.

Failure to comply with these remediation steps will result in a migration warning alert during the migration process and prevent the fabric interconnects from synchronizing.

Port Mapping for Upgrades

The upgrade described here is primarily for upgrading a Cisco UCS 6248 Fabric Interconnect to a Cisco UCS 6400 Series Fabric Interconnect. The same considerations will also apply when upgrading a Cisco UCS 6296 Fabric Interconnect to a Cisco UCS 6400 Series Fabric Interconnect. When migrating from Cisco UCS 6296 Fabric Interconnect to Cisco UCS 64108 Fabric Interconnect, the ports on the 64108 Fabric Interconnect that are connected to the extended module on the 6296 Fabric Interconnect must be reacknowledged.


Note

If you are using appliance ports for direct attached storage, you must add VLANs to the ethernet uplinks. This will ensure that vNICS can properly pin on boot.

Fixed Ports

On the UCS 6248 Fabric Interconnect, you can separate the 32 physical ports in slot one into two contiguous pools, low numbered ports being Ethernet ports and high numbered ports being Fibre Channel ports. On the UCS 6400 Series Fabric Interconnect, you can configure the first 16 ports as Fibre Channel ports.

Because a UCS 6248 Fabric Interconnect has 32 ports in slot 1 and a UCS 6400 Series Fabric Interconnect has all ports in slot 1, any ports on GEM slots will be removed during the hardware upgrade process.


Caution

If you ever need to change the pool sizes for slot 1, you must reboot the Fabric Interconnect which can lead to a service disruption. If you ever need to change the pool sizes for slot 2, you must reset the expansion module in slot 2. To minimize disruption, plan to have at least a few Ethernet uplink and Fibre Channel uplink ports configured on slot 1. Implement this fail safe after the upgrade is complete and the system restabilizes.

Figure 1. Cisco UCS 64108 Port Numbering

1

Ports 1-16

Unified ports:

  • 10/25 Gbps Ethernet or FCoE

  • 8/16/32 Gbps Fibre Channel

2

Ports 1-96

(10/25 Gbps Ethernet or FCoE)

3

Uplink Ports 97-108 (40/100 Gbps Ethernet or FCoE)

Each of these ports can be 4 x 10/25 Gbps Ethernet or FCoE uplink ports when using a breakout cable.

4

Ports 89-96

  • 10/25 Gbps Ethernet or FCoE

  • 1 Gbps Ethernet

5

System environment (fan fault) LED

6

System status LED

7

Beacon LED
Figure 2. Cisco UCS 6454 Port Numbering

1

Ports 1-16 (Unified Ports 10/25 Gbps Ethernet or FCoE or 8/16/32 Gbps Fibre Channel)

Note

 

When using Cisco UCS Manager releases earlier than 4.0(4), only ports 1-8 are Unified Ports.

2

Ports 17-44 (10/25 Gbps Ethernet or FCoE)

Note

 

When using Cisco UCS Manager releases earlier than 4.0(4), ports 9-44 are 10/25 Gbps Ethernet or FCoE.

3

Ports 45-48 (1/10/25 Gbps Ethernet or FCoE)

4

Uplink Ports 49-54 (40/100 Gbps Ethernet or FCoE)

Each of these ports can be 4 x 10/25 Gbps Ethernet or FCoE uplink ports when using an appropriate breakout cable.

Migrating from UCS 6200 Series Fabric Interconnects to UCS 6400 Series Fabric Interconnects

Cisco UCS Manager supports migration of Cisco UCS 6200 Series Fabric Interconnects to Cisco UCS 6400 Series Fabric Interconnects.

Cisco UCS 6200 Series Fabric Interconnects include:

  • Cisco UCS 6248UP 48-Port Fabric Interconnect

  • Cisco UCS 6296UP 96-Port Fabric Interconnect

Cisco UCS 6400 Series Fabric Interconnects include:

  • Cisco UCS 6454 Fabric Interconnect

  • Cisco UCS 64018 Fabric Interconnect

Cisco UCS 6200 series Fabric Interconnect must be on Cisco UCS Manager Release 4.0(1) or a later release version to migrate to Cisco UCS 6454 Fabric Interconnect, and Release 4.1(1) or a later release version to migrate to Cisco UCS 64108 Fabric Interconnects. After migrating to UCS 6454 or UCS 64108 Fabric Interconnect, Cisco recommends not to migrate back to any of the 6200 Series Fabric Interconnects.

The Cisco UCS 6400 Series Fabric Interconnect supports only port-channel mode for chassis-discovery. On changing the chassis or FEX discovery policy to port-channel, the chassis needs to be re-acknowledged before proceeding with the migration.

Ensure that you enter the show fex detail command on the Fabric Interconnect (NXOS mode) to confirm that all the FEXes or IOMs are in a port-channel even if the port-channel mode is configured in the UCS Manager. If any FEX or IOM is not in a port-channel, then you must re-acknowledge the FEX or IOM.

To acknowledge the chassis in Cisco UCS Manager, do the following:

  1. In the Navigation pane of Cisco UCS Manager, click Equipment.

  2. Click the Equipment node.

  3. In the Work pane, click the Policies tab.

  4. Click the Global Policies subtab.

  5. In the Chassis/FEX Discovery Policy area, set the Link Grouping Preference field to Port Channel.

  6. Expand Equipment > Chassis, and choose the chassis that you want to acknowledge.

  7. In the Work pane, click the General tab.

  8. In the Actions area, click Acknowledge Chassis.


Note

For more information about how to perform configuration procedures in Cisco UCS Manager, see the appropriate Cisco UCS Manager Configuration Guide.

Procedure

Step 1

Move the traffic to the primary fabric interconnect(Cisco UCS 6454 or UCS 64108). This can be performed in two ways: fabric evacuation and uplink disablement. Use the below flowchart to choose between the two based on your Cisco UCS domain server and connectivity.

Note

 

For direct-attached rack servers, only uplink disablement is supported.

Step 2

Verify that all traffic has failed over to the primary fabric interconnect. Unconfigure all the server ports or fibre channel ports on the subordinate fabric interconnect.

Note

 

For more information, see the Fabric Interconnect Traffic Evacuation section in the Guidelines and Prerequisites chapter of the Cisco UCS Manager Firmware Management Guide.

Step 3

Power down the UCS 6200 series subordinate fabric interconnect and disconnect the power and the L1/L2 cables.

Step 4

Mount the replacement, UCS 6400 series fabric interconnect into either the same rack or an adjacent rack.

Note

 

As a best practice, you should label the cables.

Step 5

Connect the L1/L2 cables and the server ports according to your port mapping plan.

Step 6

Power up the new fabric interconnect. If it is connected correctly, the new subordinate fabric interconnect will detect that it is being added to an existing cluster.

Step 7

Enter the IP address information for the new subordinate fabric interconnect.

Step 8

Unified ports on the 6400 series fabric interconnect are different from the unified port ordering on the 6200 series fabric interconnect.

Note

 

  • When you convert from ethernet ports to fibre channel ports, a reboot is required.

  • For more information on Configuring FC Uplink port or Converting FC Storage Port to FC Uplink port, refer the LAN Ports and Port Channels chapter in Cisco UCS Manager Network Management Guide.

Step 9

Configure the network uplink ports on the new Cisco UCS 6400 series fabric interconnect.

Step 10

Match the old configuration for the port-channel. Add uplink ports to the necessary port-channel or any other previous configuration required for the port-channel. Wait for configuration to complete before proceeding to the next step.

Note

 

Waiting to enable the server ports prevents the svc_sam_bladeAG service from communicating with chassis and servers. In previous migrations, when enabling server ports at the same time as the uplink ports, it would cause topping out (pinning) the CPU to near 100% on the primary fabric interconnect. When there is high CPU usage, the user interface becomes unresponsive and the svc_sam_bladeAG service must be restarted to recover.

Step 11

Reconfigure the server ports or fibre channel ports.

  1. If you have changed port mappings, you may need to reacknowledge the IOM, FEX, or direct-connect rack server connected to the subordinate fabric interconnect.

  2. Verify and if necessary, you can reconfigure ethernet ports as server ports.

Step 12

The Cisco UCS 6400 series subordinate fabric interconnect will automatically synchronize the configuration and database/state information from the primary Cisco UCS 6200 series fabric interconnect.

Synchronization between primary and subordinate fabric interconnects can take several minutes. You may see an error message that will persist until the server ports are enabled.

The port configuration is copied from the subordinate fabric interconnect to the new fabric interconnect.

Step 13

Reconfigure the Ethernet ports, Fibre Channel ports, or unified ports.

  1. If you have changed port mappings for direct-attach rack server, reacknowledge the server.

  2. It is recommended to reacknowledge the IOM or FEX.

Step 14

Verify that the data path is ready and all faults are resolved before proceeding with the next steps.

For more information, see the Verifying that Dynamic vNICs Are Up and Running section in the Guidelines and Prerequisites chapter of the Cisco UCS Manager Firmware Management Guide.

  1. Verify, and if necessary, reconfigure the SAN pin group for Fibre Channel ports in the associated service profile.

  2. Verify, and if necessary, reconfigure the LAN pin group for Ethernet ports in the associated service profile.

  3. Verify, and if necessary, reconfigure the port-channel for uplink Ethernet ports.

Step 15

Move the traffic back to the new subordinate fabric interconnect. If you used the fabric evacuation method, then deselect fabric evacuation. If you disabled the uplinks (Ethernet and Fibre Channel), then reenable the uplinks. Verify that traffic is flowing correctly on the new subordinate fabric interconnect.

Step 16

After verifying that traffic is flowing on the subordinate fabric interconnect, promote the subordinate fabric interconnect to primary using the below commands:

  • UCS-A #connect local-mgmt: This command connects to the local management interface of the cluster.

  • UCS-A (local-mgmt) #cluster{lead {a|b}} or UCS-A (local-mgmt) #cluster{force primary{a|b}}: The cluster lead command and cluster force primary command are two separate commands that can be used for promoting the fabric interconnect.

Note

 

Ensure that the primary fabric interconnect is one of the Cisco UCS 6400 series fabric interconnects.

Step 17

Cable the second new fabric interconnect identically to the first and repeat the steps for the other fabric interconnect replacements to complete the migration.

Migrating Offline from Cisco UCS 6200 Series Fabric Interconnect to Cisco UCS 6400 Series Fabric Interconnects

Offline migration involves transitioning the configurations and settings from Cisco UCS 6200 Series Fabric Interconnect to Cisco UCS 6400 Series Fabric Interconnect, while the system is offline. This approach includes taking back up of the current configuration, decommissioning the components, and transferring the configurations to the new Fabric Interconnect.

Procedure

Step 1

Use one of the supported protocols (SCP, TFTP, FTP, SFTP) to perform a remote backup of all configurations from the source Fabric Interconnect.

Step 2

Decommission the connected servers and unconfigure the ports on Cisco UCS 6200 Series Fabric Interconnect.

Step 3

Transfer the ports and preserved port mappings to the Cisco UCS 6400 Series Fabric Interconnect.

Note

 

Ensure to use compatible cables or transceivers for the connection.

Step 4

Import the remote backup file and restore the configuration on Cisco UCS 6400 Series Fabric Interconnect.

Step 5

Validate that the servers automatically restore their previous configurations upon reconnection.

Step 6

Verify that all configurations, including operating system installations that are preserved and functioning as expected.

Cisco UCS 6454 Fabric Interconnect to Cisco UCS 64108 Fabric Interconnect Migration

Cisco UCS 6400 Series Fabric Interconnect Migration Considerations

Cisco UCS Manager provides support for migrating Cisco UCS 6454 Fabric Interconnect to Cisco UCS 64108 Fabric Interconnect with B-Series servers, C-Series, or S-Series servers.

To migrate from Cisco UCS 6454 Fabric Interconnect to Cisco UCS 64108 Fabric Interconnect, both the Fabric Interconnect must be loaded with the same Infrastructure Firmware version.

Prerequisites

Before performing the migration from Cisco UCS 6454 Fabric Interconnects to Cisco UCS 64108 Fabric Interconnect, ensure that the following prerequisites are met for a successful migration:

  • Back up and export Cisco UCS Manager configuration before initiating the upgrade.

  • Take an inventory of the Cisco UCS domain and remove any unsupported hardware.

  • Ensure to enable the cluster failover.

  • Do not attempt to implement new software features from the new Cisco UCS software version until all required hardware is installed.

  • Make sure both Cisco UCS 6400 series Fabric Interconnects are on the same UCSM build before migration.

  • Standalone installations should expect down time. In a cluster configuration, migrating the Fabric Interconnects can result in a small traffic disruption when the traffic fails over from one Fabric Interconnect to another. To avoid that there is no permanent traffic loss during migration, ensure that there is redundancy in the UCS domain on both Fabric Interconnects before migration and test the redundancy before starting the migration.

  • Ensure the latest firmware bundle is downloaded and upgraded through GUI or CLI. Incase of attempting to upgrade the firmware bundle using other methods (loader prompt/erase configuration) can result in missing package version.

  • Before migration, make sure that the FC Speed is 8Gbps on Cisco UCS 6454 Fabric Interconnects or the connected switch supports 8Gbps speed.


    Note

    Post migration, you can configure the FC Port (Scalability Port) speed on Cisco UCS 64108 Fabric Interconnects.

  • Migrating to different IOM models can result in peer communication issue between IOMs of the Primary and Secondary Fabric Interconnects.

  • Make a detailed record of the cabling between FEX and fabric interconnects. You must preserve the physical port mapping to maintain the server pinning already configured and minimize down time.

  • For a cluster configuration, both fabric interconnects must have symmetrical connection topologies between fabric interconnect and FEX.

  • Use the same speed cables on all the adapter ports that are connected to same Fabric Interconnect. Cisco UCS VIC adapter ports connected to Cisco UCS 64108 fabric interconnect through a mix of 10G and 25G cables can result in UCS rack-mount server discovery failure and ports moving to suspended state.

  • A WWN pool can include only WWNNs or WWPNs in the ranges from 20:00:00:00:00:00:00:00 to 20:FF:00:FF:FF:FF:FF:FF or from 50:00:00:00:00:00:00:00 to 5F:FF:00:FF:FF:FF:FF:FF. All other WWN ranges are reserved. When fibre channel traffic is sent through the UCS infrastructure the source WWPN is converted to a MAC address. You cannot use WWPN pool which can translate to source multicast MAC addresses. To ensure the uniqueness of the Cisco UCS WWNNs and WWPNs in the SAN fabric, Cisco recommends using the following WWN prefix for all blocks in a pool: 20:00:00:25:B5:XX:XX:XX

Recommendations

Following are the best practices for a successful migration:

  • For minimal disruption during migration, ensure that there is redundancy for Ethernet and FC traffic from the servers in the UCS domain across both 6454 Fabric Interconnects before migration.

  • Changes to the topology, such as the number of servers or uplink connections, should be performed after the fabric interconnect migration is complete.

  • During the migration of Fabric Interconnects, ensure the Cluster ID is not changed.

  • During the migration, image synchronization between fabric interconnects is not allowed. This is done to prevent incompatible images from getting synchronized. It is necessary to download B-Series, C-Series, and S-Series server software bundles again after migration is complete.

  • Unconfigure the fibre channel ports on the migrating subordinate Cisco UCS 6454 Fabric Interconnect and reconfigure on the Cisco UCS 64108 Fabric Interconnect.


    Note

    For more information on migrating Cisco UCS 6454 Fabric Interconnect to Cisco UCS 64108 Fabric Interconnect with UCS Central, see Considerations for migrating Cisco UCS 6454 Fabric Interconnects to Cisco UCS 64108 Fabric Interconnects with Cisco UCS Central.

Validating Feature Configurations for Cisco UCS 64108 Fabric Interconnect before Upgrade

Table 2. Features that needs special attention prior to upgrading

Feature

Remediation

Chassis and fabric extender I/O port channel

Select a port channel to the I/O module (IOM).

Multicast optimization

Verify that multicast optimization is not enabled under the LAN quality-of-service (QoS) system classes

Fabric forwarding mode for Ethernet

Verify that the Ethernet forwarding mode is set to End Host Mode Only.

Fabric forwarding mode for Fibre Channel

Verify that Fibre Channel forwarding mode is set to End Host Mode or FC Switching Mode.

Cisco NetFlow

Unconfigure NetFlow.

MAC Security

Select Allow for MAC security.

VM-FEX

Remove port profiles and Cisco UCS Manager ESXi or SCVMM related configurations.

Dynamic vNIC connection policies

Set the dynamic vNIC connection policy in the vNIC profile to Not set.

Cisco Switched Port Analyzer (SPAN)

Use receive (RX) direction only. The installer will change SPAN to the RX direction and send an alert indicating that this setting is being changed.

Failure to comply with these remediation steps will result in a migration warning alert during the migration process and prevent the fabric interconnects from synchronizing.

Considerations for migrating Cisco UCS 6454 Fabric Interconnects to Cisco UCS 64108 Fabric Interconnects with Cisco UCS Central

In addition to Cisco UCS 6400 Series Fabric Interconnect Migration Considerations, consider the following prerequisites when migrating with Cisco UCS Central:

  • Before initiating the migration, ensure to have a complete backup of Cisco UCS Manager and UCS Central configurations.

  • To avoid any configuration issues during migration, make sure the following policies on Policy Resolution Control is set to Local in Cisco UCS Manager:

    • Infrastructure and Catalog Firmware Policy

    • Equipment Policy

    • Port Configuration Policy

Migrating from UCS 6454 Fabric Interconnects to UCS 64108 Fabric Interconnects

Beginning with Cisco UCS Manager Release 4.1, you can migrate from Cisco UCS 6454 Fabric Interconnect to Cisco UCS 64108 Fabric Interconnect.

To acknowledge the chassis in Cisco UCS Manager, do the following:

  1. In the Navigation pane of Cisco UCS Manager, click Equipment.

  2. Click the Equipment node.

  3. In the Work pane, click the Policies tab.

  4. Click the Global Policies subtab.

  5. In the Chassis/FEX Discovery Policy area, set the Link Grouping Preference field to Port Channel.

  6. Expand Equipment > Chassis, and choose the chassis that you want to acknowledge.

  7. In the Work pane, click the General tab.

  8. In the Actions area, click Acknowledge Chassis.


Note

For more information about how to perform configuration procedures in Cisco UCS Manager, see the appropriate Cisco UCS Manager Configuration Guide.

Procedure

Step 1

Move the traffic to the primary fabric interconnect (Cisco UCS 6454). This can be performed in two ways: fabric evacuation and uplink disablement. Use the below flowchart to choose between the two based on your Cisco UCS domain server and connectivity.

Note

 

For direct-attached rack servers, only uplink disablement is supported.

Step 2

Verify that all traffic has failed over to the primary fabric interconnect. Unconfigure all the server ports or fibre channel ports on the subordinate fabric interconnect.

Note

 

For more information, see the Fabric Interconnect Traffic Evacuation section in the Guidelines and Prerequisites chapter of the Cisco UCS Manager Firmware Management Guide.

Step 3

Power down the UCS 6454 subordinate fabric interconnect and disconnect the power and the L1/L2 cables.

Step 4

Mount the replacement UCS 64108 Fabric Interconnect into either the same rack or an adjacent rack.

Note

 

As a best practice, you should label the cables.

Step 5

Connect the L1/L2 cables and the server ports according to your port mapping plan.

Step 6

Power up the new fabric interconnect. If it is connected correctly, the new subordinate fabric interconnect will detect that it is being added to an existing cluster.

Step 7

Enter the IP address information for the new subordinate fabric interconnect.

Step 8

Unified ports on the UCS 6454 fabric interconnect are similar to the unified port ordering on the 64108 fabric interconnect.

Note

 

  • When you convert from ethernet ports to fibre channel ports, a reboot is required.

  • For more information on Configuring FC Uplink port or Converting FC Storage Port to FC Uplink port, refer the LAN Ports and Port Channels chapter in Cisco UCS Manager Network Management Guide.

Step 9

Configure the network uplink ports on the new Cisco UCS 64108 fabric interconnect.

Step 10

Match the old configuration from 6454 for the port-channel. Add uplink ports to the necessary port-channel or any other previous configuration required for the port-channel. Wait for configuration to complete before proceeding to the next step.

Note

 

Waiting to enable the server ports prevents the svc_sam_bladeAG service from communicating with chassis and servers. In previous migrations, when enabling server ports at the same time as the uplink ports, it would cause topping out (pinning) the CPU to near 100% on the primary fabric interconnect. When there is high CPU usage, the user interface becomes unresponsive and the svc_sam_bladeAG service must be restarted to recover.

Step 11

Reconfigure the server ports or fibre channel ports.

  1. If you have changed port mappings, you may need to reacknowledge the IOM, FEX, or direct-connect rack server connected to the subordinate fabric interconnect.

  2. Verify and if necessary, you can reconfigure ethernet ports as server ports.

Step 12

The 64108 subordinate fabric interconnect will automatically synchronize the configuration and database/state information from the primary UCS 6454 fabric interconnect.

Synchronization between primary and subordinate fabric interconnects can take several minutes. You may see an error message that will persist until the server ports are enabled.

The port configuration is copied from the subordinate fabric interconnect to the new fabric interconnect.

Step 13

Reconfigure the Ethernet ports, Fibre Channel ports, or unified ports.

  1. If you have changed port mappings for direct-attach rack server, reacknowledge the server.

  2. It is recommended to reacknowledge the IOM or FEX.

Note

 

You should login directly to the new Cisco UCS 6454 Fabric Interconnect GUI to configure the unified ports during the migration.

Step 14

Verify that the data path is ready.

For more information, see the Verifying that Dynamic vNICs Are Up and Running section in the Guidelines and Prerequisites chapter of the Cisco UCS Manager Firmware Management Guide.

Ensure that all faults are resolved before proceeding with next step.

  1. Verify, and if necessary, reconfigure the SAN pin group for Fibre Channel ports in the associated service profile.

  2. Verify, and if necessary, reconfigure the LAN pin group for Ethernet ports in the associated service profile.

  3. Verify, and if necessary, reconfigure the port channel for uplink Ethernet ports.

Step 15

Move the traffic back to the new subordinate fabric interconnect. If you used the fabric evacuation method, then deselect fabric evacuation. If you disabled the uplinks (Ethernet and Fibre Channel), then reenable the uplinks. Verify that traffic is flowing correctly on the new subordinate fabric interconnect.

Step 16

After verifying that traffic is flowing on the subordinate fabric interconnect, promote the subordinate fabric interconnect to primary using the below commands:

  • UCS-A #connect local-mgmt: This command connects to the local management interface of the cluster.

  • UCS-A (local-mgmt) #cluster{lead {a|b}} or UCS-A (local-mgmt) #cluster{force primary{a|b}}: The cluster lead command and cluster force primary command are two separate commands that can be used for promoting the fabric interconnect.

Note

 

Ensure that the primary fabric interconnect is Cisco UCS 64108 fabric interconnect.

Step 17

Cable the second new fabric interconnect identically to the first and repeat the steps for the other fabric interconnect replacement to complete the migration.