Migrating to Cisco UCS 6600 Series Fabric Interconnects

Cisco UCS 6600 Series Fabric Interconnect Migration

Migrating Cisco UCS Fabric Interconnects: 6500 to 6600 Series

Cisco UCS 6500 Series Fabric Interconnect Migration Considerations

Beginning with Cisco UCS Manager, Release 6.0(1b), you can migrate Cisco UCS 6500 Series Fabric Interconnect to Cisco UCS 6664 Fabric Interconnect with X-Series servers or C-Series rack servers.

To migrate from Cisco UCS 6500 Series Fabric Interconnect to Cisco UCS 6664 Fabric Interconnect:

  • Cisco UCS 6500 Series Fabric Interconnect must be on Cisco UCS Manager 6.0(1b) or a later release versions.

  • Cisco UCS 6664 Fabric Interconnect must be loaded with the same Infrastructure Firmware version that is on the Cisco UCS 6536 Fabric Interconnect that it will replace.

Prerequisites

Before performing the migration from Cisco UCS 6500 Series Fabric Interconnects to Cisco UCS 6664 Fabric Interconnect, ensure that the following prerequisites are met for a successful migration:

  • Back up and export Cisco UCS Manager configuration before initiating the upgrade.

  • Take an inventory of the Cisco UCS domain and remove any unsupported hardware.

  • Ensure to enable the cluster failover.

  • Do not attempt to implement new software features from the new Cisco UCS software version until all required hardware is installed.

  • Validate the software features and configurations between Cisco UCS 6500 series and UCS 6664 Fabric Interconnect. Before migration, ensure that 6500 series Fabric Interconnect is reconfigured to only have features that are supported with 6600 Series Fabric Interconnect.


    Note

    This is one of the common reasons for migration failures.

  • Make sure both Cisco UCS 6500 Series Fabric Interconnect and Cisco UCS 6600 Series Fabric Interconnect are on the same UCSM build before migration.

  • Licenses from Cisco UCS 6500 Series Fabric Interconnect are not transferable to Cisco UCS 6600 Series Fabric Interconnect. Cisco UCS 6600 Series Fabric Interconnect has perpetual license enabled by default.

  • Standalone installations should expect down time. In a cluster configuration, migrating the Fabric Interconnects can result in a small traffic disruption when the traffic fails over from one Fabric Interconnect to another. To avoid permanent traffic loss during migration, ensure that there is redundancy in the UCS domain on both Fabric Interconnects before migration and test the redundancy before starting the migration.

  • Ensure the latest firmware bundle is downloaded and upgraded through GUI or CLI. Incase of attempting to upgrade the firmware bundle using other methods (loader prompt/erase configuration) can result in missing package version.

  • Before migration, make sure that the FC Speed settings on Cisco UCS Fabric Interconnects are set to operate on a mutually supported value.

    • Cisco UCS 6500 Series Fabric Interconnect supported FC speeds: 8/16/32 Gbps

    • Cisco UCS 6600 Series Fabric Interconnect supported FC speeds: 16/32/64 Gbps

  • Ensure the IOM or IFM modules on both Primary and Secondary Fabric Interconnects match and are supported. Incompatible modules can result in peer communication issues.

  • Make a detailed record of the cabling between FEX and fabric interconnects. You must preserve the physical port mapping to maintain the server pinning already configured and minimize down time.

  • For a cluster configuration, both primary and secondary fabric interconnects must have symmetrical connection topologies between fabric interconnect and FEX.

  • Ensure all cables used for connecting servers, storage, and network devices to the Fabric Interconnect support the required speeds and are compatible with fabric interconnects.

  • A WWN pool in Cisco UCS can include only WWNNs or WWPNs within the ranges 20:00:00:00:00:00:00:00 to 20:FF:00:FF:FF:FF:FF:FF or 50:00:00:00:00:00:00:00 to 5F:FF:00:FF:FF:FF:FF:FF; all other WWN ranges are reserved.

    When Fibre Channel traffic passes through the UCS system, the source WWPN is converted to a MAC address. Avoid configuring WWPN pools that could translate into multicast MAC addresses. To ensure uniqueness in the SAN fabric, Cisco recommends using the WWN prefix 20:00:00:25:B5:XX:XX:XX for all pool blocks.

  • Enable Fabric Evacuation on the subordinate Cisco UCS 6536 Fabric Interconnect. After evacuation is complete, unconfigure all port roles and port channel configurations on the subordinate Fabric Interconnect, then configure them on the Cisco UCS 6664 Fabric Interconnect during migration.

Recommendations

Following are the best practices for a successful migration:

  • For minimal disruption during migration, ensure that there is redundancy for Ethernet and FC traffic from the servers in the UCS domain across both Cisco UCS 6500 Series Fabric Interconnect before migration.

  • Changes to the topology, such as the number of servers or uplink connections, should be performed after the fabric interconnect migration is complete.

  • During the migration of Fabric Interconnects, ensure the Cluster ID is not changed.

  • Image synchronization between Fabric Interconnects is disabled during migration to avoid syncing incompatible images. Once migration is complete, download and reinstall the appropriate server software bundles.

  • During the migration, ensure that VLAN is not created in the range of 3915 to 4042 which are the reserved VLAN range for Cisco UCS 6664 Fabric Interconnects.


Note

For more information on migrating Cisco UCS 6500 series Fabric Interconnects to Cisco UCS 6600 series Fabric Interconnect with UCS Central, see Considerations for migrating Cisco UCS 6536 Fabric Interconnect to Cisco UCS 6664 Fabric Interconnect with Cisco UCS Central.

Validating Feature Configurations for Cisco UCS 6664 Fabric Interconnect before Upgrade

Cisco UCS 6664 Fabric Interconnect does not support some software features that were allowed with Cisco UCS 6536 Fabric Interconnect. Some of these features will become available at a later software release.

Table 1. Features that needs special attention prior to upgrading

Feature

Remediation

License Management

Cisco UCS 6536 Fabric Interconnect and Cisco UCS 6664 Fabric Interconnect requires perpetual software license.

All ports and software features are activated through a perpetual software license. No additional license management is required.

Multicast optimization

Verify that multicast optimization is not enabled under the LAN quality-of-service (QoS) system classes

Fabric forwarding mode for Ethernet

Verify that the Ethernet forwarding mode is set to End Host Mode Only.

Fabric forwarding mode for Fibre Channel

Verify that Fibre Channel forwarding mode is set to End Host Mode or FC Switching Mode.

Cisco NetFlow

Unconfigure NetFlow.

MAC Security

Select Allow for MAC security.

Cisco Switched Port Analyzer (SPAN)

Use receive (RX) direction only. The installer will change SPAN to the RX direction and send an alert indicating that this setting is being changed.

Failure to comply with these remediation steps will result in a migration warning alert during the migration process and prevent the fabric interconnects from synchronizing.

Considerations for migrating Cisco UCS 6536 Fabric Interconnect to Cisco UCS 6664 Fabric Interconnect with Cisco UCS Central

In addition to Cisco UCS 6500 Series Fabric Interconnect Migration Considerations, consider the following prerequisites when migrating with Cisco UCS Central:

  • Before initiating the migration, ensure to have a complete backup of Cisco UCS Manager and UCS Central configurations.

  • To avoid any configuration issues during migration, make sure the following policies on Policy Resolution Control is set to Local in Cisco UCS Manager:

    • Infrastructure and Catalog Firmware Policy

    • Equipment Policy

    • Port Configuration Policy

Migrating from Cisco UCS 6536 Fabric Interconnect to Cisco UCS 6664 Fabric Interconnect

Cisco UCS 6500 Series Fabric Interconnect include Cisco UCS 6536 Fabric Interconnect and Cisco UCS 6600 Series Fabric Interconnect include Cisco UCS 6664 Fabric Interconnect.

You can migrate from Cisco UCS 6536 Fabric Interconnect to Cisco UCS 6664 Fabric Interconnect. After migrating to Cisco UCS 6664 Fabric Interconnect, Cisco recommends not to migrate back to Cisco UCS 6536 Fabric Interconnect.

The Cisco UCS 6664 Fabric Interconnect supports only port-channel mode for chassis-discovery. On changing the chassis or FEX discovery policy to port-channel, the chassis needs to be re-acknowledged before proceeding with the migration.

To acknowledge the chassis in Cisco UCS Manager, do the following:

  1. In the Navigation pane of Cisco UCS Manager, click Equipment.

  2. Click the Equipment node.

  3. In the Work pane, click the Policies tab.

  4. Click the Global Policies subtab.

  5. Expand Equipment > Chassis, and choose the chassis that you want to acknowledge.

  6. In the Work pane, click the General tab.

  7. In the Actions area, click Acknowledge Chassis.


Note

For more information about how to perform configuration procedures in Cisco UCS Manager, see the appropriate Cisco UCS Manager Configuration Guide.

Procedure

Step 1

Move the traffic to the primary fabric interconnect. This can be performed in two ways: fabric evacuation and uplink disablement. Use the below flowchart to choose between the two based on your Cisco UCS domain server and connectivity.

Note

 

For direct-attached rack servers, only uplink disablement is supported.

Step 2

Verify that all traffic has failed over to the primary fabric interconnect. Unconfigure all server ports or fibre channel ports on the subordinate fabric interconnect.

For more information, see the Fabric Interconnect Traffic Evacuation section in the Guidelines and Prerequisites chapter of the Firmware Management Guide.

Step 3

Power down the old subordinate fabric interconnect and disconnect the power and the L1/L2 cables.

If you are monitoring the migration using a KVM session to a server host, you may need to reconnect it after you powered down the secondary fabric interconnect.

Step 4

Mount the replacement Cisco UCS 6664 Fabric Interconnect into either the same rack or an adjacent rack.

Note

 

It is recommended to label cables as a best practice.

Step 5

Connect the L1/L2 cables and the server ports according to your port mapping plan.

Step 6

Power up the new fabric interconnect. If it is connected correctly, the new subordinate fabric interconnect will detect that it is being added to an existing cluster.

Step 7

The subordinate Cisco UCS 6664 Fabric Interconnect will automatically synchronize the configuration and database/state information from the primary Cisco UCS 6536 Fabric Interconnect.

Synchronization between primary and subordinate fabric interconnects can take several minutes. You may see an error message that will persist until the server ports are enabled.

The port configuration is copied from the subordinate Fabric Interconnect to the new Fabric Interconnect.

Note

 

Skip to Step 10 incase of replacing with a different IOM, IFM, or FEX during migration. For more information on the compatibility matrix of supported IOM, IFM, or FEX on the Fabric Interconnect, see Ports on the Cisco UCS Fabric Interconnects.

Step 8

Unified ports on the Cisco UCS 6600 Series Fabric Interconnect are different from the unified port ordering on the Cisco UCS 6536 Fabric Interconnect.

Note

 

  • When you convert from ethernet ports to fibre channel ports, a reboot is required. Fibre channel ports are converted in blocks of four (breakout ports).

  • For more information on Configuring FC Uplink port or Converting FC storage port to FC uplink port, see the LAN Ports and Port Channels chapter in Network Management Guide.

Step 9

Configure the network uplink ports on the new Cisco UCS 6664 Fabric Interconnect

Step 10

Match the old configuration from Cisco UCS 6500 Series Fabric Interconnect for the port-channel. Add uplink ports to the necessary port-channel or any other previous configuration required for the port-channel. Wait for configuration to complete before proceeding to the next step.

Note

 

Waiting to enable the server ports prevents the svc_sam_bladeAG service from communicating with chassis and servers. In previous migrations, when enabling server ports at the same time as the uplink ports, it would cause topping out (pinning) the CPU to near 100% on the primary fabric interconnect. When there is high CPU usage, the user interface becomes unresponsive and the svc_sam_bladeAG service must be restarted to recover.

Step 11

Reconfigure the server ports or fibre channel ports.

  1. If you have changed port mappings, you may need to reacknowledge the IOM, FEX, or direct-connect rack server connected to the subordinate fabric interconnect.

  2. Verify and if necessary, you can reconfigure ethernet ports as server ports.

  3. Skip to Step 13 to verify the data path.

Step 12

To remove and replace the new IOM, IFM, or FEX and reconfigure the server ports, do the following:

  1. Remove the existing module connected to the subordinate Fabric Interconnect and replace it with a supported module of Cisco UCS 6664 Fabric Interconnect.

  2. Verify if cable connections require modifications.

  3. Reconfigure to maintain the same port mappings. If not preserved, configure server ports accordingly.

  4. Reacknowledge the IOM, FEX, or IFM connected to the subordinate Fabric Interconnect.

Step 13

Verify that the data path is ready.

For more information, see the Verifying that Dynamic vNICs Are Up and Running section in the Guidelines and Prerequisites chapter of the Cisco UCS Manager Firmware Management .

Ensure that all faults are resolved before proceeding with next step.

  1. Verify, and if necessary, reconfigure the SAN pin group for fibre channel ports in the associated service profile.

  2. Verify, and if necessary, reconfigure the LAN pin group for ethernet ports in the associated service profile.

  3. Verify, and if necessary, reconfigure the port channel for uplink ethernet ports.

Step 14

Move the traffic back to the new subordinate fabric interconnect. If you used the fabric evacuation method, then deselect fabric evacuation. If you disabled the uplinks (Ethernet and Fibre Channel), then re-enable the uplinks. Verify that traffic is flowing correctly on the new subordinate fabric interconnect.

Step 15

After verifying that traffic is flowing on the subordinate fabric interconnect, promote the subordinate fabric interconnect to primary using the below commands:

  • UCS-A #connect local-mgmt: This command connects to the local management interface of the cluster.

  • UCS-A (local-mgmt) #cluster{lead {a|b}} or UCS-A (local-mgmt) #cluster{force primary{a|b}}: The cluster lead command and cluster force primary command are two separate commands that can be used for promoting the fabric interconnect.

Note

 

Ensure that the primary fabric interconnect is Cisco UCS 6664 Fabric Interconnect.

Step 16

Cable the second new fabric interconnect identically to the first and repeat the steps for the other fabric interconnect replacements to complete the migration.

Migrating Cisco UCS Fabric Interconnects: 6400 to 6600 Series

Considerations for Migrating Cisco UCS 6400 Series Fabric Interconnects

Beginning with Cisco UCS Manager, Release 6.0(1b), you can migrate Cisco UCS 6400 Series Fabric Interconnects to Cisco UCS 6664 Fabric Interconnect with C-Series rack servers.

To migrate from Cisco UCS 6400 Series Fabric Interconnects (Cisco UCS 6454 or Cisco UCS 64108) to Cisco UCS 6664 Fabric Interconnect:

  • Cisco UCS 6400 Series Fabric Interconnects (Cisco UCS 6454 or Cisco UCS 64108) must be on Cisco UCS Manager 6.0(1b) or a later release versions.

  • Cisco UCS 6664 Fabric Interconnect must be loaded with the same Infrastructure Firmware version that is on the Cisco UCS 6454 or Cisco UCS 64108 that it will replace.

Prerequisites

Before performing the migration from Cisco UCS 6400 Series Fabric Interconnects to Cisco UCS 6664 Fabric Interconnect, ensure that the following prerequisites are met for a successful migration:

  • Back up and export Cisco UCS Manager configuration before initiating the upgrade.

  • Take an inventory of the Cisco UCS domain and remove any unsupported hardware.

  • Ensure to enable the cluster failover.

  • Do not attempt to implement new software features from the new Cisco UCS software version until all required hardware is installed.

  • Validate the software features and configurations between Cisco UCS 6400 series and UCS 6664 Fabric Interconnect. Before migration, ensure that 6400 series Fabric Interconnect is reconfigured to only have features that are supported with 6600 Series Fabric Interconnect.


    Note

    This is one of the common reasons for migration failures.

  • Make sure both Cisco UCS 6400 series Fabric Interconnect and Cisco UCS 6600 Series Fabric Interconnect are on the same UCSM build before migration.

  • Licenses from Cisco UCS 6400 Series Fabric Interconnects are not transferable to Cisco UCS 6600 Series Fabric Interconnect. Cisco UCS 6600 Series Fabric Interconnect has perpetual license enabled by default.

  • Standalone installations should expect down time. In a cluster configuration, migrating the Fabric Interconnects can result in a small traffic disruption when the traffic fails over from one Fabric Interconnect to another. To avoid permanent traffic loss during migration, ensure that there is redundancy in the UCS domain on both Fabric Interconnects before migration and test the redundancy before starting the migration.

  • Ensure the latest firmware bundle is downloaded and upgraded through GUI or CLI. Incase of attempting to upgrade the firmware bundle using other methods (loader prompt/erase configuration) can result in missing package version.

  • Before migration, make sure that the FC Speed settings on Cisco UCS Fabric Interconnects are set to operate on a mutually supported value.

    • Cisco UCS 6400 Series Fabric Interconnect supported FC speeds: 8/16 Gbps

    • Cisco UCS 6600 Series Fabric Interconnect supported FC speeds: 16/32/64 Gbps

  • Ensure the IOM or IFM modules on both Primary and Secondary Fabric Interconnects match and are supported. Incompatible modules can result in peer communication issues.

  • Make a detailed record of the cabling between FEX and fabric interconnects. You must preserve the physical port mapping to maintain the server pinning already configured and minimize down time.

  • For a cluster configuration, both primary and secondary fabric interconnects must have symmetrical connection topologies between fabric interconnect and FEX.

  • Ensure all cables used for connecting servers, storage, and network devices to the Fabric Interconnect support the required speeds and are compatible with fabric interconnects.

  • A WWN pool in Cisco UCS can include only WWNNs or WWPNs within the ranges 20:00:00:00:00:00:00:00 to 20:FF:00:FF:FF:FF:FF:FF or 50:00:00:00:00:00:00:00 to 5F:FF:00:FF:FF:FF:FF:FF; all other WWN ranges are reserved.

    When Fibre Channel traffic passes through the UCS system, the source WWPN is converted to a MAC address. Avoid configuring WWPN pools that could translate into multicast MAC addresses. To ensure uniqueness in the SAN fabric, Cisco recommends using the WWN prefix 20:00:00:25:B5:XX:XX:XX for all pool blocks.

  • Enable Fabric Evacuation on the subordinate Cisco UCS 6400 Series Fabric Interconnect. After evacuation is complete, unconfigure all port roles and port channel configurations on the subordinate Fabric Interconnect, then configure them on the Cisco UCS 6664 Fabric Interconnect during migration.

Recommendations

Following are the best practices for a successful migration:

  • For minimal disruption during migration, ensure that there is redundancy for Ethernet and FC traffic from the servers in the UCS domain across both 6400 series fabric interconnects before migration.

  • Changes to the topology, such as the number of servers or uplink connections, should be performed after the fabric interconnect migration is complete.

  • During the migration of Fabric Interconnects, ensure the Cluster ID is not changed.

  • Image synchronization between Fabric Interconnects is disabled during migration to avoid syncing incompatible images. Once migration is complete, download and reinstall the appropriate server software bundles.

  • During the migration, ensure that VLAN is not created in the range of 3915 to 4042 which are the reserved VLAN range for Cisco UCS 6664 Fabric Interconnects.


Note

For more information on migrating Cisco UCS 6400 series Fabric Interconnects to Cisco UCS 6600 series Fabric Interconnect with UCS Central, see Considerations for migrating Cisco UCS 6400 Series Fabric Interconnects to UCS 6664 Fabric Interconnects with Cisco UCS Central.

Validating Feature Configurations for Cisco UCS 6664 before Upgrade

Cisco UCS 6664 Fabric Interconnect does not support some software features that were supported with Cisco UCS 6400 Series Fabric Interconnect.

Table 2. Features that needs special attention prior to upgrading

Feature

Remediation

License Management

Licensing for Cisco UCS 6664 Fabric Interconnect is not a port-based license like in previous generation Fabric Interconnects.

All ports and software features are activated through a perpetual software license. No additional license management is required.

Multicast optimization

Verify that multicast optimization is not enabled under the LAN quality-of-service (QoS) system classes

Fabric forwarding mode for Ethernet

Verify that the Ethernet forwarding mode is set to End Host Mode Only.

Fabric forwarding mode for Fibre Channel

Verify that Fibre Channel forwarding mode is set to End Host Mode or FC Switching Mode.

Cisco NetFlow

Unconfigure NetFlow.

MAC Security

Select Allow for MAC security.

VM-FEX

Remove port profiles and Cisco UCS Manager ESXi or SCVMM related configuations.

Dynamic vNIC Connection policies

Set the dynamic vNIC connection policy in the vNIC profile to Not Set.

Cisco Switched Port Analyzer (SPAN)

Use receive (RX) direction only. The installer will change SPAN to the RX direction and send an alert indicating that this setting is being changed.

Failure to comply with these remediation steps will result in a migration warning alert during the migration process and prevent the fabric interconnects from synchronizing.

Considerations for migrating Cisco UCS 6400 Series Fabric Interconnects to UCS 6664 Fabric Interconnects with Cisco UCS Central

In addition to Considerations for Migrating Cisco UCS 6400 Series Fabric Interconnects, consider the following prerequisites when migrating with Cisco UCS Central:

  • Before initiating the migration, ensure to have a complete backup of Cisco UCS Manager and UCS Central configurations.

  • To avoid any configuration issues during migration, make sure the following policies on Policy Resolution Control is set to Local in Cisco UCS Manager:

    • Infrastructure and Catalog Firmware Policy

    • Equipment Policy

    • Port Configuration Policy

Migrating from UCS 6400 Series Fabric Interconnects to Cisco UCS 6664 Fabric Interconnects

Beginning with Cisco UCS Manager, Release 6.0(1), you can migrate Cisco UCS 6400 Series Fabric Interconnects (Cisco UCS 6454 and Cisco UCS 64108 Fabric Interconnects) to Cisco UCS 6664 Fabric Interconnect. After migrating to Cisco UCS 6664 Fabric Interconnect, Cisco recommends not to migrate back to UCS 6400 Series Fabric Interconnect.

The Cisco UCS 6664 Fabric Interconnect supports only port-channel mode for chassis-discovery. On changing the chassis or FEX discovery policy to port-channel, the chassis needs to be re-acknowledged before proceeding with the migration.

To acknowledge the chassis in Cisco UCS Manager, do the following:

  1. In the Navigation pane of Cisco UCS Manager, click Equipment.

  2. Click the Equipment node.

  3. In the Work pane, click the Policies tab.

  4. Click the Global Policies subtab.

  5. Expand Equipment > Chassis, and choose the chassis that you want to acknowledge.

  6. In the Work pane, click the General tab.

  7. In the Actions area, click Acknowledge Chassis.


Note

For more information about how to perform configuration procedures in Cisco UCS Manager, see the appropriate Cisco UCS Manager Configuration Guide.

Procedure

Step 1

Move the traffic to the primary fabric interconnect. This can be performed in two ways: fabric evacuation and uplink disablement. Use the below flowchart to choose between the two based on your Cisco UCS domain server and connectivity.

Note

 

For direct-attached rack servers, only uplink disablement is supported.

Step 2

Verify that all traffic has failed over to the primary fabric interconnect. Unconfigure all server ports or fibre channel ports on the subordinate fabric interconnect.

For more information, see the Fabric Interconnect Traffic Evacuation section in the Guidelines and Prerequisites chapter of the Cisco UCS Manager Firmware Management Guide.

Step 3

Power down the old subordinate fabric interconnect and disconnect the power and the L1/L2 cables.

If you are monitoring the migration using a KVM session to a server host, you may need to reconnect it after you powered down the secondary fabric interconnect.

Step 4

Mount the replacement Cisco UCS 6664 Fabric Interconnect into either the same rack or an adjacent rack.

Note

 

It is recommended to label cables as a best practice.

Step 5

Connect the L1/L2 cables and the server ports according to your port mapping plan.

Step 6

Power up the new fabric interconnect. If it is connected correctly, the new subordinate fabric interconnect will detect that it is being added to an existing cluster.

Step 7

The subordinate Cisco UCS 6664 Fabric Interconnect will automatically synchronize the configuration and database/state information from the primary UCS 6400 Series Fabric Interconnect.

Synchronization may take several minutes. Error messages may persist until server ports are enabled.

The port configuration is copied from the subordinate Fabric Interconnect to the new Fabric Interconnect.

Note

 

Skip to Step 10 incase of replacing with a different IOM, IFM, or FEX during migration. For more information on the compatibility matrix of supported IOM, IFM, or FEX on the Fabric Interconnect, see Ports on the Cisco UCS Fabric Interconnects.

Step 8

Unified ports on the Cisco UCS 6600 Series Fabric Interconnect are different from the unified port ordering on the 6400 series Fabric Interconnect.

Note

 

  • When you convert from ethernet ports to fibre channel ports, a reboot is required. Fibre channel ports are converted in blocks of four (breakout ports).

  • For more information on Configuring FC Uplink port or Converting FC storage port to FC uplink port, see the LAN Ports and Port Channels chapter in Cisco UCS Manager Network Management Guide.

Step 9

Configure the network uplink ports on the new Cisco UCS 6664 Fabric Interconnect

Step 10

Match the old configuration from Cisco UCS 6400 Fabric Interconnect for the port-channel. Add uplink ports to the necessary port-channel or any other previous configuration required for the port-channel. Wait for configuration to complete before proceeding to the next step.

Note

 

Waiting to enable the server ports prevents the svc_sam_bladeAG service from communicating with chassis and servers. In previous migrations, when enabling server ports at the same time as the uplink ports, it would cause topping out (pinning) the CPU to near 100% on the primary fabric interconnect. When there is high CPU usage, the user interface becomes unresponsive and the svc_sam_bladeAG service must be restarted to recover.

Step 11

Reconfigure the server ports or fibre channel ports.

  1. If you have changed port mappings, you may need to reacknowledge the IOM, FEX, or direct-connect rack server connected to the subordinate fabric interconnect.

  2. Verify and if necessary, you can reconfigure ethernet ports as server ports.

  3. Skip to Step 13 to verify the data path.

Step 12

To remove and replace the new IOM, FEX, or IFM and reconfigure the server ports, do the following:

  1. Remove the existing module connected to the subordinate Fabric Interconnect and replace it with a supported module of Cisco UCS 6664 Fabric Interconnect.

  2. Verify if cable connections require modifications.

  3. Reconfigure to maintain the same port mappings. If not preserved, configure server ports accordingly.

  4. Reacknowledge the IOM, FEX, or IFM connected to the subordinate Fabric Interconnect.

Step 13

Verify that the data path is ready.

For more information, see the Verifying that Dynamic vNICs Are Up and Running section in the Guidelines and Prerequisites chapter of the Cisco UCS Manager Firmware Management Guide.

Ensure that all faults are resolved before proceeding with next step.

  1. Verify, and if necessary, reconfigure the SAN pin group for fibre channel ports in the associated service profile.

  2. Verify, and if necessary, reconfigure the LAN pin group for ethernet ports in the associated service profile.

  3. Verify, and if necessary, reconfigure the port channel for uplink ethernet ports.

Step 14

Move the traffic back to the new subordinate fabric interconnect. If you used the fabric evacuation method, then deselect fabric evacuation. If you disabled the uplinks (Ethernet and Fibre Channel), then re-enable the uplinks. Verify that traffic is flowing correctly on the new subordinate fabric interconnect.

Step 15

After verifying that traffic is flowing on the subordinate fabric interconnect, promote the subordinate fabric interconnect to primary using the below commands:

  • UCS-A #connect local-mgmt: This command connects to the local management interface of the cluster.

  • UCS-A (local-mgmt) #cluster{lead {a|b}} or UCS-A (local-mgmt) #cluster{force primary{a|b}}: The cluster lead command and cluster force primary command are two separate commands that can be used for promoting the fabric interconnect.

Note

 

Ensure that the primary fabric interconnect is Cisco UCS 6664 Fabric Interconnect.

Step 16

Cable the second new fabric interconnect identically to the first and repeat the steps for the other fabric interconnect replacements to complete the migration.