Migrating to Cisco UCS 6500 Series Fabric Interconnects

Cisco UCS 6500 Series Fabric Interconnect Hardware Compatibility Matrix

This section provides information on compatible hardware component with Cisco UCS 6500 Series Fabric Interconnect. Before upgrading to Cisco UCS-FI-6536, ensure that you acquire all the compatible components.

Cisco UCS Rack Servers

Table 1. Supported Rack Servers and FEX with Cisco 15000 Series VIC Adapters

FI

Rack Server Compatibility

15428

UCSC-M-V5Q50G

15238

(UCSC-M-V5D200G)

UCS-FI-6536

Direct Attach (40/100G)

Not Supported

C225 M6, C245 M6, C220 M6, C240 M6

Direct Attach (4x25G or 25G QSA28)

C225 M6, C245 M6, C220 M6, C240 M6

Not Supported

93180YC-FX3 (25G server ports)

C225 M6, C245 M6, C220 M6, C240 M6

Not Supported

2348 UPQ (10G server ports)

C225 M6, C245 M6, C220 M6, C240 M6

Not Supported
Table 2. Supported Rack Servers and FEX with Cisco 1400 Series VIC Adapters

FI

Rack Server Compatibility

1455- 10G/25G

UCSC-PCIE-

C25Q-04

1457 - 10G/25G

UCSC-MLOM-

C25Q-04

1467 - 10G/25G

UCSC-M-

V25-04

1495 - 40G/100G

UCSC-PCIE-

C100-04

1497 - 40G/100G

UCSC-MLOM-

C100-04

1477 - 40G/100G

UCSC-M-

V100-04

UCS-FI-6536

Direct Attach (40/100G)

Not Supported

Not Supported

Not Supported

C220 M5, C240 M5, C480 M5, C480 ML M5, C125 M5, S3260 M5

C220 M5, C240 M5, C480 M5, C480 ML M5, C125 M5

C225 M6, C245 M6, C220 M6, C240 M6

Direct Attach (4x25G or 25G QSA28)

C225 M6, C245 M6, C220 M5, C240 M5, C480 M5, C480 ML M5, C125 M5, S3260 M5

C220 M5, C240 M5

C225 M6, C245 M6, C220 M6, C240 M6

Not Supported

Not Supported

Not Supported

93180YC-FX3 (25G server ports)

C225 M6, C245 M6, C220 M5, C240 M5, C480 M5, C480 ML M5, C125 M5, S3260 M5

C220 M5, C240 M5

C225 M6, C245 M6, C220 M6, C240 M6

Not Supported

Not Supported

Not Supported

2348 UPQ (10G server ports)

C225 M6, C245 M6, C220 M5, C240 M5, C480 M5, C480 ML M5, C125 M5, S3260 M5

C220 M5, C240 M5

C225 M6, C245 M6, C220 M6, C240 M6

Not Supported

Not Supported

Not Supported
Table 3. Supported Rack Servers and FEX with Cisco 1300 Series VIC Adapters

FI

FEX

1385 - 40G

UCSC-PCIE-C40Q-03

1387 - 40G

UCSC-MLOM-C40Q-03

UCS-FI-6536

Direct Attach (40/100G)

C220 M5, C240 M5, C480 M5, C480 ML M5, C125 M5, C220 M4, C240 M4, C460 M4

C220 M5, C240 M5, C480 M5, C480 ML M5, C125 M5, C220 M4, C240 M4, C460 M4

Direct Attach (4x25G or 25G QSA28)

Not Supported

Not Supported

93180YC-FX3 (25G server ports)

Not Supported

Not Supported

2348 UPQ (10G server ports)

C220 M5, C240 M5, C480 M5, C480 ML M5, C125 M5, C220 M4, C240 M4, C460 M4

C220 M5, C240 M5, C480 M5, C480 ML M5, C125 M5, C220 M4, C240 M4, C460 M4

(QSA at the adapter)

Cisco UCS Blade Servers

Table 4. Supported Blade Servers and IOM with Cisco 15000 Series VIC Adapters

FI

IOM

15411

UCSB-ML-V5Q10G

15411 + Port Expander

UCSB-ML-V5Q10G + UCSB-MLOM-PT-01

UCS-FI-6536

2304V1/V2 (40G)

B200 M6

B200 M6

2408 (25G)

B200 M6

B200 M6
Table 5. Supported Blade Servers and IOM with Cisco 1400 Series VIC Adapters

FI

IOM

1440 - 10/40

UCSB-MLOM-40G-04

1440 + Port Expander

UCSB-MLOM-40G-04 + UCSB-MLOM-PT-01

1480 - 10/40

UCSB-VIC-M84-4P

1440 + 1480

1440 + 1480 + Port Expander

1440 + 1480 + 1480

UCS-FI-6536

2304V1/V2 (40G)

B200 M6, B200 M5, B480 M5

B200 M6, B200 M5, B480 M5

B200 M6, B200 M5, B480 M5

B200 M6, B200 M5, B480 M5

B480 M5

B480 M5

2408 (25G)

B200 M6, B200 M5, B480 M5

B200 M6, B200 M5, B480 M5

B200 M6, B200 M5, B480 M5

B200 M6, B200 M5, B480 M5

B480 M5

B480 M5
Table 6. Supported Blade Servers and IOM with Cisco 1300 Series VIC Adapters

FI

IOM

1340 - 10G/40G

UCSB-MLOM-

40G-03

1380 - 10/40

UCSB-VIC-

M83-8P

1340 + 1380

1340 + Port Expander - 10G/40G

UCSB-MLOM-

40G-03 + UCSB-

MLOM-PT-01

1340 + 1380 + Port Expander

1340 + 1380 + 1380

UCS-FI-6536

2304V1/V2 (40G)

B200 M5, B480 M5, B200 M4, B260 M4, B460 M4, B420 M4

B200 M5, B480 M5, B200 M4, B260 M4, B460 M4, B420 M4

B200 M5, B480 M5, B200 M4, B260 M4, B460 M4, B420 M4

B200 M5, B480 M5, B200 M4, B260 M4, B460 M4, B420 M4

B260 M4, B460M4, B420M4, B480M5

B260 M4, B460M4, B420M4, B480M5

2408 (25G)

B200 M5, B480 M5, B200 M4, B260 M4, B460 M4, B420 M4

B200 M5, B480 M5, B200 M4, B260 M4, B460 M4, B420 M4

B200 M5, B480 M5, B200 M4, B260 M4, B460 M4, B420 M4

B200 M5, B480 M5, B200 M4, B260 M4, B460 M4, B420 M4

B260 M4, B460M4, B420M4, B480M5

B260 M4, B460M4, B420M4, B480M5

Cisco UCS 6300 Series Fabric Interconnect Migration

Cisco UCS 6300 Series Fabric Interconnect Migration Considerations

Beginning with Cisco UCS Manager, Release 4.2(3b), Cisco UCS Manager provides support for Cisco UCS 6536 Fabric Interconnect. You can migrate Cisco UCS 6300 Series Fabric Interconnects to Cisco UCS 6536 Fabric Interconnect with B-Series servers, C-Series, or S-Series servers.

To migrate from Cisco UCS 6300 Series Fabric Interconnects to Cisco UCS 6536 Fabric Interconnect:

  • Cisco UCS 6300 Series Fabric Interconnects must be on Cisco UCS Manager 4.2(3b) or a later release.

  • Cisco UCS 6536 Fabric Interconnect must be loaded with the same Infrastructure Firmware version that is on the Cisco UCS 6300 Series Fabric Interconnect that it will replace.

Prerequisites

Before performing the migration from Cisco UCS 6300 Series Fabric Interconnects to Cisco UCS 6536 Fabric Interconnect, ensure that the following prerequisites are met for a successful migration:

  • Back up and export Cisco UCS Manager configuration before initiating the upgrade. Ensure that the backup file from the 6300 Series Fabric Interconnects is compatible with the 6536 Fabric Interconnects. You can follow any of the backup methods:

  • Take an inventory of the Cisco UCS domain and remove any unsupported hardware.

  • Ensure to enable the cluster failover.

  • Do not attempt to implement new software features from the new Cisco UCS software version until all required hardware is installed.

  • Validate the software features and configurations between Cisco UCS 6300 series and UCS 6536 Fabric Interconnects. Before migration, ensure that 6300 series Fabric Interconnect is reconfigured to only have features that are supported with 6536 Fabric Interconnects.


    Note

    This is one of the most common reasons for migration failures.

  • Make sure both Cisco UCS 6300 series and 6500 series Fabric Interconnects are on the same UCSM build before migration.

  • Licenses from Cisco UCS 6300 Series Fabric Interconnects are not transferable to Cisco UCS 6500 Series Fabric Interconnects.

  • Standalone installations should expect down time. In a cluster configuration, migrating the Fabric Interconnects can result in a small traffic disruption when the traffic fails over from one Fabric Interconnect to another. To avoid that there is no permanent traffic loss during migration, ensure that there is redundancy in the UCS domain on both Fabric Interconnects before migration and test the redundancy before starting the migration.

  • Cisco UCS 6536 Fabric Interconnect use the IDLE fill pattern for FC uplink ports and FC storage ports when using 8 Gbps speed.

    When migrating to Cisco UCS 6536 Fabric Interconnect and configuring FC Uplink Ports or FC Storage Ports at 8Gbps speed, ensure that the fill pattern is set as IDLE on the corresponding FC switch ports and the direct-attached FC storage array ports. If the fill pattern is not set as IDLE, FC uplink ports and FC storage ports operating at 8 Gbps might go to an errDisabled state, lose SYNC intermittently, or receive errors or bad packets.

    Cisco UCS 6536 Fabric Interconnects supports 8 Gbps only with fill-pattern set to IDLE for direct-attached FC connectivity (FC uplink ports or FC storage ports). This limitation is not applicable for Cisco UCS 6536 Fabric Interconnects with Fibre Channel (FC) ports at 16 Gbps and 32 Gbps. When migrating to Cisco UCS 6536 Fabric Interconnect for direct-attached storage arrays that don’t support IDLE fill-pattern at 8 Gbps do one of the following:

    • Use a SAN switch between the Cisco UCS 6536 Fabric Interconnect and the storage array with 8 GB FC connectivity.

    • Upgrade the storage array to 16 GB or 32 GB FC connectivity.

  • Ensure the latest firmware bundle is downloaded and upgraded through GUI or CLI. Incase of attempting to upgrade the firmware bundle using other methods (loader prompt/erase configuration) can result in missing package version.

  • Before migration, make sure that the FC Speed is 8Gbps on Cisco UCS 6300 Fabric Interconnects or the connected switch supports 8Gbps speed.


    Note

    Post migration, you can configure the FC Port (Scalability Port) speed on Cisco UCS 6500 Fabric Interconnects.

  • Migrating to different IOM models can result in peer communication issue between IOMs of the Primary and Secondary Fabric Interconnects.

  • Make a detailed record of the cabling between FEX and fabric interconnects. You must preserve the physical port mapping to maintain the server pinning already configured and minimize down time.

  • For a cluster configuration, both fabric interconnects must have symmetrical connection topologies between fabric interconnect and FEX.

  • Use the same speed cables on all the adapter ports that are connected to same Fabric Interconnect. Cisco UCS VIC adapter ports connected to Cisco UCS 6536 fabric interconnect through a mix of 10G and 25G cables can result in UCS rack-mount server discovery failure and ports moving to suspended state.

  • Cisco UCS 6536 Fabric Interconnect only supports 25/40/100G direct connectivity for C-series rack servers and 10G direct-connect is not supported on C-series rack servers. The Cisco UCS 6536 Fabric Interconnect supports 10G server connectivity only with 2348-UPQ FEX.

  • A WWN pool can include only WWNNs or WWPNs in the ranges from 20:00:00:00:00:00:00:00 to 20:FF:00:FF:FF:FF:FF:FF or from 50:00:00:00:00:00:00:00 to 5F:FF:00:FF:FF:FF:FF:FF. All other WWN ranges are reserved. When fibre channel traffic is sent through the UCS infrastructure the source WWPN is converted to a MAC address. You cannot use WWPN pool which can translate to source multicast MAC addresses. To ensure the uniqueness of the Cisco UCS WWNNs and WWPNs in the SAN fabric, Cisco recommends using the following WWN prefix for all blocks in a pool: 20:00:00:25:B5:XX:XX:XX

  • Unconfigure the fibre channel ports on the migrating subordinate Cisco UCS 6300 series Fabric Interconnect and reconfigure on the Cisco UCS 6536 Fabric Interconnect.

Recommendations

Following are the best practices for a successful migration:

  • For minimal disruption during migration, ensure that there is redundancy for Ethernet and FC traffic from the servers in the UCS domain across both 6300 series fabric interconnects before migration.

  • Changes to the topology, such as the number of servers or uplink connections, should be performed after the fabric interconnect migration is complete.

  • During the migration of Fabric Interconnects, ensure the Cluster ID is not changed.

  • During the migration, image synchronization between fabric interconnects is not allowed. This is done to prevent incompatible images from getting synchronized. It is necessary to download B-Series, C-Series, and S-Series server software bundles again after migration is complete.

  • During the migration, ensure that VLAN is not created in the range of 3915 to 4042 which are the reserved VLAN range for Cisco UCS 6536 Fabric Interconnects.

  • The Cisco UCS 6536 Fabric Interconnect supports only port-channel mode for chassis-discovery. Hence change the Chassis/FEX discovery policy on the Cisco UCS 6300 series Fabric Interconnect to port-channel and immediately re-acknowledge the Cisco UCS 5108 chassis.


    Note

    The migration can fail when the chassis is not immediately re-acknowledged.

Validating Feature Configurations for Cisco UCS 6536 before Upgrade

Cisco UCS 6536 Fabric Interconnect does not support some software features that were allowed with Cisco UCS 6300 Fabric Interconnect. Some of these features will become available at a later software release.

Table 7. Features that needs special attention prior to upgrading

Feature

Remediation

License Management

Licensing for Cisco UCS 6536 Fabric Interconnect is not a port-based license like in previous generation Fabric Interconnects.

All ports are enabled through a perpetual license in Cisco UCS 6536 Fabric Interconnect and no license installation is required.

Chassis and fabric extender I/O port channel

Select a port channel to the I/O module (IOM).

Multicast optimization

Verify that multicast optimization is not enabled under the LAN quality-of-service (QoS) system classes

Fabric forwarding mode for Ethernet

Verify that the Ethernet forwarding mode is set to End Host Mode Only.

Fabric forwarding mode for Fibre Channel

Verify that Fibre Channel forwarding mode is set to End Host Mode or FC Switching Mode.

Cisco NetFlow

Unconfigure NetFlow.

MAC Security

Select Allow for MAC security.

VM-FEX

Remove port profiles and Cisco UCS Manager ESXi or SCVMM related configurations.

Dynamic vNIC connection policies

Set the dynamic vNIC connection policy in the vNIC profile to Not set.

Cisco Switched Port Analyzer (SPAN)

Use receive (RX) direction only. The installer will change SPAN to the RX direction and send an alert indicating that this setting is being changed.

Failure to comply with these remediation steps will result in a migration warning alert during the migration process and prevent the fabric interconnects from synchronizing.

Migrating from UCS 6300 Series Fabric Interconnects to UCS 6536 Fabric Interconnects

Cisco UCS 6300 Series Fabric Interconnects include Cisco UCS 6332 and Cisco UCS 6332-16UP. You can migrate from Cisco UCS 6300 Series Fabric Interconnects to Cisco UCS 6536 Fabric Interconnect.

The Cisco UCS 6536 Fabric Interconnect supports only port-channel mode for chassis-discovery. On changing the chassis or FEX discovery policy to port-channel, the chassis needs to be re-acknowledged before proceeding with the migration.

To acknowledge the chassis in Cisco UCS Manager, do the following:

  1. In the Navigation pane of Cisco UCS Manager, click Equipment.

  2. Click the Equipment node.

  3. In the Work pane, click the Policies tab.

  4. Click the Global Policies subtab.

  5. In the Chassis/FEX Discovery Policy area, set the Link Grouping Preference field to Port Channel.

  6. Expand Equipment > Chassis, and choose the chassis that you want to acknowledge.

  7. In the Work pane, click the General tab.

  8. In the Actions area, click Acknowledge Chassis.


Note

For more information about how to perform configuration procedures in Cisco UCS Manager, see the appropriate Cisco UCS Manager Configuration Guide.

Procedure

Step 1

Move the traffic to the primary fabric interconnect. This can be performed in two ways: fabric evacuation and uplink disablement. Use the below flowchart to choose between the two based on your Cisco UCS domain server and connectivity.

Note

 

For direct-attached rack servers, only uplink disablement is supported.

Step 2

Verify that all traffic has failed over to the primary fabric interconnect. Unconfigure all server ports or fibre channel ports on the subordinate fabric interconnect.

Note

 

For more information, see the Fabric Interconnect Traffic Evacuation section in the Guidelines and Prerequisites chapter of the Cisco UCS Manager Firmware Management Guide.

Step 3

Power down the old subordinate fabric interconnect and disconnect the power and the L1/L2 cables.

If you are monitoring the migration using a KVM session to a server host, you may need to reconnect it after you powered down the secondary fabric interconnect.

Step 4

Mount the replacement Cisco UCS 6536 Fabric Interconnect into either the same rack or an adjacent rack.

Note

 

As a best practice, you should label the cables.

Step 5

Connect the L1/L2 cables and the server ports according to your port mapping plan.

Step 6

Power up the new fabric interconnect. If it is connected correctly, the new subordinate fabric interconnect will detect that it is being added to an existing cluster.

Step 7

The subordinate Cisco UCS 6536 Fabric Interconnect will automatically synchronize the configuration and database/state information from the primary UCS 6300 Series Fabric Interconnect.

Synchronization between primary and subordinate fabric interconnects can take several minutes. You may see an error message that will persist until the server ports are enabled.

The port configuration is copied from the subordinate Fabric Interconnect to the new Fabric Interconnect.

Note

 

Skip to Step 10 incase of replacing with a different IOM or FEX during migration. For more information on the compatibility matrix of supported IOM or FEX on the Fabric Interconnect, see Ports on the Cisco UCS Fabric Interconnects

Step 8

Unified ports on the 6500 series fabric interconnect are different from the unified port ordering on the 6300 series Fabric Interconnect.

Note

 

  • When you convert from ethernet ports to fibre channel ports, a reboot is required. Fibre channel ports are converted in blocks of four (breakout ports).

  • For more information on Configuring FC Uplink port or Converting FC Storage Port to FC Uplink port, refer the LAN Ports and Port Channels chapter in Cisco UCS Manager Network Management Guide.

Step 9

Reconfigure the server ports or fibre channel ports that were unconfigured in Step 2.

  1. If you have changed port mappings, you may need to reacknowledge the IOM, FEX, or direct-connect rack server connected to the subordinate fabric interconnect.

  2. Verify and if necessary, you can reconfigure ethernet ports as server ports.

  3. Skip to Step 11 to verify the data path.

Step 10

To remove and replace the new IOM or FEX and reconfigure the server ports, do the following:

  1. Remove the existing IOM connected to the subordinate Fabric Interconnect and replace it with a supported IOM of Cisco UCS 6536 Fabric Interconnect.

  2. Verify if any modifications are required for the cable connections.

    Note

     

    For more information, see Cisco UCS 6500 Series Fabric Interconnect Hardware Compatibility Matrix.

  3. Reconfigure the new IOM to maintain the same port mappings. If port mapping were not preserved, configure the server ports accordingly.

  4. Reacknowledge the IOM connected to the subordinate Fabric Interconnect.

Note

 

When migrating from Cisco UCS 6300 to Cisco UCS 6536, IOM-220x can be replaced with IOM-2304v2 or IOM-2408 and IOM-2304v1/v2 with IOM-2408.

Step 11

Verify that the data path is ready.

For more information, see the Verifying that Dynamic vNICs Are Up and Running section in the Guidelines and Prerequisites chapter of the Cisco UCS Manager Firmware Management Guide.

Ensure that all faults are resolved before proceeding with next step.

  1. Verify, and if necessary, reconfigure the SAN pin group for Fibre Channel ports in the associated service profile.

  2. Verify, and if necessary, reconfigure the LAN pin group for Ethernet ports in the associated service profile.

  3. Verify, and if necessary, reconfigure the port channel for uplink Ethernet ports.

Step 12

Move the traffic back to the new subordinate fabric interconnect. If you used the fabric evacuation method, then deselect fabric evacuation. If you disabled the uplinks (Ethernet and Fibre Channel), then re-enable the uplinks. Verify that traffic is flowing correctly on the new subordinate fabric interconnect.

Step 13

After verifying that traffic is flowing on the subordinate fabric interconnect, promote the subordinate fabric interconnect to primary using the below commands:

  • UCS-A #connect local-mgmt: This command connects to the local management interface of the cluster.

  • UCS-A (local-mgmt) #cluster{lead {a|b}} or UCS-A (local-mgmt) #cluster{force primary{a|b}}: The cluster lead command and cluster force primary command are two separate commands that can be used for promoting the fabric interconnect.

Note

 

Ensure that the primary fabric interconnect is Cisco UCS 6536 Fabric Interconnect.

Step 14

Cable the second new fabric interconnect identically to the first and repeat the steps for the other fabric interconnect replacements to complete the migration.

Migrating Offline from Cisco UCS 6300 Series Fabric Interconnect to Cisco UCS 6536 Fabric Interconnects

Offline migration involves transitioning the configurations and settings from Cisco UCS 6300 Series Fabric Interconnect to Cisco UCS 6536 Fabric Interconnect, while the system is offline. This approach includes taking back up of the current configuration, decommissioning the components, and transferring the configurations to the new Fabric Interconnect.

Procedure

Step 1

Use one of the supported protocols (SCP, TFTP, FTP, SFTP) to perform a remote backup of all configurations from the source Fabric Interconnect.

Step 2

Decommission the connected servers and unconfigure the ports on Cisco UCS 6300 Series Fabric Interconnect.

Step 3

Transfer the ports and preserved port mappings to the Cisco UCS 6536 Fabric Interconnect.

Note

 

Ensure to use compatible cables or transceivers for the connection.

Step 4

Import the remote backup file and restore the configuration on Cisco UCS 6536 Fabric Interconnect.

Step 5

Validate that the servers automatically restore their previous configurations upon reconnection.

Step 6

Verify that all configurations, including operating system installations that are preserved and functioning as expected.

Cisco UCS 6200 Series Fabric Interconnect Migration

Cisco UCS 6200 Series Fabric Interconnect Migration Considerations

Beginning with Cisco UCS Manager, Release 4.2(3b), Cisco UCS Manager provides support for Cisco UCS 6536 Fabric Interconnect. You can migrate Cisco UCS 6200 Series Fabric Interconnects to Cisco UCS 6536 Fabric Interconnect with B-Series servers, C-Series, or S-Series servers.

To migrate from Cisco UCS 6200 Series Fabric Interconnects to Cisco UCS 6536 Fabric Interconnect:

  • Cisco UCS 6200 Series Fabric Interconnects must be on Cisco UCS Manager 4.2(3b) or a later release.

  • Cisco UCS 6536 Fabric Interconnect must be loaded with the same infrastructure firmware version as on the Cisco UCS 6200 Series Fabric Interconnect that it will replace.

Prerequisites

Before performing the migration from Cisco UCS 6200 Series Fabric Interconnects to Cisco UCS 6536 Fabric Interconnect, ensure that the following prerequisites are met for a successful migration:

  • Back up and export Cisco UCS Manager configuration before initiating the upgrade. Ensure that the backup file from the 6200 Series Fabric Interconnects is compatible with the Cisco UCS 6536 Fabric Interconnects. You can follow any of the backup methods:

    • Online Migration: This method allows for backing up the configuration while the Fabric Interconnect is operational. It ensures minimal disruption and maintains continuous availability during the migration process.

    • Offline Migration: This method involves taking the fabric interconnect offline to perform the backup, which may result in downtime.


    Note

    For more information, see Migrating Offline from Cisco UCS 6200 Series Fabric Interconnect to Cisco UCS 6536 Fabric Interconnects.

  • Take an inventory of the Cisco UCS domain and remove any unsupported hardware.

  • Ensure to enable the cluster failover.

  • Do not attempt to implement new software features from the new Cisco UCS software version until all required hardware is installed.

  • Validate the software features and configurations between Cisco UCS 6200 series and UCS 6536 Fabric Interconnects. Ensure that the 6200 series fabric interconnect is reconfigured to only have features that are supported with 6536 Fabric Interconnect.


    Note

    This is one of the most common reasons for migration failures.

  • Make sure both Cisco UCS 6200 series and 6500 series Fabric Interconnects are on the same UCSM build before migration.

  • Licenses from Cisco UCS 6200 Series Fabric Interconnects are not transferable to Cisco UCS 6500 Series Fabric Interconnects.

  • Standalone installations should expect down time. In a cluster configuration, migrating the Fabric Interconnect can result in a small traffic disruption when traffic fails over from one Fabric Interconnect to another. To avoid that there is no permanent traffic loss during migration, ensure that there is redundancy in the UCS domain on both Fabric Interconnects before migration and test the redundancy before starting the migration.

  • Cisco UCS 6536 Fabric Interconnect use the IDLE fill pattern for FC uplink ports and FC storage ports when using 8 Gbps speed.

    Ensure that the fill pattern is set as IDLE on the corresponding FC switch ports and the direct-attached FC storage array ports when configuring FC Uplink Ports or FC Storage Ports at 8 Gbps speed. If the fill pattern is not set as IDLE, FC uplink ports and FC storage ports operating at 8 Gbps might go to an errDisabled state, lose SYNC intermittently, or receive errors or bad packets.

    Cisco UCS 6536 Fabric Interconnect supports 8 Gbps only with fill-pattern set to IDLE for direct-attached FC connectivity (FC uplink ports or FC storage ports). This limitation is not applicable for Cisco UCS 6536 Fabric Interconnects with Fibre Channel (FC) ports at 16 Gbps and 32 Gbps. When migrating to Cisco UCS 6536 Fabric Interconnect for direct-attached storage arrays that don’t support IDLE fill-pattern at 8 Gbps do one of the following:

    • Use a SAN switch between the Cisco UCS 6536 Fabric Interconnect and the storage array with 8 GB FC connectivity.

    • Upgrade the storage array to 16 GB or 32 GB FC connectivity.

  • Ensure the latest firmware bundle is downloaded and upgraded through GUI or CLI. Incase of attempting to upgrade the firmware bundle using other methods (loader prompt/erase configuration) can result in missing package version.

  • Before migration, make sure that the FC Speed is 8Gbps on Cisco UCS 6200 Fabric Interconnects or the connected switch supports 8Gbps speed.


    Note

    Post migration, you can configure the FC Port (Scalability Port) speed on Cisco UCS 6500 Fabric Interconnects.

  • Migrating to different IOM models can result in peer communication issue between IOMs of the Primary and Secondary Fabric Interconnects.

  • Make a detailed record of the cabling between FEX and fabric interconnect. You must preserve the physical port mapping to maintain the server pinning already configured and minimize down time.

  • For a cluster configuration, both fabric interconnects must have symmetrical connection topologies between fabric interconnect and FEX.

  • Use the same speed cables on all the adapter ports that are connected to same Fabric Interconnect. Cisco UCS VIC adapter ports connected to Cisco UCS 6536 fabric interconnect through a mix of 10G and 25G cables can result in UCS rack-mount server discovery failure and ports moving to suspended state.

  • Cisco UCS 6536 Fabric Interconnect only supports 25/40/100G direct connectivity for C-series rack servers and 10G direct-connect is not supported on C-series rack servers. The Cisco UCS 6536 Fabric Interconnect supports 10G server connectivity only with 2348-UPQ FEX.

  • A WWN pool can include only WWNNs or WWPNs in the ranges from 20:00:00:00:00:00:00:00 to 20:FF:00:FF:FF:FF:FF:FF or from 50:00:00:00:00:00:00:00 to 5F:FF:00:FF:FF:FF:FF:FF. All other WWN ranges are reserved. When fibre channel traffic is sent through the UCS infrastructure the source WWPN is converted to a MAC address. You cannot use WWPN pool which can translate to source multicast MAC addresses. To ensure the uniqueness of the Cisco UCS WWNNs and WWPNs in the SAN fabric, Cisco recommends using the following WWN prefix for all blocks in a pool: 20:00:00:25:B5:XX:XX:XX.

  • Incase of migrating from Cisco UCS 6296UP to Cisco UCS 6536 Fabric Interconnect, the ports that are connected to the extended modules of Cisco UCS 6296UP require a reconfiguration after migrating to Cisco UCS 6536 Fabric Interconnect.

  • Unconfigure the fibre channel ports on the migrating subordinate Cisco UCS 6200 series Fabric Interconnect and reconfigure on the Cisco UCS 6536 Fabric Interconnect.

  • Recommendations

    Following are the best practices for a successful migration:

    • For minimal disruption during migration, ensure that there is redundancy for Ethernet and FC traffic from the servers in the UCS domain across both 6200 series fabric interconnects before migration.

    • Changes to the topology, such as the number of servers or uplink connections, should be performed after the fabric interconnect migration is complete.

    • During the migration of Fabric Interconnects, ensure the Cluster ID is not changed.

    • During the migration, image synchronization between fabric interconnects is not allowed. This is done to prevent incompatible images from getting synchronized. It is necessary to download B-Series, C-Series, and S-Series server software bundles again after migration is complete.

    • During the migration, ensure that VLAN is not created in the range of 3915 to 4042 which are the reserved VLAN range for Cisco UCS 6536 Fabric Interconnects.

    • The Cisco UCS 6536 Fabric Interconnect supports only port-channel mode for chassis-discovery. Hence change the Chassis/FEX discovery policy on the Cisco UCS 6200 series Fabric Interconnect to port-channel and immediately re-acknowledge the Cisco UCS 5108 chassis.


      Note

      The migration can fail when the chassis is not immediately re-acknowledged.

Validating Feature Configurations for Cisco UCS 6536 before Upgrade

Cisco UCS 6536 Fabric Interconnect does not support some software features that were allowed with Cisco UCS 6200 Fabric Interconnect. Some of these features will become available at a later software release.

Table 8. Features that needs special attention prior to upgrading

Feature

Remediation

License Management

Licensing for Cisco UCS 6536 Fabric Interconnect is not a port-based license like in previous generation Fabric Interconnects.

All ports are enabled through a perpetual license in Cisco UCS 6536 Fabric Interconnect and no license installation is required.

Chassis and fabric extender I/O port channel

Select a port channel to the I/O module (IOM).

Multicast optimization

Verify that multicast optimization is not enabled under the LAN quality-of-service (QoS) system classes

Fabric forwarding mode for Ethernet

Verify that the Ethernet forwarding mode is set to End Host Mode Only.

Fabric forwarding mode for Fibre Channel

Verify that Fibre Channel forwarding mode is set to End Host Mode Only or FC Switching Mode.

Cisco NetFlow

Unconfigure NetFlow.

MAC Security

Select Allow for MAC security.

VM-FEX

Remove port profiles and Cisco UCS Manager ESXi or SCVMM related configurations.

Dynamic vNIC connection policies

Set the dynamic vNIC connection policy in the vNIC profile to Not set.

Cisco Switched Port Analyzer (SPAN)

Use receive (RX) direction only. The installer will change SPAN to the RX direction and send an alert indicating that this setting is being changed.

Failure to comply with these remediation steps will result in a migration warning alert during the migration process and prevent the fabric interconnects from synchronizing.

Migrating from UCS 6200 Series Fabric Interconnects to UCS 6536 Fabric Interconnects

Cisco UCS 6200 Series Fabric Interconnects include Cisco UCS 6248UP 48-Port Fabric Interconnect and Cisco UCS 6296UP 96-Port Fabric Interconnect. You can migrate from Cisco UCS 6200 Series Fabric Interconnects to Cisco UCS 6536 Fabric Interconnect. After migrating to Cisco UCS 6536 Fabric Interconnect, Cisco recommends not to migrate back to UCS 6200 Series Fabric Interconnect.

The Cisco UCS 6536 Fabric Interconnect supports only port-channel mode for chassis-discovery. On changing the chassis or FEX discovery policy to port-channel, the chassis needs to be re-acknowledged before proceeding with the migration.

To acknowledge the chassis in Cisco UCS Manager, do the following:

  1. In the Navigation pane of Cisco UCS Manager, click Equipment.

  2. Click the Equipment node.

  3. In the Work pane, click the Policies tab.

  4. Click the Global Policies subtab.

  5. In the Chassis/FEX Discovery Policy area, set the Link Grouping Preference field to Port Channel.

  6. Expand Equipment > Chassis, and choose the chassis that you want to acknowledge.

  7. In the Work pane, click the General tab.

  8. In the Actions area, click Acknowledge Chassis.


Note

For more information about how to perform configuration procedures in Cisco UCS Manager, see the appropriate Cisco UCS Manager Configuration Guide.

Procedure

Step 1

Move the traffic to the primary fabric interconnect. This can be performed in two ways: fabric evacuation and uplink disablement. Use the below flowchart to choose between the two based on your Cisco UCS domain server and connectivity.

Note

 

For direct-attached rack servers, only uplink disablement is supported.

Step 2

Verify that all traffic has failed over to the primary fabric interconnect. Unconfigure all the server ports or fibre channel ports on the subordinate fabric interconnect.

Note

 

For more information, see the Fabric Interconnect Traffic Evacuation section in the Guidelines and Prerequisites chapter of the Cisco UCS Manager Firmware Management Guide.

Step 3

Power down the old subordinate fabric interconnect and disconnect the power and the L1/L2 cables.

If you are monitoring the migration using a KVM session to a server host, you may need to reconnect it after you powered down the secondary fabric interconnect.

Step 4

Mount the replacement Cisco UCS 6536 Fabric Interconnect into either the same rack or an adjacent rack.

Note

 

As a best practice, you should label the cables.

Step 5

Connect the L1/L2 cables and the server ports according to your port mapping plan.

Step 6

Power up the new fabric interconnect. If it is connected correctly, the new subordinate fabric interconnect will detect that it is being added to an existing cluster.

Step 7

Unified ports on the 6500 series fabric interconnect are different from the unified port ordering on the 6200 series Fabric Interconnect.

Note

 

  • When you convert from ethernet ports to fibre channel ports, a reboot is required. Fibre channel ports are converted in blocks of four (breakout ports).

  • For more information on Configuring FC Uplink port or Converting FC Storage Port to FC Uplink port, refer the LAN Ports and Port Channels chapter in Cisco UCS Manager Network Management Guide.

Step 8

The subordinate Cisco UCS 6536 Fabric Interconnect will automatically synchronize the configuration and database/state information from the primary UCS 6200 Series Fabric Interconnect.

Synchronization between primary and subordinate fabric interconnects can take several minutes. You may see an error message that will persist until the server ports are enabled.

The port configuration is copied from the subordinate Fabric Interconnect to the new Fabric Interconnect.

Note

 

Skip to Step 10 incase of replacing with a different IOM or FEX during migration. For more information on the compatibility matrix of supported IOM or FEX on the Fabric Interconnect, see Ports on the Cisco UCS Fabric Interconnects

Step 9

Reconfigure the server ports or fibre channel ports that were unconfigured in Step 2.

  1. If you have changed port mappings, you may need to reacknowledge the IOM, FEX, or direct-connect rack server connected to the subordinate fabric interconnect.

  2. Verify and if necessary, you can reconfigure ethernet ports as server ports.

  3. Skip to Step 11 to verify the data path.

Step 10

To remove and replace the new IOM or FEX and reconfigure the server ports, do the following:

  1. Remove the existing IOM (UCS-IOM-2204/UCS-IOM-2208) connected to the subordinate Fabric Interconnect and replace it with a supported IOM of Cisco UCS 6536 Fabric Interconnect.

  2. Verify if any modifications are required for the cable connections.

    Note

     

    For more information, see Cisco UCS 6500 Series Fabric Interconnect Hardware Compatibility Matrix.

    .

  3. Reconfigure the new IOM to maintain the same port mappings. If port mapping were not preserved, configure the server ports accordingly.

  4. Reacknowledge the IOM connected to the subordinate Fabric Interconnect.

Note

 

When migrating from Cisco UCS 6200 to Cisco UCS 6536, IOM-220x can be replaced with IOM-2304v2 or IOM-2408.

Step 11

Verify that the data path is ready and all faults are resolved before proceeding with the next steps.

For more information, see the Verifying that Dynamic vNICs Are Up and Running section in the Guidelines and Prerequisites chapter of the Cisco UCS Manager Firmware Management Guide.

  1. Verify, and if necessary, reconfigure the SAN pin group for Fibre Channel ports in the associated service profile.

  2. Verify, and if necessary, reconfigure the LAN pin group for Ethernet ports in the associated service profile.

  3. Verify, and if necessary, reconfigure the port-channel for uplink Ethernet ports.

Step 12

Move the traffic back to the new subordinate fabric interconnect. If you used the fabric evacuation method, then deselect fabric evacuation. If you disabled the uplinks (Ethernet and Fibre Channel), then reenable the uplinks. Verify that traffic is flowing correctly on the new subordinate fabric interconnect.

Step 13

After verifying that traffic is flowing on the subordinate fabric interconnect, promote the subordinate fabric interconnect to primary using the below commands:

  • UCS-A #connect local-mgmt: This command connects to the local management interface of the cluster.

  • UCS-A (local-mgmt) #cluster{lead {a|b}} or UCS-A (local-mgmt) #cluster{force primary{a|b}}: The cluster lead command and cluster force primary command are two separate commands that can be used for promoting the fabric interconnect.

Note

 

Ensure that the primary fabric interconnect is Cisco UCS 6536 Fabric Interconnect.

Step 14

Cable the second new fabric interconnect identically to the first and repeat the steps for the other fabric interconnect replacements to complete the migration.

Migrating Offline from Cisco UCS 6200 Series Fabric Interconnect to Cisco UCS 6536 Fabric Interconnects

Offline migration involves transitioning the configurations and settings from Cisco UCS 6200 Series Fabric Interconnect to Cisco UCS 6536 Fabric Interconnect, while the system is offline. This approach includes taking back up of the current configuration, decommissioning the components, and transferring the configurations to the new Fabric Interconnect.

Procedure

Step 1

Use one of the supported protocols (SCP, TFTP, FTP, SFTP) to perform a remote backup of all configurations from the source Fabric Interconnect.

Step 2

Decommission the connected servers and unconfigure the ports on Cisco UCS 6200 Series Fabric Interconnect.

Step 3

Transfer the ports and preserved port mappings to the Cisco UCS 6536 Fabric Interconnect.

Note

 

Ensure to use compatible cables or transceivers for the connection.

Step 4

Import the remote backup file and restore the configuration on Cisco UCS 6536 Fabric Interconnect.

Step 5

Validate that the servers automatically restore their previous configurations upon reconnection.

Step 6

Verify that all configurations, including operating system installations that are preserved and functioning as expected.