Generic

The guidelines that apply to all upgrades irrespective of the releases are

• A pre-upgrade generic checklist includes:

  • Schedule the upgrade when your network is stable and steady.

  • Avoid any power interruption, which could corrupt the software image, during the installation procedure.

  • On devices with dual supervisor modules, both supervisor modules must have connections on the console ports to maintain connectivity when switchovers occur during a software upgrade. For more information about your specific chassis, see the relevant Hardware Installation Guide.

  • Perform the installation on the active supervisor module, not the standby supervisor module.

• When you use install all with no-reload option, the saved configuration cannot be used before you reload the device. Saving configuration in this state can result in incorrect startup configuration once you reload the device with new version of NX-OS.

• During upgrade, while performing device reload, if ASCII replay is triggered without binary restore, primary key gets lost. The primary key must be reconfigured after device reload. Use the key config-key ascii command to reconfigure the primary key and avoid encryption issues. However, upgrade with binary restore retains the primary key after the reboot.

• ISSU is blocked if boot poap enable is configured.

• Make sure that both vPC peers are in the same mode (regular mode or enhanced mode) before performing a non-disruptive upgrade.

  Note

  vPC peering between an enhanced ISSU mode (boot mode lxc) configured switch and a non-enhanced ISSU mode switch is not supported.

• During an ISSU, the software reload process on the first vPC device locks its vPC peer device by using CFS messaging over the vPC communications channel. Only one device at a time is upgraded. When the first device completes its upgrade, it unlocks its peer device. The second device then performs the upgrade process, locking the first device as it does so. During the upgrade, the two vPC devices temporarily run different releases of NX-OS; however, the system functions correctly because of its backward compatibility support.

• ISSU is not supported when onePK is enabled. You can run the show feature | include onep command to verify that this feature is disabled before performing an ISSU or enhanced ISSU.

• Occasionally, while the switch is operationally up and running, the Device not found logs are displayed on the console. This issue is observed because the switch attempts to find an older ASIC version and the error messages for the PCI probe failure are enabled in the code. There is no functionality impact or traffic loss due to this issue.

• For secure POAP, ensure that DHCP snooping is enabled and set firewall rules to block unintended or malicious DHCP servers. For more information on POAP, see the Cisco Nexus 9000 Series Fundamentals Configuration Guide.

• When you upgrade from an earlier release to a NX-OS release that supports switch profiles, you have the option to move some of the running-configuration commands to a switch profile. For more information on configuration, see the Cisco Nexus 9000 Series NX-OS System Management Configuration Guide.

• Guest Shell is disabled during an ISSU and reactivated after the upgrade. All applications running in the Guest Shell are affected.

• While performing an ISSU, VRRP and VRRPv3 display these messages:

  • If VRRPv3 is enabled:

    2015 Dec 29 20:41:44 MDP-N9K-6 %$ VDC-1 %$ %USER-0-SYSTEM_MSG: ISSU ERROR: Service
    "vrrpv3" has sent the following message: Feature vrrpv3 is configured. User can change
    vrrpv3 timers to 120 seconds or fine tune these timers based on upgrade time on all Vrrp
    Peers to avoid Vrrp State transitions. – sysmgr

  • If VRRP is enabled:

    2015 Dec 29 20:45:10 MDP-N9K-6 %$ VDC-1 %$ %USER-0-SYSTEM_MSG: ISSU ERROR: Service "vrrp-
    eng" has sent the following message: Feature vrrp is configured. User can change vrrp
    timers to 120 seconds or fine tune these timers based on upgrade time on all Vrrp Peers to
    avoid Vrrp State transitions. – sysmgr

• An error occurs when you try to perform an ISSU if you changed the reserved VLAN without entering the copy running-config save-config and reload commands.

Software image and SMU

• Beginning with NX-OS Release 10.5(1)F, s1 image is introduced specifically for Nexus 9800 switches.

  Upgrade of Nexus 9800 switches from earlier releases that have cs image format to the s1 image format in NX-OS Release 10.5(1)F and later is supported.

• Beginning with NX-OS Release 10.4(2)F, for Nexus 9300-R platforms, to upgrade bios to the latest version you should first upgrade to nxos image. This release onwards, the install all nxos command only upgrades the nxos sw to the latest version but the bios image will be upgraded to the last bios released prior to 10.4(2)F version.

  To upgrade to bios released with 10.4(2)F or higher version, first upgrade the nxos image and then use bios-force option to upgrade the bios. For example,

  1. Install all nxos bootflash:nxos64-msll.10.4.2.F.bin.

     The system reloads and boots up with 10.4(2)F image.

  2. Install all nxos bios-force.

     Note

     The switch reloads twice, once for nxos upgrade and then again for bios upgrade.

• For platforms that need to be upgraded from any release to nxos64-cs.10.3(1)F or higher release, use nxos.9.3.10.bin or nxos64-cs.10.2(3)F or higher release as an interim hop. This restriction is applicable to both disruptive and non-disruptive upgrades. The nxos64-msll.10.3(1)F does not have this restriction.

• Loading an unsupported image on Nexus 9800 platform switches cause the switch to be stuck. Only a power cycle can reset it.

• Beginning with NX-OS Release 10.2(2)F, Nexus 9504 and 9508 platform switches, and Nexus 9508-R, R2, and RX line cards support NX-OS 64-bit images. Disruptive upgrade from earlier releases to 10.2(2)F 64-bit NX-OS image is supported. NX-OS 32-bit image is not supported on these platform switches anymore.

• Beginning with NX-OS Release 10.1(x), when an existing SMU is active, if you install a bundle that contains the existing active SMU, the installer installs only the non-existing SMUs.

• The install all command is the recommended method for software upgrades because it performs configuration compatibility checks and BIOS upgrades automatically. In contrast, changing the boot variables and reloading the device bypasses these checks and the BIOS upgrade and therefore is not recommended.

  Note

  For Nexus 9500 platform switches with -R line cards, you must save the configuration and reload the device to upgrade from NX-OS Release 7.0(3)F3(5) to 9.3(1). To upgrade from NX-OS Release 9.2(2) or 9.2(3), we recommend that you use the install all command.

• You can detect an incomplete or corrupt NX-OS software image prior to performing an upgrade by verifying the MD5, SHA256 or SHA512 checksum of the software image. To verify the MD5 checksum of the software image, run the show file bootflash:<IMAGE-NAME>md5sum command and compare the resulting value to the published MD5 checksum for the software image on the Software Download website. To verify the SHA512 checksum of the software image, run the show file bootflash:<IMAGE-NAME>sha512sum command and compare the resulting value to the published SHA512 checksum for the software image on the Software Download website.

• The install all command is the recommended method for software upgrades because it performs configuration compatibility checks and BIOS upgrades automatically. In contrast, changing the boot variables and reloading the device bypasses these checks and the BIOS upgrade and therefore it is not recommended.

EPLD

• From NX-OS Release 10.6(1)F, while installing nx-os using the install all nx-os command on switches affected by secure boot vulnerability, if the IO FPGA version of the device is lower than the Fixed IO FPGA version, EPLD upgrade does not take place. To upgrade the FPGA, use the install epld command. For more information about switches affected by secure boot vulnerability and Fixed IO FPGA version, refer to Table 1 in the FPGA/EPLD Upgrade Procedure to Address Secure Boot Vulnerability document.

• From NX-OS Release 10.5(3)F, while installing nx-os using the install all nx-os command on switches affected by secure boot vulnerability, EPLD upgrade does not take place. To upgrade the FPGA, use the install epld command. For more information about switches affected by secure boot vulnerability, refer to Table 1 in the FPGA/EPLD Upgrade Procedure to Address Secure Boot Vulnerability document.

• Beginning with NX-OS Release 10.5(3)F, all NX-OS images are bundled with EPLD image and EPLD upgrade is triggered automatically as part of install all nxos command. However, you have the option to skip EPLD image upgrade.

• ISSU supports EPLD image upgrades using install all nxos <nxos-image> epld <epld-image> command, during disruptive system (NX-OS) upgrade. Beginning with NX-OS Release 10.5(3)F, do not use the epld <epld_image> option as the EPLD image is bundled with the NXOS images and a separate EPLD image is no longer provided.

• ISSU is not supported if EPLD is not at NX-OS Release 7.0(3)I3(1) or later.

Release specific

• Upgrade from release 10.4(6)F to 10.5(1)F, 10.5(2)F, or 10.5(3)F releases is not supported and can result in configuration loss or its corruption. To upgrade from 10.4(6)M or 10.4(7)M release to 10.6(x) releases, the recommended path is to first upgrade to 10.5(4)M and then to 10.6(x). See CSCwr21007 in the Cisco Nexus 9000 Series NX-OS Release Notes, Release 10.4(6)M and Release 10.4(7)M.

• Beginning with NX-OS Release 10.4(2)F, SR ISSU is not supported with underlay ISIS.

• When upgrading from earlier release to NX-OS Release 10.3(3)F and later, if the hardware rate-limiter span-egress command is configured then it must be removed and reapplied after the upgrade/ISSU is complete.

• Beginning with NX-OS Release 10.3(2)F, 2xSFP Eth10/1-2 are not supported on N9K-C9400-SW-GX2A. However, from NX-OS Release 10.4(2)F onwards, N9K-C9400-SW-GX2A Sup card ports 2xSFP Eth10/1-2 are supported.

• ASIC SFP+ ports Eth10/1-2 are not supported in NX-OS Releases 10.3(2)F, 10.3(3)F, and 10.4(1)F. Beginning with NX-OS Release 10.4(2)F, these ports are supported. However, note that after reloading the system, these ASIC SFP+ Eth10/1-2 ports can take up to 3 minutes to link up.

• While performing an ISSU on the L2 switch in a vPC complex or a LAN scenario, the IGMP group timeout must be configured with higher value as the L2 switch will not be able to forward the reports/queries during the control plane down time. The L2 snooping querier interval must also be matched to the L3 querier interval.

• While performing an ISSU from NX-OS Release 9.3(5), 9.3(6), 9.3(7), 10.1(1), or 10.1(2) to NX-OS Release 10.2(1) or higher release, ISSU is blocked.

• ISSU is blocked when the delay configuration is present in track list Boolean/weight.

• If the IPv6 ND timeouts during ISSU, then the IPv6 BFD session may flap after the ISSU.

• When upgrading directly to NX-OS Release 10.1(x) from any release prior to 7.0(x), the upgrade is disruptive. For a non-disruptive upgrade, an intermediate upgrade to NX-OS Release 9.x is required. We recommend upgrading to the latest release of NX-OS Release 9.3(x) as an intermediate hop for the upgrade. For information about the supported upgrade paths, see the Cisco Nexus 9000 and 3000 Upgrade and ISSU Matrix.

• When upgrading from NX-OS Release 7.0(3)I6(1) or 7.0(3)I7(1) to NX-OS Release 10.1(x), if the Nexus 9000 Series switches are running vPC and they are connected to an IOS-based switch via Layer 2 vPC, there is a likelihood that the Layer 2 port channel on the IOS side will become error disabled. The workaround is to disable the spanning-tree etherchannel guard misconfig command on the IOS switch before starting the upgrade process.

  Once both the Nexus 9000 Series switches are upgraded, you can re-enable the command.

• If you are upgrading from NX-OS Release 7.0(3)I5(2) to NX-OS Release 10.1(x) by using the install all command, BIOS will not be upgraded due to CSCve24965. When the upgrade to NX-OS Release 10.1(x) is complete, use the install all command again to complete the BIOS upgrade, if applicable.

• When upgrading from NX-OS Release 9.2(4) or earlier releases to NX-OS Release 9.3(4) or later, running configuration contains extra TCAM configuration lines. You can ignore these extra lines as they do not have an effect on the upgrade and configuration.

• When performing an ISSU from NX-OS Release 9.3(1) or 9.3(2) to NX-OS Release 9.3(3) or later, ensure that the features with user-defined ports, such as <ssh port>, are within the prescribed port range. If the port range is incorrect, follow the syslog message recommendation. For more information about the port range, see Cisco Nexus 9000 Series NX-OS IP SLAs Configuration Guide.

• For any prior release version upgrading to NX-OS Release 9.3(5) using ISSU, if the following logging level commands are configured, they are missing in the upgraded version and must be reconfigured:

  • logging level evmc value

  • logging level mvsh value

  • logging level fs-daemon value

• For any prior release version upgrading to NX-OS Release 9.3(6) using ISSU, if the following logging level commands are configured, they are missing in the upgraded version and must be reconfigured:

  • logging level evmc value

  • logging level mvsh value

Switch specific

• During an ISSU on a Nexus 9300 Series switch, all First-Hop Redundancy Protocols (FHRPs) will cause the other peer to become active if the node undergoing the ISSU is active.

• Beginning with NX-OS Release 10.6(2)F, Nexus 9300-GX2, H2R, and H1 series switches support non-disruptive ISSU on switches that have MACsec-enabled interfaces.

• While performing non-disruptive ISSU from NX-OS Release 10.4(6)M to 10.6(1)F and later releases, on Nexus 9300-FX switches and line cards, IGMP traffic is forwarded on vPC legs towards the vPC pair. When there are multiple FEX devices on the vPC peer undergoing ISSU, multicast traffic loss can occur during the upgrade of the FEX devices. To resolve this, configure the ip igmp group-timeout 450 command on all VLANs that carry IGMP traffic across the vPC peer link.

• Beginning with NX-OS Release 10.4(3)F, non-disruptive ISSU is not supported on Nexus 92348GC-X.

• Beginning with NX-OS Release 10.4(1)F, only the LXC mode is supported on Nexus 9300-FX and 9300-FX2 switches, in addition to Nexus 9300-FX3 and 9300-GX switches. This allows you to perform enhanced non-disruptive ISSU with minimal downtime. However, on the rest of the Nexus 9000 switches, you have an option to perform a non-disruptive ISSU in the enhanced LXC mode with minimal downtime.

• Beginning with NX-OS Release 10.4(1)F, only the enhanced LXC mode is supported on Nexus N9K-C9332D-H2R, N9K-C9348GC-FX3, and N9K-C9348GC-FX3PH switches by default.

• Non-disruptive ISSU is not supported on interfaces with 2.5G or 5G speed on N9K-C93108TC-FX3P platform. For more information, see CSCwq38959.

• After disruptive upgrade from NX-OS Release 10.3(x) to 10.5(x) and later releases, Nexus 9300-FX switches lose FC ports configuration and the FC ports turn into Ethernet ports. However, if port x - y mode fc exists under slot z in the running configuration, though such ports are changed to Ethernet ports, after a switch reload, they change back to FC ports.

• Beginning with NX-OS Release 10.2(2)F, FCoE/FC NPV is supported on N9K-C9336C-FX2-E platform switches.

  ISSU with with FCoE (Fiber Channel over Ethernet)/FC (Fiber Channel) NPV (N-port Virtualization) is supported on some Nexus 9000 switches. An ISSU allows you to upgrade the device software while the switch continues to forward traffic. You can perform an in-service software upgrade (ISSU), also known as a nondisruptive upgrade, for some Nexus 9000 switches. The default upgrade process is disruptive. Using the nondisruptive option helps ensure a nondisruptive upgrade.

  Fibre Channel N-port Virtualization (NPV) can co-exist with VXLAN on different fabric uplinks but on same or different front panel ports on the Nexus 93180YC-FX, N9K-C9336C-FX2-E, and N9k-C93360YC-FX2 switches.

• Before enabling the FHS on the interface, we recommend that you carve the ifacl TCAM region on Nexus 9300 and 9500 platform switches. If you carved the ifacl TCAM region in a previous release, you must reload the system after upgrading to NX-OS Release 10.1(x). Uploading the system creates the required match qualifiers for the FHS TCAM region, ifacl.

• Before enabling the FHS, we recommend that you carve the ing-redirect TCAM region on Nexus 9200 and 9300-EX platform switches. If you carved the ing-redirect TCAM region in a previous release, you must reload the system after upgrading to NX-OS Release 10.1(x). Uploading the system creates the required match qualifiers for the FHS TCAM region, ing-redirect.

• When upgrading from Nexus 94xx, 95xx, and 96xx line cards to Nexus 9732C-EX line cards and their fabric modules, upgrade the NX-OS software before inserting the line cards and fabric modules. Failure to do so can cause a diagnostic failure on the line card and no TCAM space to be allocated. You must use the write_erase command followed by the reload command.

Disruptive and non-disruptive ISSU

• When upgrading Nexus 9300-FX2 switch from NX-OS Release 10.5(3)F to any later releases, only disruptive upgrade is supported, and ND ISSU is not supported when the system is enabled with routing template security group. However, ND ISSU is supported from NX-OS Release 10.6(1)F.

• While performing ND ISSU, if a router is configured with BGP prefix peers, prefix-peer-timeout (default value - 30s) should be greater than GR timer (default value - 120s), to allow the prefix peers to resume the connection after ISSU.

• While performing multi-hop ND ISSU upgrade to higher releases, use 10.3(5)M or higher release as an intermediate hop.

  • If the switch has been previously upgraded using multi-hop ND ISSU and experiences unexpected CoPP drops, open a TAC case to determine if remediation is required.

• Beginning with NX-OS Release 10.2(3)F, non-disruptive ISSU is supported for VPC fabric peering on all Nexus 9300-X TORs. Both standard and enhanced non-disruptive upgrades are supported. Note that ISSU should be started or triggered when there is no failure. An example for failure would be one of the VPC legs is down.

• The recommended routing protocol graceful restart timer is 600 seconds and nve source-interface hold-down-time is 400 seconds.

• It is recommended to set disable-fka on VFC interfaces in E or F mode, when invoking ND native ISSU on switch mode testbed. If not, it can be disruptive.

• Disruptive upgrade from any version before 9.3(10) or 10.2(3)F may fail due to CSCwb63451. You must upgrade to 9.3(10) or 10.2(3)F first, before upgrading to 10.3(1)F or later.

• Beginning from NX-OS Release 10.2(8)M onwards, Nexus 9300-FX3 supports non-disruptive upgrade.

• Beginning with NX-OS Release 10.2(3)F, for switches that are in LXC mode and for non-desruptive upgrade, a new option skip-kernel-upgrade is added to install command.

• MPLS strip, GRE strip, and any underlying ACL configuration is not ISSU compatible when you perform ND ISSU to NX-OS Release 10.2(2)F from a previous release.

  After ND ISSU to NX-OS Release 10.2(2)F or 10.2(3)F from a previous release, post GRE strip dot1q tunnel VLAN_tag might be missing. Workaround for this issue is to remove and add port ACL from L2 interfaces for GRE strip enabled interface.

• For a device that is running on NX-OS Release 10.1(2), 10.2(1)F, and 10.2(2)F, ND-ISSU is not supported if Layer 2 sub-interfaces are configured.

• When performing ND ISSU using BGP non-default hold timers, ensure that the BGP graceful-restart timer is reasonably long enough, for example, 180 seconds.

• If there is a VRF scale, for a non-disruptive ISSU under each VRF, you must configure graceful restart timer to 300 seconds.

• OpenFlow and LACP fast timer rate configurations are not supported for Non-Disruptive ISSU.

• Beginning with NX-OS Release 9.3(5), standard, nondisruptive ISSU, on switches that are configured with uRPF, is supported on:

  • Nexus 9300-EX platform switches

  • Nexus 9300-FX/FX2 platform switches, and

  • Nexus 9300-GX platform switches

  Note

  Prior to NX-OS Release 9.3(5), if any of the above switches were configured with uRPF, standard, non-disruptive ISSU was not supported.

Enhanced mode

• Beginning with NX-OS Release 10.3(3)F, only the LXC mode is supported on Nexus 9300-FX3 and 9300-GX switches, which allows you to perform enhanced non-disruptive ISSU with minimal downtime.

• ND ISSU can be performed in LXC mode in two methods:

  • ND ISSU in LXC mode - Switchover-based ISSU that is similar to EOR. Second SUP is brought up in new container and switchover is done. The second SUP now becomes the new active. There is no change to the kernel.

  • Fallback ND LXC ISSU - This is only done when the above switchover-based ISSU cannot be done (SRG Kernel incompatible or less memory). The kernel is upgraded.

  • skip-kernel-upgrade option will force ND ISSU in LXC mode - Switchover-based ISSU (even in case when running) and target kernels are incompatible.

• For switches that are in LXC boot mode, the enhanced LXC upgrade will fall back to standard ND ISSU as the target image kernels are likely be different than the current image.

Feature specific

• From NX-OS Release 10.6(1)F, on Nexus modular switches, if the Backplane diagnostic test fails and a BACKPLANE_AUTHENTICATION_FAIL syslog appears, do not perform an upgrade or a system reload.

• While upgrading from an earlier release to 10.5(3)F or later releases, as a part of sFlow ISSU Consistency Checker, pre and post configuration files are created in the booflash. To remove the snapshot files, use the clear system internal sflow consistency pss-snapshot command. However, if the snapshot files are removed, the show system internal sflow consistency issu-pss command does not provide the expected output. For more information, refer to Cisco Nexus 9000 Series NX-OS System Management Configuration Guide.

• When the PTP feature is configured on Nexus 9300-H2R and C93400LD-H1 switches, performing an ND ISSU upgrade from version 10.4(1)F to 10.5(2)F triggers a kernel panic, resulting in an additional switch reload and causing a traffic outage. For more information, refer to CSCwh34732.

• While upgrading from NX-OS releases prior to 10.4(3) to 10.4(3) or later releases with mode tap-aggregation command enabled on the Layer 2 interface, make sure to enable the global hardware acl tap-agg command and reload.

• Beginning with NX-OS Release 10.4(2)F, non-disruptive ISSU is supported for segment routing traffic engineering (SR-TE) features with BGP as underlay on Nexus 9300 and 92348GC-X platform switches; and on 9300-HX platform switches from NX-OS Release 10.4(3)F. However, the following features are not supported on nondisruptive ISSU:

  • SR L2EVPN

  • ISIS and OSPF underlay

  • vPC configuration with segment-routing

  • Egress Peer engineering

  • Segment routing and GRE co existence

• While upgrading from NX-OS releases prior to 10.2(2)F to 10.2(2)F or later releases, configure the mode tap-aggregation command before attaching TapAgg ACLs on Layer 2 interface.

• If you upgrade from a NX-OS release that supports the CoPP feature to a NX-OS release that supports the CoPP feature with additional classes for new protocols, you must either run the setup utility using the setup command or use the copp profile command for the new CoPP classes to be available. For more information on these commands of configuring control plane policing, see the Cisco Nexus 9000 Series NX-OS Security Configuration Guide.

• Beginning with NX-OS Release 10.1(2), CoPP is supported on N9K-X9624D-R2 and N9K-C9508-FM-R2 platform switches.

• Beginning with NX-OS Release 10.1(2), RACL is supported on N9K-X9624D-R2 and N9K-C9508-FM-R2 platform switches.

• Beginning with NX-OS Release 10.1(1), during the disruptive upgrade to the 64-bit image or a downgrade from 64-bit to 32-bit image, if feature ITD is enabled, refer to Guidelines and Limitations for ITD in the Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 10.1(x), if the upgrade or downgrade proceeds with an ASIC reload.

• Beginning with NX-OS Release 10.1(1), Fs_daemon does not support snmpwalk on devices with more than 5000 files. When performing snmpwalk on a device with more than 5000 files, the error resourceUnavailable (This is likely a out-of-memory failure within the agent) is an expected behavior.

• When you upgrade a Nexus 9000 device to NX-OS Release 10.1(x), if a QSFP port is configured with the manual breakout command and is using a QSA, the configuration of the interface Ethernet 1/50/1 is no longer supported and must be removed. To restore the configuration, you must manually configure the interface Ethernet 1/50 on the device.

• When upgrading from NX-OS Release 9.3(3) to NX-OS Release 9.3(6) or later, if you do not retain configurations of the TRM enabled VRFs from NX-OS Release 9.3(3), or if you create new VRFs after the upgrade, the auto-generation of ip multicast multipath s-g-hash next-hop-based command, when feature ngmvpn is enabled, will not happen. You must enable the command manually for each TRM enabled VRF. For the configuration instructions, see the Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide.

• Upgrading from NX-OS Release 9.3(1), 9.3(2) or 9.3(3) to a higher release, with Embedded Event Manager (EEM) configurations that are saved to the running configuration, may cause a DME error to be presented. The error is in the output of the show consistency-checker dme running-config enhanced command, specifically, the event manager commands. If this error occurs, delete all EEM applet configurations after completing the ISSU, then reapply the EEM configurations.

• When redistributing static routes, NX-OS requires the default-information originate command to successfully redistribute the default static route starting in 7.0(3)I7(6).

FEX

• When upgrading from NX-OS Release 9.3x to NX-OS Release 10.4x using the install all command, ensure that storm control is disabled on all attached Fabric Extenders (FEXs). If you do not disable storm control, the switch fails to boot up after the upgrade. (See CSCws43646.)

• Beginning with NX-OS Release 10.2(2)F, ND ISSU is supported for FEX and you need to re-adjust the BGP graceful-restart restart time command for the upgrade to work non-disruptively. This must be done for each FEX upgrade one-by-one.

  The following example shows the time taken to re-adjust bgp-graceful restart-time for each non-disruptive FEX upgrade.

  In the Non-disruptive upgrade with FEX, each FEX will upgrade taking about 90 secondss 
  (1.5 minutess) sequentially (one-by-one and not a parallel upgrade). 
    Total non-disruptive upgrade time for all FEX = No. of fex * time taken per fex
                                       For 10 FEX = 10 * 90
                                                  = 900 seconds or 15 minutes

• When you upgrade a Nexus 9000 switch from NX-OS Release 7.x with an attached FEX in straight-through mode to 9.x and then to 10.x, the FEX Layer 2 Host Interface (HIF) configuration can be lost after upgrading to a 10.x Release. This occurs due to a design change in handling Layer 2 FEX HIF ports at boot time from Release 9.x to 10.x.

  Note

  The issue occurs only for FEX connected in a straight-through mode and not for dual-homed (A-A) mode.

  To resolve this, run the following non-intrusive commands before upgrading the switch from 9.x to 10.x:

  1. Apply no switchport only on all Layer 3 (L3) physical and Layer 3 (L3) port-channel interfaces. For example,

     switch(config)# interface e1/1
     switch(config-if)# no switchport
     

  2. Configure system default switchport globally and save the configuration. For example,

     switch(config)# system default switchport
     switch(config)# copy r s
     

  Note

  The issue does not occur if:

  • the switch was originally booted in 9.x with an attached FEX and then upgraded to 10.x.

  • the switch was upgraded from 7.x to 9.x without an attached FEX, and the FEX was added later in 9.x before upgrading to 10.x.

• An upgrade that is performed via the install all command for NX-OS Release 7.0(3)I2(2b) to Release 10.1(x) might result in the VLANs being unable to be added to the existing FEX HIF trunk ports. To recover from this, the following steps should be performed after all FEXs have come online and the HIFs are operationally up:

  1. Enter the copy run bootflash:fex_config_restore.cfg command at the prompt.

  2. Enter the copy bootflash:fex_config_restore.cfg running-config echo-commands command at the prompt.

• In NX-OS Release 7.0(3)I6(1) and earlier, performing an ASCII replay or running the copy file run command on a FEX HIF configuration requires manually reapplying the FEX configuration after the FEX comes back up.

FC/FCoE NPV

• Beginning with NX-OS Release 10.1(1), ISSU is supported on FC/FCoE switch mode on Nexus 93360YC-FX2. For more information about the FC/FCoE switch mode and supported hardware, see the Cisco Nexus 9000 Series NX-OS SAN Switching Configuration Guide.

• Beginning with NX-OS Release 10.1(1), enhanced ISSU is supported on FC/FCoE switch mode for Nexus 93180YC-FX and 93360YC-FX2 switches. For more information about the FC/FCoE switch mode and supported hardware, see the Cisco Nexus 9000 Series NX-OS SAN Switching Configuration Guide.

• Beginning with NX-OS Release 10.1(1), Enhanced ISSU is supported on FC/FCoE NPV mode for Nexus 93180YC-FX and 93360YC-FX2 switches. For more information about the FC/FCoE NPV mode and supported hardware, see the Cisco Nexus 9000 Series NX-OS FC-NPV and FCoE NPV Configuration Guide.

VXLAN with TRM

Caution

Following changes must be done during a maintenance window.

After upgrading a NX-OS 9000 Series switches configured with VXLAN (specifically VRF-related configurations) from NX-OS Release 7.x through 9.3 to 10.3(6) or earlier, two issues arise:

• The startup-config displays both legacy and new Layer 3 VNID configuration modes

• TRM traffic’s RPF changes to the new mode for S,Gs, causing multicast traffic forwarding problems.

To avoid these issues, follow these steps:

• Enable the REST configuration input using the following commands:

  feature nxapi
              nxapi http port 80

• Open a browser and enter the management IP address of the switch. This will open the Sandbox page. Use the same credentials as the switch admin login to sign in.

• In the top input textbox, enter the following command for each VRF that has an issue with the VNI ID:

  vrf context tenant-1 
          no vni 50000 l3

• On the right side of the page, set the Method to NXAPI-REST(DME) and keep the Input Type as cli.

• Click the Convert (with DN) button in the middle of the page. This will generate the XML equivalent of the configuration change.

• When the XML appears in the second textbox, click Send to apply the changes and remove the VNI ID configuration from the switch.

• To ensure the changes are applied, run the command:

  copy running-config startup-config

Unsupported PIDs

The table displays the list of unsupported PIDs from various NX-OS Releases.