Guest

Cisco Nexus 5000 Series Switches

Cisco Nexus 5000 Series NX-OS Software Upgrade and Downgrade Guide, Release 5.1

  • Viewing Options

  • PDF (2.3 MB)
  • Feedback
Cisco Nexus 5000 Series NX-OS Software Upgrade and Downgrade Guide, Release 5.1

Table Of Contents

Cisco Nexus 5000 Series NX-OS Software Upgrade and Downgrade Guide, Release 5.1

Information About Software Images

Supported Hardware

Cisco Nexus 5000 Series Switches and Associated Expansion Modules

Cisco Nexus 5500 Platform Switches and Associated Expansion Modules

Cisco Nexus 2000 Series Fabric Extenders

Upgrade Guidelines

Using the Install All Command

Upgrading the BIOS and Power Sequencer Images

Impact of Upgrades and Downgrades

In-Service Software Upgrades

ISSU and Layer 3

ISSU Supported Topologies

ISSU Support For Cisco Nexus 2000 Series Fabric Extenders

ISSU Support for vPC Topologies

ISSU Support for vPC Topologies with Fabric Extenders

ISSU Support With Fibre Channel and FCoE Topologies

Summary of ISSU-Supported Topologies

Summary of ISSU Unsupported Topologies

ISSU Prerequisites

Management Services After an ISSU

Fiber Channel/FCoE Protocol and Services During an ISSU

Layer-2 Protocols Impact

Ethernet Interfaces on the Switch and the Fabric Extenders

PreInstallation Checks

Upgrading Procedures

Installation At-A-Glance

Copying the Running Configuration from an External Flash Memory Device

Copying the Startup Configuration from an External Flash Memory Device

Upgrade Process in a Non-vPC Topology

Upgrade Process for a vPC Topology on the Primary Switch

Upgrade Process for a vPC Topology on the Secondary Switch

Disruptive Installation Process

Forcing an Upgrade

Upgrading From Cisco NX-OS Release 4.1(3)N2(1) and Earlier Releases (Disruptive Upgrade)

Minimizing the Impact of a Disruptive Upgrade

Upgrading a Direct vPC or a Single-Homed FEX Access Layer

Upgrading a Dual-Homed FEX Access Layer

Monitoring the Upgrade Status

Downgrading from a Higher Release

Troubleshooting ISSUs and Disruptive Installations

Related Documentation

Release Notes

Configuration Guides

Maintain and Operate Guides

Installation and Upgrade Guides

Licensing Guide

Command References

Technical References

Error and System Messages

Troubleshooting Guide

Obtaining Documentation and Submitting a Service Request


Cisco Nexus 5000 Series NX-OS Software Upgrade and Downgrade Guide, Release 5.1


First Published: December 2011
Last Modified: May 2012

This document describes how to upgrade or downgrade Cisco NX-OS software on Cisco Nexus 5000 Series switches and Cisco Nexus 2000 Series Fabric Extenders. Use this document in combination with documents listed in the "Related Documentation" section on page 46.

This document includes these sections:

Information About Software Images, page 2

Supported Hardware, page 2

Upgrade Guidelines, page 3

Using the Install All Command, page 5

In-Service Software Upgrades, page 6

Upgrading Procedures, page 22

Disruptive Installation Process, page 30

Forcing an Upgrade, page 30

Upgrading From Cisco NX-OS Release 4.1(3)N2(1) and Earlier Releases (Disruptive Upgrade), page 31

Monitoring the Upgrade Status, page 45

Downgrading from a Higher Release, page 46

Troubleshooting ISSUs and Disruptive Installations, page 46

Related Documentation, page 46

Information About Software Images

Cisco Nexus 5000 Series switches are shipped with the Cisco NX-OS software preinstalled on the switches. Before upgrading or downgrading from an existing image, you should read through the information in this document to understand the guidelines, prerequisites, and procedures for upgrading the software. For updated information about the Cisco NX-OS software for the Cisco Nexus 5000 Series switch, see the Cisco Nexus 5000 Series and Cisco Nexus 2000 Series Fabric Extender Release Notes.

The Cisco NX-OS software consists of the kickstart image and the system image. The system image includes the software for the Cisco Nexus 5000 Series switch and the Cisco Nexus 2000 Series Fabric Extenders (FEXs) that are connected to the switch. The images contain a major release identifier, a minor release identifier, and a maintenance release identifier, and they can also contain a rebuild identifier, which may also be referred to as a support patch. The following figure shows the version identifiers that are used with a combination of platform-independent and platform-dependent schemes for the Cisco NX-OS software.

Figure 1 Cisco NX-OS Version Identifies

The platform designator is N for the Nexus 5000 Series Switches, E for the Nexus 4000 Series Switches, and S for the Nexus 1000 Series Switches. Applicable features, functions, and fixes in the platform-independent code are present in the platform-dependent release.

Aplicable fixes in the Cisco NX-OS Software Release 4.0(1a) are present in the 4.0(1a)N1(1a) release.

Supported Hardware

Cisco Nexus 5000 Series switches are shipped with the Cisco NX-OS software preinstalled. Cisco NX-OS upgrades and downgrades are supported on the hardware listed in the following sections:

Cisco Nexus 5000 Series Switches and Associated Expansion Modules

Cisco Nexus 5000 2RU Chassis (N5K-C5020P-BF)

Cisco Nexus 5000 1RU Chassis (N5K-C5010P-BF)

Cisco Nexus 5000 Series Module 6-port 10 Gigabit Ethernet (N5K-M1600)

Cisco Nexus 5000 Series Module 4x10GE 4xFC 4/2/1 (N5K-M1404)

Cisco Nexus 5000 Series Module 6xFC 8/4/2/1 (N5K-M1060)

Cisco Nexus 5000 Series Module 8xFC 4/2/1 (N5K-M1008)

Cisco Nexus 5500 Platform Switches and Associated Expansion Modules

Cisco Nexus 5500 1RU Chassis (N5K-C5548P-FA)

Cisco Nexus 5500 Series 8x10GE 8xFC 8/4/2/1 (N55-M8P8FP)

Cisco Nexus 5500 Series 16-port 10GE (N55-M16FP)

Cisco Nexus 2000 Series Fabric Extenders

Cisco Nexus 2148T Fabric Extender

Cisco Nexus 2248TP Fabric Extender

Cisco Nexus 2224TP Fabric Extender

Cisco Nexus 2232PP Fabric Extender

Cisco Nexus 2232TM Fabric Extender

Cisco Nexus 2232TT Fabric Extender

Cisco Nexus 2248T Fabric Extender

Cisco Nexus 2248TP-E Fabric Extender

Upgrade Guidelines

When upgrading system software, follow these guidelines:


Caution An upgrade from Cisco NX-OS Release 4.2(1)N2(1) to Cisco NX-OS Release 5.1(3)N1(1) is disruptive. To have a non-disruptive upgrade, upgrade from Cisco NX-OS Release 4.2(1)N2(1) to Cisco NX-OS Release 5.0(3)N2(1) and then upgrade to Cisco NX-OS Release 5.1(3)N1(1).


Note Following an upgrade to Cisco NX-OS Release 5.1(3)N1(1a) on a Cisco Nexus 5000 or 5500 Series switch, the VLAN configuration is added to the running configuration. This change appears if the upgrade is nondisruptive or disruptive. The addition of the VLAN configuration is the result of the conversion of some CLI commands in an older version of Cisco NX-OS software to a newer version. The operation of the switch is not affected by the VLAN configuration being a part of the running configuration.


Configuration changes

You cannot enter configuration mode during an upgrade. You should save, commit, or discard any active configuration sessions before upgrading or downgrading the Cisco NX-OS software image. The active configuration session is deleted without a warning during a reload.

Use the show configuration session summary command to verify that there are no active configuration sessions.

switch# show configuration session summary
There are no active configuration sessions
 
   

For more information on configuration sessions, see the Cisco Nexus 5000 Series NX-OS System Management Configuration Guide.


Note CLI and SNMP configuration change requests are denied during a in-service software upgrade (ISSU).


Topology—You should make topology changes such as Spanning Tree Protocol (STP) or Fiber Channel (FC) fabric changes, that affect zoning, Fabric Shortest Path First (FSPF), or FC domains before you perform an upgrade. You should perform module installations or removals only before or after an upgrade.

Scheduling—You should upgrade when your network is stable and steady. Ensure that everyone who has access to the switch or the network is not configuring the switch or the network during this time. You cannot configure a switch during an upgrade.

Space—Verify that sufficient space is available in the location where you are copying the images. The internal bootflash requires approximately 200 MB of free space.

Hardware—Avoid power interruptions during an installation procedure. Power interruptions can corrupt the software image.

Connectivity to remote servers

Configure the IPv4 address or IPv6 address for the 10/100/1000 BASE-T Ethernet port connection (interface mgmt0). Ensure that the switch has a route to the remote server. The switch and the remote server must be in the same subnetwork if you do not have a router to route traffic between subnets.

Software image—Ensure that the specified system and kickstart images are compatible with each other as follows:

If the kickstart image is not specified, the switch uses the current running kickstart image.

If you specify a different system image, ensure that it is compatible with the running kickstart image.

Retrieve compatible images in one of two ways:

Locally—Images are locally available on the switch.

Remotely—Images are in a remote location and you specify the destination using the remote server parameters and the filename to be used locally.

Command—Use the following commands to prepare for and install the new software:

Use the ping command to verify connectivity to the remote server.

Use the dir command to verify the required space is available for the image files to be copied.

Use the show install all impact command to identify the upgrade impact. This command displays information describing the impact of the upgrade on each Fabric Extender such as the current and upgrade-image versions. This command also displays if the upgrade is disruptive or the reason why the upgrade is disruptive, if the Fabric Extender needs to be rebooted, and the reason why it needs to be rebooted.


Note We recommended that you log in to the console port to begin the upgrade process. In Virtual Port Channel (vPC) topologies, the first upgrade can be performed on either the primary or secondary switch in the topology


Terminology

Table 1-1 summarizes the terms used in the install all command output to verify module and software image compatibility.

Table 1-1 install all Commands and Output Terminology

Term
Definition

bootable

Ability of the module to boot or not boot based on image compatibility.

Impact

Type of software upgrade mechanism—disruptive or nondisruptive.

install-type reset

sw-reset

rolling

copy-only

Resets the module.

Resets the module immediately after a switchover.

Upgrades each module in sequence.

Updates the software for BIOS, loader, or boot ROM.

force

Option to force a disruptive upgrade, even when an ISSU is possible


Using the Install All Command

The install all command triggers an ISSU on Cisco Nexus 5000 Series switches and Cisco Nexus 2000 Series Fabric Extenders. The following images are upgraded during the installation:

Kickstart image

System image

Fabric Extender image

System BIOS

Power sequencers on the system

The install-all command provides the following benefits:

You can upgrade the Cisco Nexus 5000 Series switches and the Nexus 2000 Series Fabric Extenders using just one command.

You can receive descriptive information about the intended changes to your system before you continue with the installation. For example, it identifies potential disruptive upgrades.

You can continue or cancel the upgrade when you see this question (the default is no):

Do you want to continue (y/n) [n] : y
 
   

You can upgrade the Cisco NX-OS software using a non disruptive procedure, when supported.

The command automatically checks the image integrity, which includes the running kickstart and system images. The command sets the kickstart and system boot variables.

The command performs a platform validity check to verify that a wrong image is not used.

Pressing Ctrl +C gracefully ends the install all command. The command sequence completes the update step in progress and returns to the EXEC prompt.

After entering the install all command, if any step in the sequence fails, the upgrade ends.

The following message appears to warn you about the impact of upgrading the power sequencer:

Warning: please do not remove or power off the module at this time.
Note: Power-seq upgrade needs a power-cycle to take into effect.
 
   

Note After a successful power sequence upgrade, you must switch off the power to the system and then power it up.


You can force a disruptive upgrade. For information on forcing an upgrade, see Forcing an Upgrade.

Upgrading the BIOS and Power Sequencer Images

Changes to BIOS and power sequencers are rare; however, when they occur, they are included in the Cisco NX-OS system image, and the BIOS and power sequencer are upgraded. The summary displayed by the installer during the installation process indicates the current version of the BIOS and power sequencer and the target version.


Note After a successful power sequence upgrade, you must switch off the power to the system and then power it up.


Impact of Upgrades and Downgrades

Cisco NX-OS supports in-service software upgrades (ISSUs) that allow a Cisco Nexus 5000 Series switch and any connected FEXs to be upgraded without any traffic disruption (with a brief control plane disruption). A few conditions have to be met for the system to be upgraded via ISSU process - the access layer topology should be ISSU compliant, the current and target versions should be ISSU capable and the network should be stable.

If the conditions required for ISSU are not met or if the user intends to downgrade the software version, the installation process will be disruptive. For example the Cisco Nexus 5000 Series switch and any connected FEX are rebooted, which causes a disruption. If Cisco's virtual port channel (vPC) is configured on Cisco Nexus 5000 Series switches, it is possible to achieve an upgrade/downgrade with very minimal traffic disruption to servers/hosts.

In-Service Software Upgrades

Support for an ISSU on Cisco Nexus 5000 Series switches and Cisco Nexus 2000 Series Fabric Extenders was added with the 4.2(1)N1(1) NX-OS software release.

With a single supervisor system like a Cisco Nexus 5000 Series switch, an ISSU on the Cisco Nexus 5000 Series switch causes the supervisor CPU to reset and load the new software version. The control plane is inactive, but the data plane keeps forwarding packets that lead to an upgrade with no service disruption. After the CPU loads the updated version of Cisco NX-OS, the system restores the control plane to a previous ly known configuration and the runtime state and the data plane are synchronized, because the data plane keeps forwarding packets while the control plane is upgraded, any servers connected to the Cisco Nexus 5000 Series switch access layer should see no traffic disruption.

ISSU and Layer 3

Cisco Nexus 5500 Platform switches support Layer 3 functionality. However, the system cannot be upgraded with the ISSU process (non disruptive upgrade) when Layer 3 is enabled. It is required to unconfigure all Layer 3 features to be able to upgrade in a non disruptive way with an ISSU.

ISSU Supported Topologies

This section includes the following topics:

ISSU Support For Cisco Nexus 2000 Series Fabric Extenders, page 7

ISSU Support for vPC Topologies, page 7

ISSU Support for vPC Topologies with Fabric Extenders, page 9

ISSU Support With Fibre Channel and FCoE Topologies, page 9

Summary of ISSU-Supported Topologies, page 9

Summary of ISSU Unsupported Topologies, page 13

Management Services After an ISSU, page 16

Fiber Channel/FCoE Protocol and Services During an ISSU, page 17

ISSU Support For Cisco Nexus 2000 Series Fabric Extenders

Cisco Nexus 2000 Series Fabric Extenders act as line cards to Cisco Nexus 5000 Series switches. The fabric extenders add flexibility to datacenter networking infrastructure by decoupling the physical and logical (Layer 2) topology, reducing the operation expense by lowering management and troubleshooting points, and building a larger layer 2 fabric that is loop-free, with a single layer of switching.

The ISSU process initiated on the Cisco Nexus 5000 Series switches upgrades the entire access layer including the switch and the FEXs that are connected to the switch.

An ISSU first upgrades the switches. Once the switch is operational with the upgraded software, the FEX's are upgraded. This upgrade of the FEX is done in a rolling fashion, one FEX at a time. This upgrade on the Fabric Extenders is non-disruptive, similar to the upgrade of the switch.

The time required for an ISSU to complete depends on the number of FEX's that are connected. You should plan a maintenance window with the total upgrade time in mind. The entire upgrade is a non disruptive and is not expected to cause any outage to connected servers.

ISSU Support for vPC Topologies

An ISSU is completely supported when two switches are paired in a vPC configuration. In a vPC configuration, one switch functions as a primary switch and the other functions as a secondary switch .They both run the complete switching control plane, but coordinate forwarding decisions to have optimal forwarding to devices at the other end of the vPC. Additionally, the two devices appear as a single device that supports EtherChannel (static and 802.3ad) and provide simultaneously data forwarding services to that device.

While upgrading devices in a vPC topology,you should start with the switch that is the primary switch. The vPC secondary device should be upgraded after the ISSU process completes successfully on the primary device. The two vPC devices continue their control plane communication during the entire ISSU process (except when the ISSU process resets the CPU of the switch being upgraded).

This example shows how to determine the vPC operational role of the switch:

n5020-2# show vpc brief
Legend:
                (*) - local vPC is down, forwarding via vPC peer-link
 
   
vPC domain id                    : 1000
Peer status                      : peer adjacency formed ok      
vPC keep-alive status            : peer is alive                 
Configuration consistency status : success 
vPC role                         : primary                       
Number of vPCs configured        : 247 
Peer Gateway                     : Disabled
Dual-active excluded VLANs       : -

You can monitor the status of an ISSU on the primary from the primary device command, after the primary switch reloads by using the show install all status command.

Any attempt to initiate an upgrade on the vPC peer switch, when ISSU is progress on the other switch, is blocked.


Note During an upgrade, the configuration on peer switches is locked and the vPC state on vPC peer switches is suspended until the upgrade is complete.


Verifying the vPC Status on a Peer Switch During an Upgrade

To view the vPC status, enter the show vpc command on a peer switch as follows:.

switch-2# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
 
   
vPC domain id : 1000
Peer status : peer adjacency formed ok
vPC keep-alive status : Suspended during ISSU
Configuration consistency status: success
vPC role : primary, operational secondary
Number of vPCs configured : 100
Peer Gateway : Disabled
Dual-active excluded VLANs : 40
 
   
vPC Peer-link status
------------------------------------------------
id   Port   Status Active vlans
--   ----   ------ -----------------------------
1    Po100  up     1,40
 
   

The following message is displayed on the vPC peer switch when an ISSU is started on the other switch:

2010 Feb  4 00:07:16 MN5020-4 %$ VDC-1 %$ %VPC-2-VPC_ISSU_START: Peer vPC switch ISSU
start, locking configuration
 
   

Viewing System Messages on Peer Switches

A keepalive message such as the following may appear on a peer switch during an upgrade:

2010 Feb  4 00:09:26 MN5020-4 %$ VDC-1 %$ %VPC-2-PEER_KEEP_ALIVE_RECV_FAIL: In domain
1000, VPC peer keep-alive receive has failed
 
   

Installation status messages such as the following may appear on peer switches as the primary switch is upgraded.

switch-2# 2010 Jun 10 18:27:25 N5K2 %$ VDC-1 %$ %SATCTRL-2-SATCTRL_IMAGE: FEX100 Image
update in progress. 
switch-2# 2010 Jun 10 18:32:54 N5K2 %$ VDC-1 %$ %SATCTRL-2-SATCTRL_IMAGE: FEX100 Image
update complete. Install pending

ISSU Support for vPC Topologies with Fabric Extenders

An ISSU is supported in vPC topologies that include FEXs that are connected in dual-homed topologies to a parent switch and when the FEX is in a single-homed topology.

ISSU Support With Fibre Channel and FCoE Topologies

ISSUs are supported on access layer switches when Fibre Channel and Fibre Channel over Ethernet (FCoE) is enabled. You must ensure that the FC fabric is stable before initiating an ISSU in this topology.

Summary of ISSU-Supported Topologies

The following figure shows an access switch topology.

Figure 2 Access Switch Topology

The following figure shows a vPC peering topology.

Figure 3 vPC Peering Topology

The following figure shows a virtual modular system with static fabric connectivity for FEXs.

Figure 4 Virtual Modular System With Static Fabric Connectivity For FEXs

The following figure shows a vertical modular system.

Figure 5 Virtual Modular System

The following figure shows a vPC-peered dual-supervisor virtual modular system with dual-homed FEXs.

Figure 6 vPC-Peered Dual-Supervisor Virtual Modular System Dual-Homed FEXs

The following figure shows a vPC-peered dual-supervisor virtual modular system with dual-homed and single-homed FEXs.

Figure 7 vPC-Peered Dual-Supervisor Virtual Modular System Dual-Homed and Single-Homed FEXs

The following figure shows a vPC-peered dual-supervisor virtual modular system with dual-homed FEXs.

Figure 8 vPC Peered Dual-Supervisor Virtual Modular System Dual-Homed FEXs

Summary of ISSU Unsupported Topologies

Two important spanning tree-related requirements for a Cisco Nexus 5000 Series switch undergoing an ISSU are as follows. Note that a switch undergoing an ISSU has its control plane inactive while the switch is reset and the new software version is loaded. Not having these restrictions could render the network unstable, if there are any unexpected topology changes:

STP enabled switches cannot be present downstream to the switch undergoing an ISSU.

The STP Bridge Assurance feature cannot be configured except on a vPC peer link. Bridge Assurance is enabled by configuring an interface as a spanning-tree port type network.

If the STP conditions are not met, the installation check will indicate that the upgrade would be disruptive. In this case, you can perform an upgrade at a later time after making necessary changes to the topology to meet these conditions or perform a disruptive upgrade.

The following figure shows a Cisco Nexus 5000 Series switch that is connected to a blade switch that is running STP.

Figure 9 Connection to a Blade Switch that is Running STP

The following figure shows a Cisco Nexus 5000 Series switch that is connected to a downstream switch that is running STP.

Figure 10 Connection to a Downstream Switch that is Running STP

The following figure shows a Cisco Nexus 5000 Series switch that is running Bridge Assurance with another switch.

Figure 11 Cisco Nexus 5000 Series Switch Running Bridge Assurance with Another Switch

The following figure shows dual-homed FEXs connected to a stub switch.

Figure 12 Dual-Homed FEXs Connected to a Stub Switch

The following figure shows a single-homed FEX that is connected to stub switches.

Figure 13 Single-Homed FEX Connected to Stub Switches

The following figure shows a dual-homed FEX that is connected to stub switches.

Figure 14 Dual-Homed FEX Connected to Stub Switches

ISSU Prerequisites

All the upgrade guidelines listed in Cisco NX-OS Upgrade Guidelines section should be strictly adhered to for ISSU to work smoothly. In particular, make sure that the network is stable and no changes are made while an ISSU is in progress. Make sure that you check for feature compatibility between the current running release and the target release.

The following figure shows upgrade restrictions.

Figure 15 Upgrade Restrictions

In addition, there are some specific requirements for a non disruptive upgrade (ISSU).

Topology requirements— A Cisco Nexus 5000 Series switch on which an ISSU is being initiated should not be in one of the unsupported topologies listed in the previous figure. No interface should be in a spanning-tree designated forwarding state. Also, bridge assurance should not be configured on any interface of the Cisco Nexus 5000 series switch. vPC peer-link is an exception to these requirements.

Layer 2 requirement— The ISSU process will be aborted if the system has any Link Agregration Control Protocol (LACP) fast timers configured.

FC/FCoE requirements—Check that the topology is stable for an ISSU to work smoothly. The following is a list of things you must check:

Domain Manager—As part of the installation process, domain manager checks if the fabric is in a stable state. If the fabric is not stable, the installation will abort.

CFS—As part of the installation process, CFS checks if any application (ntp,fsm, rcsn, fctime) is locked. If any application is holding a CFS lock, the installation will abort.

Zone Server— The installation process aborts if a zone merge or zone change request is in progress.

FSPF—As part of the upgrade process, Fabric Shortest Path First (FSPF) verifies if the configured interface dead interval is more than 80 seconds; otherwise, installation will abort.

Management Services After an ISSU

Before the switch is reset for an ISSU, inband and management ports are brought down and are brought back up after the ISSU completes. Services that depend on the inband and management ports are impacted during this time.

Table 1-2 Inband and Management Ports Services Impacted During ISSU Reset

Service
Description

Telnet/SSH

When an ISSU resets the system to load the target Cisco NX-OS version, all Telnet/SSH sessions are disconnected and need to be reestablished after the ISSU completes.

AAA/RADIUS

Applications that leverage the AAA Service (such as login) are disabled during an ISSU, because all Network Management services are disabled during this time, this behavior is consistent.

HTTP

HTTP sessions to the switch are disconnected during an ISSU reboot. After the reboot, the HTTP is restarted and the switch will accept an HTTP sessions.

NTP

NTP sessions to and from the switch are disrupted during an ISSU reboot. After the reboot, NTP session are reestablished based on the saved startup configuration.


Fiber Channel/FCoE Protocol and Services During an ISSU

During an ISSU, the control plane is offline for up to 80 seconds. Any state changes in the network during this time are not processed. Depending on the change, the impact may vary. We always recommended that you ensure a stable fabric during an ISSU. See the following table for other ISSU impacts.

Table 1-3 ISSU Impact to Fibre Channel and FCoE Services

   

Name Server

When a new switch in the fabric is brought up and queries the Name Server on the ISSU switch, the ISSU switch cannot respond and does not receive Nx_port information.

Domain Manager

Domain Manager on a switch undergoing an ISSU does not process any BF/RCF/DIA/RDI caused by topology changes, which might result in traffic disruption in the fabric.

CFS

During an ISSU upgrade, CFS applications on other switches cannot obtain CFS locks on the ISSU switch, which might result in CFS distribution failures until the ISSU completes.

N-Port Virtualization

During an ISSU, the NPV process is down. Any FLOGI/fdisc or logo request from a server fails until the ISSU has completes.

Zone Server

During an ISSU, because EPP and merge requests are not processed, the peer switch cannot bring up E and TE ports connected to the ISSU switch until the ISSU completes.

A peer switch zone change request is not answered by the switch undergoing an ISSU. Any zone configuration changes on other switches connected to the ISSU switch fails until the ISSU completes.

FSPF

Before the switch reboots for an ISSU, the switch transmits a FSPF hello on all interfaces to prevent neighbor switches from marking routes to the ISSU switch as down. Any topology changes during this time are also not acted upon until the ISSU completes.

EPP

During an ISSU process, EPP messages are not received/transmitted on the ISSU switch. New ports in FC port channels are not negotiated until the ISSU completes. Additionally, FC Trunk Mode changes (E port to TE Port and vice versa and the allowed VSAN list) are also not processed.

FCoE NPV Links

When NPV/FCoE NPV switch is logged into a core switch through an FCoE NPV link, it will punch heartbeats (FIP keep alives - FKA), towards the core switch for its own internal login session and all the host login sessions pinned through this FCoE NPV link. This FKA interval of 8 seconds is lesser than the ISSU downtime. Set disable-fka on the core switch VFC parameters to ensure that the core switch ignores any FKA events.


Layer-2 Protocols Impact

Table 1-4 ISSU Impact to Layer 2 Protocols

   

LACP

IEEE 802.3ad provides for the default slow aging timers to be transmitted once every 30 seconds in steady state and to expire after 90 seconds. ISSU should not impact peers that rely on LACP because the recovery time is less than 90 seconds.

Note that a Fast LACP timers (hello=1 sec, dead=3 sec) are not supported with a nondisruptive ISSU.

IGMP

IGMP does not disrupt existing flows of multicast traffic that are already present, but new flows are not learned (and are dropped) until an ISSU completes. New router ports or changes to router ports are not detected during this time.

DCBX and LLDP

DCBX uses LLDP to exchange parameters between peer devices. Because DCBX is a link-local protocol, when the switch undergoes an ISSU, the age time is increased on all ports on the switches and FEXs that are being upgraded.

Manual configurations are ignored during this time.

CDP

During an ISSU, the time-to-live value is increased (180 seconds) if it is less than the recommended timeout value. The configuration is ignored if manually specified.

L2MP IS-IS

Before a switch reboots for an ISSU, the switch transmits L2 IS-IS hellos on all interfaces to prevent neighbor switches from marking routes to the ISSU switch as down. Any topology changes during this time are also not acted upon until the ISSU completes.


The following table lists the ISSU impacts to Layer 2 protocols.

Ethernet Interfaces on the Switch and the Fabric Extenders

To avoid link down to link up transitions during the control plane outage time, the laser is turned off for administratively up ports that are operationally down. This situation occurs during the ISSU reboot starting state when the switch and the FEX applications stop communicating with each other. After the ISSU reboot and a stateful restart, the laser is turned back on. This action prevents the link state from transitioning from down to up during an ISSU.

PreInstallation Checks

You should do certain sanity checks to ensure that the system is ready for an ISSU and to understand the impact of ISSU:

Enter the show incompatibility command to verify that the target image is feature-wise compatible with the current image.

Enter the show logging level command to ensure that the severity level for all processes is set to 5 or below.

Enter the show install all impact command to identify the upgrade impact.

Enter the show fex command to verify that all the FEXs are online.

Enter the show vpc role command to verify the vPC switch role in a vPC topology.

Enter the install all command to update to the latest Cisco NX-OS software.

Review the installer impact analysis and choose to continue.


Note The switch might reload at this time and cause a traffic disruption if the upgrade is not an ISSU.


Monitor the installation progress.

Verify the upgrade.

Enter the show install all status command to verify the status of the installation

The following table lists the show commands that identify the impact or potential problems that may occur when performing an ISSU.

Table 1-5 Upgrade show Commands

Command
Definition

show incompatibility system

Displays incompatible configurations on the current system that will impact the upgrade version.

show logging level

Displays the facility logging severity level configuration.

Logging levels for all processes must be set at 5 or below when performing an ISSU. Processes with a logging level greater than 5 are not displayed when you enter the show install all impact command.

show install all impact

Displays information that describes the impact of the upgrade on each Fabric Extender including the current and upgrade-image versions. This command also displays if the upgrade is disruptive or not and if the Fabric Extender needs to be rebooted and the reason why.

show spanning-tree issu-impact

Displays the spanning-tree configuration and whether or not there are potential STP issues.

show lacp issu-impact

Displays the port priority information and whether or not there are potential issues.

show fcoe-npv issu-impact

Checks whether disable-fka is set on any of the FCoE NPV (VNP) ports as a pre-ISSU check.


You can also perform the following tasks to identify potential problems before they occur:

Ensure that you have enough space to store the images on bootflash:

Display incompatible configurations on the current system that will impact the upgrade version.

n5010# show incompatibility system bootflash:n5000-uk9.4.2.1.N1.1a.bin
No incompatible configurations
 
   

Display the status of FEXs connected to the system.

nexus5010# show fex
   FEX                        FEX          FEX                   FEX               
Number  Description              State      Model              Serial     
--------------------------------------------------------------------------------------
----------
100       FEX0100                Online     N2K-C2224TP-1GE   JAF1427BQME
101       FEX0101                Online     N2K-C2224TP-1GE   JAF1427BQMK
 
   

Display the STP configuration and whether potential STP issues exist.

nexus5010# show spanning-tree issu-impact 
 
   
For ISSU to Proceed, Check the Following Criteria :
1. No Topology change must be active in any STP instance
2. Bridge assurance(BA) should not be active on any port (except MCT)
3. There should not be any Non Edge Designated Forwarding port (except MCT)
4. ISSU criteria must be met on the VPC Peer Switch as well
 
   
Following are the statistics on this switch
 
   
 
   
No Active Topology change Found!
Criteria 1 PASSED !!
 
   
No Ports with BA Enabled Found!
Criteria 2 PASSED!!
 
   
No Non-Edge Designated Forwarding Ports Found!
Criteria 3 PASSED !!
 
   
ISSU Can Proceed! Check Peer Switch.
 
   

Use the show lacp issu-impact command to display if any port or a peer switch is configured in rate fast mode.

Verify that ISSU is nondisruptive. By displaying the information about the impact of the upgrade on each FEX including details such as upgrade image versions. This command also displays if the upgrade is disruptive/nondisruptive and the reason why.

N5K1# show install all impact kickstart bootflash:n5000-uk9-kickstart.4.2.1.N1.1a.bin
system bootflash:n5000-uk9.4.2.1.N1.1a.bin
 
   
Verifying image bootflash:/n5000-uk9-kickstart.4.2.1.N1.1a.bin for boot variable
 "kickstart".
[####################] 100% -- SUCCESS
 
   
Verifying image bootflash:/n5000-uk9.4.2.1.N1.1a.bin for boot variable "system".
[####################] 100% -- SUCCESS
 
   
Verifying image type.
[###########         ]  50%
[####################] 100% -- SUCCESS
 
   
Extracting "system" version from image bootflash:/n5000-uk9.4.2.1.N1.1a.bin.
[####################] 100% -- SUCCESS
 
   
Extracting "kickstart" version from image bootflash:/n5000-uk9-kickstart.4.2.1.N
1.1a.bin.
[####################] 100% -- SUCCESS
 
   
Extracting "bios" version from image bootflash:/n5000-uk9.4.2.1.N1.1a.bin.
[####################] 100% -- SUCCESS
 
   
Extracting "fex" version from image bootflash:/n5000-uk9.4.2.1.N1.1a.bin.
[####################] 100% -- SUCCESS
 
   
Notifying services about system upgrade.
[####################] 100% -- SUCCESS
 
   
Compatibility check is done:
Module  bootable          Impact  Install-type  Reason
------  --------  --------------  ------------  ------
     1       yes  non-disruptive         reset  
   100       yes  non-disruptive       rolling  
 
   
Images will be upgraded according to following table:
Module       Image         Running-Version             New-Version  Upg-Required
------  ----------  ----------------------  ----------------------  ------------
     1      system             4.2(1)N1(1)            4.2(1)N1(1a)           yes
     1   kickstart             4.2(1)N1(1)            4.2(1)N1(1a)           yes
     1        bios        v1.2.0(06/19/08)        v1.3.0(09/08/09)           yes
   100         fex             4.2(1)N1(1)            4.2(1)N1(1a)           yes
     1   power-seq                    v1.2                    v1.2            no
 
   

Check whether disable-fka is set on any of the FCoE NPV (VNP) ports as a pre-ISSU check.

N5010# show fcoe-npv issu-impact
	show fcoe-npv issu-impact
	-------------------------
	
	Please make sure to enable "disable-fka" on all logged in VFCs
	Please increase the FKA duration to 60 seconds on FCF
	
	
	Active VNP ports with no disable-fka set
	----------------------------------------
	

Upgrading Procedures

The ISSU process is triggered when you enter the install all command. This section describes the sequence of events that occur when you upgrade a single Cisco Nexus 5000 Series switch or a single Cisco Nexus 5000 Series switch that is connected to one or more FEXs. The section includes the following topics:

Installation At-A-Glance, page 22

Copying the Running Configuration from an External Flash Memory Device, page 23

Copying the Startup Configuration from an External Flash Memory Device, page 24

Upgrade Process in a Non-vPC Topology, page 25

Upgrade Process for a vPC Topology on the Primary Switch, page 29

Upgrade Process for a vPC Topology on the Secondary Switch, page 29

Forcing an Upgrade, page 30

Upgrading From Cisco NX-OS Release 4.1(3)N2(1) and Earlier Releases (Disruptive Upgrade), page 31

Minimizing the Impact of a Disruptive Upgrade, page 36

Upgrading a Direct vPC or a Single-Homed FEX Access Layer, page 36

Upgrading a Dual-Homed FEX Access Layer, page 38

Installation At-A-Glance

The following table shows an overview of the upgrade process.

Table 1-6 Upgrade Process At-A-Glance

Upgrade Preparation

1. Log in to the first Cisco Nexus 5000 Series switch. We recommend that you log in to the console port. In vPC topologies, the first upgrade can be performed on either the primary or secondary switch in the topology.

2. Log in to Cisco.com to access the Software Download Center. To log in to Cisco.com, go to http://www.cisco.com/ and click Log In at the top of the page. Enter your Cisco username and password.

3. Choose and download the kickstart and system software files to the server.

4. Verify that the required space is available in the bootflash: directory for the image file(s) to be copied.

5. If you need more space in the bootflash: directory, delete unnecessary files to make space available.

6. Copy the Cisco NX-OS kickstart and system images to the bootflash using a transfer protocol such as ftp:, tftp:, scp:, or sftp.

7. Compare the file sizes of the images that were transferred using the dir bootflash command. The file sizes of the images obtained from Cisco.com and the image sizes of the transferred files should be the same.

8. Complete the above steps through Step 9 for each Cisco Nexus 5000 Series switch in the topology.

Pre-ISSU Checks

1. Enter the show incompatibility command to verify that the target image is feature-wise compatible with the current image.

2. Enter the show install all impact command to identify the upgrade impact.

3. Enter the show spanning-tree issu-impact command to display the impact of the upgrae impact.

4. Enter the show lacp issue-impact command to display the impact of the upgrade.

5. Enter the show fex command to verify that all the FEXs are online.

Upgrade Begins

1. Enter the show vpc role command to verify the vPC switch role.

2. Enter the install all command to update to the latest Cisco NX-OS software.

3. Peruse the installer impact analysis and accept to proceed.

4. Installer on Nexus 5000 upgrades the software - the switch will now run new version of the software.

Upgrade Verification

1. Enter the show install all status command to verify the status of the installation.


Copying the Running Configuration from an External Flash Memory Device

You can copy configuration files from an external flash memory device.


Note This procedure applies to the Cisco Nexus 5500 Platform running Cisco NX-OS Release 5.0.2 and later releases.


Before You Begin

Insert the external flash memory device into the active supervisor module.

 
Command or Action
Purpose

Step 1 

dir {usb1: | usb2:}[directory/]

Example:
switch# dir usb1:

(Optional) Displays the files on the external flash memory device.

Step 2 

copy {usb1: | usb2:}[directory/]filename running-config

Example:
switch# copy usb1:dsn-config.cfg running-config

Copies the running configuration from an external flash memory device. The filename argument is case sensitive.

Step 3 

copy {usb1: | usb2:}[directory/]filename running-config

Example:
switch# copy usb1:dsn-config.cfg running-config 
bootflash:n5000-uk9.4.2.1.N1.1.bin

(Optional) You can copy the running configuration from an external flash memory device to the bootflash.

Step 4 

show running-config

Example:
switch# show running-config

(Optional) Displays the running configuration.

Step 5 

copy running-config startup-config

Example:
switch# copy running-config startup-config

(Optional) Copies the running configuration to the startup configuration.

Step 6 

show startup-config

Example:
switch# show startup-config

(Optional) Displays the startup configuration.

Copying the Startup Configuration from an External Flash Memory Device

You can recover the startup configuration on your Cisco NX-OS device by downloading a new startup configuration file saved on an external flash memory device.


Note This procedure applies to the Cisco Nexus 5500 Platform running Cisco NX-OS Release 5.0.2 and later releases.


Before You Begin

Insert the external flash memory device into the active supervisor module.

 
Command or Action
Purpose

Step 1 

dir {usb1: | usb2:}[directory/]

Example:
switch# dir usb1:

(Optional) Displays the files on the external flash memory device.

Step 2 

copy {usb1: | usb2:}[directory/]filename startup-config

Example:
switch# copy usb1:dsn-config.cfg startup-config

Copies the startup configuration from an external flash memory device. The filename argument is case sensitive.

Step 3 

copy {usb1: | usb2:}[directory/]filename startup-config

Example:
switch# copy usb1:dsn-config.cfg startup-config 
bootflash:n5000-uk9.4.2.1.N1.1.bin

(Optional) You can copy the startup configuration from an external flash memory device to the bootflash.

Step 4 

show startup-config

Example:
switch# show startup-config

(Optional) Displays the startup configuration.

Step 1 

copy running-config startup-config

Example:
switch# copy running-config startup-config

(Optional) Copies the running configuration to the startup configuration.

Step 1 

show startup-config

Example:
switch# show startup-config

(Optional) Displays the startup configuration.

Upgrade Process in a Non-vPC Topology

The following list summarizes the upgrade process in a non-vPC topology:

1. The install all command triggers the installation upgrade.

2. The compatibility checks display the impact of the upgrade.

3. The installation proceeds or not based on the upgrade impact.

4. The current state is saved.

5. The system unloads and runs the new image.

6. The stateful restart of the system software and application occurs.

7. The installer resumes with the new image.

8. The FEXs are upgraded sequentially.

9. The installation completes.

The following table describes the process based on messages displayed on the console of the switch during an ISSU.

Table 7 The Switch Output During a Non-vPC ISSU 

The install all command is entered...
The image is verified ...
The image versions are extracted...

The image compatibility check begins...
The impact is identified...
The upgrade begins...
The boot variables are set...

The configuration is synchronized...
The BIOS and power-sequence upgrade occurs...


Caution If an upgrade fails after this point in the process, the upgrade will be disruptive and the switch will reboot.
Communication between the FEX and the switch is stopped...
The upgrade cleanup process begins ...
Upgrade images load...
Runtime is saved...

 

vPC peer switches are informed of ISSU process...
Processes are terminated...
In-transit messages and reset reasons are saved...
New kernel starts ...

The supervisor resets...

The installer restarts...
The supervisor status is verified...
The nondisruptive upgrade on each module begins...
The upgrade process is complete.


Upgrade Process for a vPC Topology on the Primary Switch

The following list summarizes the upgrade process on a primary switch in a vPC topology. Steps that differ from a switch upgrade in a non-vPC topology are in bold.


Note In vPC topologies, the two peer switches must be upgraded individually. An upgrade on one peer switch does not automatically update the vPC peer switch.


1. The install all command issued on the vPC primary switch triggers the installation upgrade.

2. The compatibility checks display the impact of the upgrade.

3. The installation proceeds or not based on the upgrade impact.

4. The configuration is locked on both vPC peer switches.

5. The current state is saved.

6. The system unloads and runs the new image.

7. The stateful restart of the system software and application occurs.

8. The installer resumes with the new image.

9. The FEXs are upgraded sequentially.

10. The installation is complete.

When the installation is complete, the vPC primary switch and the FEXs that are connected to the primary switch are upgraded. The single-homed FEXs and the dual-homed FEXs are now running the upgraded software.


Note The dual-homed FEXs are now connected to the primary and secondary switches that are running two different versions of the Cisco NX-OS software. The vPC primary switch is running the upgraded version and the vPC secondary switch is running the original software version. The Cisco NX-OS software has been designed to allow an upgraded dual-home FEX to interoperate with vPC secondary switches running the original version of Cisco NX-OS while the primary switch is running the upgrade version.


Upgrade Process for a vPC Topology on the Secondary Switch

The following list summarizes the upgrade process on a secondary switch in a vPC topology. Steps that differ from a switch upgrade in a non-vPC topology are in bold.

1. The install all command issued on the vPC second switch triggers the installation upgrade.

2. The compatibility checks display the impact of the upgrade.

3. The installation proceeds or not based on the upgrade impact.

4. The current state is saved.

5. The system unloads and runs the new image.

6. The stateful restart of the system software and application occurs.

7. The installer resumes with the new image.

8. The FEXs are upgraded sequentially. The upgrade completes on the single-homed FEXs and a sanity check is performed on the dual-homed FEXs.


Note The dual-homed FEXs were upgraded by the primary switch.


9. The configuration is unlocked on the primary and secondary switches.

10. The installation is complete.

Disruptive Installation Process

The following lists conditions where a nondisruptive ISSU might not be possible when upgrading a Cisco Nexus 5000 Series access layer switch:

The topology and/or features are not ISSU ready. See the section on ISSU prerequisites for more information.

The current release or target release is lower than 4.2(1)N1(1). An ISSU can work only when both the current and target releases are equal or later than 4.2(1)N1(1).

The installation is a downgrade, such as a higher release to a lower release, unless stated otherwise in Release specific Information section.

You want to do a disruptive upgrade. See the Forcing an Upgrade section.

Forcing an Upgrade

You can choose to do a disruptive upgrade if one of the ISSU conditions are not met. One additional reason where you might choose to do a disruptive upgrade is when FEXs are upgraded in a rolling fashion (one FEX at a time), which requires a longer maintenance window. With a disruptive upgrade, all the connected FEXs are upgraded simultaneously, so the maintenance window can be shorter. If you need a shorter maintenance window (with traffic disruption), you can force a disruptive upgrade even if an ISSU can be leveraged. It is important to note the possibility of an outage if you do a disruptive upgrade.

switch # install all force kickstart bootflash:/kickstart_image.bin system
bootflash:/system_image.bin
 
   
Installer is forced disruptive
 
   
Verifying image bootflash:/kickstart_image.bin for boot variable "kickstart".
[####################] 100% -- SUCCESS
 
   
Verifying image bootflash:/system_image.bin for boot variable "system".
...

You can also add force at the end of the install all command as follows:

switch # install all kickstart bootflash:/kickstart_image.bin system
bootflash:/system_image.bin force
 
   
Installer is forced disruptive
 
   
Verifying image bootflash:/kickstart_image.bin for boot variable "kickstart".
...
 
   

Upgrading From Cisco NX-OS Release 4.1(3)N2(1) and Earlier Releases (Disruptive Upgrade)

This section describes how to upgrade from Cisco NX-OS Release 4.1(3)N2(1) and earlier releases. An upgrade from these releases will be disruptive. Upgrading a Cisco Nexus 5000 Series switch also upgrades connected Fabric Extenders.


Note To perform a nondisruptive upgrade from Cisco NX-OS Release 4.2(1)N1(1) and later releases, see the "In-Service Software Upgrades" section on page 6.


DETAILED STEPS


Step 1 Log in to Cisco.com to access the Software Download Center. To log in to Cisco.com, go to http://www.cisco.com/ and click Log In at the top of the page. Enter your Cisco username and password.


Note Unregistered Cisco.com users cannot access the links provided in this document.


Access the Software Download Center at http://www.cisco.com/public/sw-center/index.shtml. Navigate to the software downloads for Cisco Nexus 5000 Series switches. Links to the download images for the switch are listed.

Step 2 Choose and download the kickstart and system software files to a local server.

Step 3 Verify that the required space is available in the bootflash: directory for the image file(s) to be copied.

switch# dir bootflash:
       4681     May 24 02:43:52 2010  config
   13176836     May 24 07:19:36 2010  gdb.1
      49152     Jan 12 18:38:36 2009  lost+found/
     310556     Dec 23 02:53:28 2008  n1
   20058112     Jun 25 23:17:44 2010 n5000-uk9-kickstart.4.1.3.N1.1.bin
   20217856     May 09 23:17:11 2010 n5000-uk9-kickstart.4.0.1a.N1.0.62.bin
   76930262     Jun 25 23:11:47 2010 n5000-uk9.4.1.3.N1.1.bin
   103484727    May 09 23:10:02 2010 n5000-uk9.4.0.1a.N1.0.62.bin
 
   
Usage for bootflash://sup-local
   74934272 bytes used
    5550080 bytes free
   80484352 bytes total
 
   

We recommend that you keep the kickstart and system image files for at least one previous software release to use if the new image files do not load successfully.

Step 4 (Optional) If you need more space on the bootflash, delete unnecessary files to make space available.

switch# delete bootflash:n5000-uk9-kickstart.4.0.1a.N1.0.62.bin
switch# delete bootflash:n5000-uk9.4.0.1a.N1.0.62.bin
 
   

Step 5 Copy the new kickstart and system images to the switch bootflash by using a transfer protocol such as FTP, TFTP, SCP, or SFTP. The examples in this procedure use SCP.

switch# copy scp://user@scpserver.cisco.com/downloads/n5000-uk9.4.2.1.N1.1.bin 
bootflash:n5000-uk9.4.2.1.N1.1.bin
switch# copy scp://user@scpserver.cisco.com/downloads/n5000-uk9-kickstart.4.2.1.N1.1.bin
bootflash:n5000-uk9-kickstart.4.2.1.N1.1.bin
 
   

Step 6 Display the impact of the upgrade.

switch# show install all impact kickstart bootflash:n5000-uk9-kickstart.4.2.1.N1.1.bin
system bootflash:n5000-uk9.4.2.1.N1.1.bin
Verifying image bootflash:/n5000-uk9-kickstart.4.2.1.N1.1.bin for boot variable 
"kickstart".
[####################] 100% -- SUCCESS
 
   
Verifying image bootflash:/n5000-uk9.4.2.1.N1.1.bin for boot variable "system".
[####################] 100% -- SUCCESS
 
   
Verifying image type.
[###########         ]  50%
[####################] 100% -- SUCCESS
 
   
Extracting "system" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin.
[####################] 100% -- SUCCESS
 
   
Extracting "kickstart" version from image bootflash:/n5000-uk9-kickstart.4.2.1.N
1.1.bin.
[####################] 100% -- SUCCESS
 
   
Extracting "bios" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin.
[####################] 100% -- SUCCESS
 
   
Extracting "fex" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin.
[####################] 100% -- SUCCESS
 
   
Notifying services about system upgrade.
[####################] 100% -- SUCCESS
 
   
Compatibility check is done:
 
   
Module  bootable          Impact  Install-type  Reason
------  --------  --------------  ------------  ------
     1       yes      disruptive         reset  Reset due to single supervisor
   100       yes      disruptive         reset  Reset due to single supervisor
 
   
Images will be upgraded according to following table:
Module       Image         Running-Version             New-Version  Upg-Required
------  ----------  ----------------------  ----------------------  ------------
     1      system             4.1(3)N1(1)             4.2(1)N1(1)           yes
     1   kickstart             4.1(3)N1(1)             4.2(1)N1(1)           yes
     1        bios        v1.3.0(09/08/09)                                    no
   100         fex             4.1(3)N1(1)             4.2(1)N1(1)           yes
 
   

Step 7 Install the new images, specifying the new image names that you downloaded in the previous step.

switch#  install all kickstart bootflash:n5000-uk9-kickstart.4.2.1.N1.1.bin system
bootflash:n5000-uk9.4.2.1.N1.1.bin
Verifying image bootflash:/n5000-uk9-kickstart.4.2.1.N1.1.bin for boot variable
"kickstart".
[####################] 100% -- SUCCESS
 
   
Verifying image bootflash:/n5000-uk9.4.2.1.N1.1.bin for boot variable "system".
[####################] 100% -- SUCCESS
 
   
Verifying image type.
[####################] 100% -- SUCCESS
 
   
Extracting "system" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin.
[####################] 100% -- SUCCESS
 
   
Extracting "kickstart" version from image bootflash:/n5000-uk9-kickstart.4.2.1.N1.1.bin.
[####################] 100% -- SUCCESS
 
   
Extracting "bios" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin.
[####################] 100% -- SUCCESS
 
   
Extracting "fex" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin.
[####################] 100% -- SUCCESS
 
   
Notifying services about system upgrade.
[####################] 100% -- SUCCESS
Compatibility check is done:
Module  bootable          Impact  Install-type  Reason
------  --------  --------------  ------------  ------
     1       yes      disruptive         reset  Reset due to single supervisor
   100       yes      disruptive         reset  Reset due to single supervisor
 
   
Images will be upgraded according to following table:
Module       Image         Running-Version             New-Version  Upg-Required
------  ----------  ----------------------  ----------------------  ------------
     1      system             4.1(3)N1(1)             4.2(1)N1(1)           yes
     1   kickstart             4.1(3)N1(1)             4.2(1)N1(1)           yes
     1        bios        v1.3.0(09/08/09)                                    no
   100         fex             4.1(3)N1(1)             4.2(1)N1(1)           yes
 
   
Switch will be reloaded for disruptive upgrade.
Do you want to continue with the installation (y/n)?  [n] y
 
   
Install is in progress, please wait.
 
   
Setting boot variables.
[####################] 100% -- SUCCESS
 
   
Performing configuration copy.
[####################] 100% -- SUCCESS
 
   
Module 1: Refreshing compact flash and upgrading bios/loader/bootrom.
Warning: please do not remove or power off the module at this time.
[####################] 100% -- SUCCESS
 
   
Pre-loading modules.
[This step might take up to 20 minutes to complete - please wait.]
[#                   ]   0%2010 Jun 10 18:27:25 N5K1 %$ VDC-1 %$ %SATCTRL-2-SATCTRL_IMAGE:
FEX100 Image update in progress. 
[#####               ]  20%
 
   
[######              ]  25%2010 Jun 10 18:32:54 N5K1 %$ VDC-1 %$ %SATCTRL-2-SATCTRL_IMAGE:
FEX100 Image update complete. Install pending 
[####################] 100% -- SUCCESS
 
   
Finishing the upgrade, switch will reboot in 10 seconds.
switch# 
switch# 
switch#  writing reset reason 31, 
 
   
Broadcast message from root (Thu Jun 10 18:33:16 2010):
 
   
INIT: Sending processes the TERM signal
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "stp" (PID 2843) is forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "netstack" (PID 2782) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "Security Daemon" (PID 2706) is
forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "Cert_enroll Daemon" (PID 2707) is
forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "igmp" (PID 2808) is forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "Radius Daemon" (PID 2806) is
forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "AAA Daemon" (PID 2708) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "vshd" (PID 2636) is forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "vlan_mgr" (PID 2737) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "vdc_mgr" (PID 2681) is forced 
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "urib" (PID 2718) is forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "u6rib" (PID 2717) is forced exit.
 
   
Jun 10 18:33:17 %TTYD-2-TTYD_ERROR TTYD Error ttyd bad select
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "statsclient" (PID 2684) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "smm" (PID 2637) is forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "res_mgr" (PID 2688) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "licmgr" (PID 2641) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "l3vm" (PID 2715) is forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "icmpv6" (PID 2781) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "eth_dstats" (PID 2700) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "arp" (PID 2780) is forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "fs-daemon" (PID 2642) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "ascii-cfg" (PID 2704) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "adjmgr" (PID 2771) is forced
exit.
 
   
Jun 10 18:33:19 Unexporting directories for NFS kernel daemon...done.
Stopping NFS kernel daemon: rpc.mountd rpc.nfsddone.
Unexporting directories for NFS kernel daemon...
done.
Stopping portmap daemon: portmap. 
Stopping kernel log daemon: klogd.
Sending all processes the TERM signal... done.
Sending all processes the KILL signal... done.
Unmounting remote filesystems... done. 
Deactivating swap...umount: none busy - remounted read-only
done.
Unmounting local filesystems...umount: none busy - remounted read-only
done.
mount: you must specify the filesystem type
Starting reboot command: reboot
Rebooting...
Restarting system.
 
   
The switch reboots and restarts with the new image.
For detailed information about the install all command, see Using the install all Command.
Caution   
After completing the installation, all traffic through the switch is disrupted while the
switch reboots.
 
   

Step 8 Verify that the switch is running the required software release.

switch# show version
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright ) 2002-2010, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained herein are owned by
other third parties and are used and distributed under license.
Some parts of this software are covered under the GNU Public
License. A copy of the license is available at
http://www.gnu.org/licenses/gpl.html.
 
Software
  BIOS:      version 1.2.0
  loader:    version N/A
  kickstart: version 4.2(1)N1(1) [build 4.2(1)N1(0.96)]
  system:    version 4.2(1)N1(1) [build 4.2(1)N1(0.96)]
  BIOS compile time:       06/19/09
  kickstart image file is: bootflash:/n5000-uk9-kickstart.4.2.1.N1.0.96.bin
  kickstart compile time:  7/14/2010 4:00:00 [07/14/2010 04:27:38]
  system image file is:    bootflash:/n5000-uk9.4.2.1.N1.0.96.bin
  system compile time:     7/14/2010 4:00:00 [07/14/2010 05:20:12]
 
Hardware
  cisco Nexus5020 Chassis ("40x10GE/Supervisor")
  Intel) Celeron) M CPU    with 2074240 kB of memory.
  Processor Board ID JAB1232002F
 
  Device name: switch
  bootflash:    1003520 kB
 
Kernel uptime is 13 day(s), 23 hour(s), 25 minute(s), 5 second(s)
 
Last reset at 720833 usecs after  Tue Jul 14 11:18:32 2010
 
  Reason: Reset by installer
  System version: 4.2(1)N1(0.96)
  Service: 
 
plugin
  Core Plugin, Ethernet Plugin

Minimizing the Impact of a Disruptive Upgrade

A non-ISSU upgrade is a disruptive upgrade that results in the reload of the Cisco Nexus 5000 Series switch and the Cisco Nexus 2000 Series Fabric Extenders. The reload is a cold reboot that brings down the control plan and the data plane. The reload causes disruptions to the connected servers and hosts. When a vPC is deployed in the access layer, it is possible to minimize the impact of a non-ISSU upgrade. When one of the vPC switches is being reset during the upgrade process, all the server traffic can flow through its vPC peer.

Upgrading a Direct vPC or a Single-Homed FEX Access Layer

The following figures show topologies in which the access layer includes a vPC configuration to hosts or downstream switches.

Figure 16 Hosts Directly Connected Directly to vPC Peers

Figure 17 vPC Peered Dual-Supervisor Virtual Modular System Dual-Homed FEXs and Singled-Homed FEXs

Figure 18 Cisco Nexus 5000 Series Switches Connected To Downstream Switches

To upgrade the access layer without a disruption to hosts, follow these tasks:

Upgrade the first vPC switch (vPC primary switch). During this upgrade, the switch is reloaded. When the switch is reloaded, the servers or the downstream switch detects a loss of connectivity to the first switch and starts forwarding traffic to the second (vPC secondary) switch.

Verify that the upgrade of the switch has completed successfully. At the completion of the upgrade, the switch restores vPC peering, connected Nexus 2000 Fabric Extenders and all the links.

Upgrade the second switch. Repeating the same process on the second switch causes the second switch to reload during the upgrade process. During this reload, the first (upgraded) switch forwards all the traffic to/from servers.

Verify that the upgrade of the second switch has completed successfully.


Note Flows that are forwarded to a switch during an upgrade on the switch, will failover to the second switch. Also, flows are redistributed when vPC peers are active. The traffic disruption is limited to the time required for the server or host to detect the link-down and link-up events and to redistribute the flows.


Upgrading a Dual-Homed FEX Access Layer

A disruptive upgrade causes the switch and any connected FEX to reload. The time required for a FEX to reload is shorter than the time required for a switch to reload. When hosts are connected to a dual-homed FEX, it is possible to keep the traffic disruption to the hosts to the time required for the FEX to reload (approximately 120 seconds), instead of the time required for the upgrade of the entire access layer. These steps bypass the Cisco NX-OS installer while upgrading the second switch and should be used with caution. Additionally, BIOS and power-sequencer components of the switch are not upgraded, because of the bypass of the Cisco NX-OS installer.

The following are the steps:

Upgrade the first vPC switch (vPC primary switch). During this upgrade, the switch is reloaded. When the switch is reloaded, any dual-homed FEX is not be reloaded. Only singled-homed FEXs are reloaded. Servers connected to these dual-homed FEXs retain network connectivity through the second (vPC secondary) switch.

Verify that the upgrade of the first switch has completed successfully. At the completion of the upgrade, the switch restores vPC peering. However, dual-homed FEXs stay singly connected to the second switch.

Set the bootvariable on the second switch and save the configuration (preferably from the switch console).

1. Update the boot variable on the switch to point to reflect the new target image

2. Save configuration by entering the copy running-config startup-config command

Reload the dual-homed FEXs using reload fex command. Servers connected to these dual-homed FEXs lose connectivity at this point.

Wait for the FEXs to reload. They connect to the first (already upgraded) switch after the reload. The Downtime to servers connected to these FEXs is limited to this reload and reconnect time.

Reload the switch by entering the reload command.

When the second vPC switch is back up with the new software, vPC peering is reestablished and dual-homed FEXs connect to both peer switches and start forwarding traffic to both of them.

Detailed Steps


Step 1 Log in to Cisco.com to access the Software Download Center. To log in to Cisco.com, go to http://www.cisco.com/ and click Log In at the top of the page. Enter your Cisco username and password.


Note Unregistered Cisco.com users cannot access the links provided in this document.


Access the Software Download Center at http://www.cisco.com/public/sw-center/index.shtml. Navigate to the software downloads for Cisco Nexus 5000 Series switches. Links to the download images for the switch are listed.

Step 2 Choose and download the kickstart and system software files to a local server.

Step 3 Verify that the required space is available in the bootflash: directory for the image file(s) to be copied.

switch# dir bootflash:
       4681     May 24 02:43:52 2010  config
   13176836     May 24 07:19:36 2010  gdb.1
      49152     Jan 12 18:38:36 2009  lost+found/
     310556     Dec 23 02:53:28 2008  n1
   20058112     Jun 25 23:17:44 2010 n5000-uk9-kickstart.4.1.3.N1.1.bin
   20217856     May 09 23:17:11 2010 n5000-uk9-kickstart.4.0.1a.N1.0.62.bin
   76930262     Jun 25 23:11:47 2010 n5000-uk9.4.1.3.N1.1.bin
   103484727    May 09 23:10:02 2010 n5000-uk9.4.0.1a.N1.0.62.bin
 
   
Usage for bootflash://sup-local
   74934272 bytes used
    5550080 bytes free
   80484352 bytes total
 
   

We recommend that you keep the kickstart and system image files for at least one previous software release to use if the new image files do not load successfully.

Step 4 (Optional) If you need more space on the bootflash, delete unnecessary files to make space available.

switch# delete bootflash:n5000-uk9-kickstart.4.0.1a.N1.0.62.bin
switch# delete bootflash:n5000-uk9.4.0.1a.N1.0.62.bin
 
   

Step 5 Copy the new kickstart and system images to each switch bootflash by using a transfer protocol such as FTP, TFTP, SCP, or SFTP. The examples in this procedure use SCP.

switch-1# dir bootflash:
 21778944     May 25 23:17:44 2010 n5000-uk9-kickstart.4.2.1.N1.1.bin
 22557184     Jun 09 23:17:11 2010 n5000-uk9-kickstart.4.1.3.N1.1.bin
181095489     May 25 23:11:47 2010 n5000-uk9.4.2.1.N1.1.bin
181204582     Jun 09 23:10:02 2010 n5000-uk9.4.1.3.N1.1.bin
 
   

Step 6 Enter the show install all impact command.

switch-1# show install all impact kickstart bootflash:n5000-uk9-kickstart.4.2.1.N1.1.bin
system bootflash:n5000-uk9.4.2.1.N1.1.bin
 
   
Verifying image bootflash:/n5000-uk9-kickstart.4.2.1.N1.1.bin for boot variable 
"kickstart".
[####################] 100% -- SUCCESS
 
   
Verifying image bootflash:/n5000-uk9.4.2.1.N1.1.bin for boot variable "system".
[####################] 100% -- SUCCESS
 
   
Verifying image type.
[###########         ]  50%
[####################] 100% -- SUCCESS
 
   
Extracting "system" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin.
[####################] 100% -- SUCCESS
 
   
Extracting "kickstart" version from image bootflash:/n5000-uk9-kickstart.4.2.1.N
1.1.bin.
[####################] 100% -- SUCCESS
 
   
Extracting "bios" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin.
[####################] 100% -- SUCCESS
 
   
Extracting "fex" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin.
[####################] 100% -- SUCCESS
 
   
Notifying services about system upgrade.
[####################] 100% -- SUCCESS
 
   
 
   
 
   
Compatibility check is done:
Module  bootable          Impact  Install-type  Reason
------  --------  --------------  ------------  ------
     1       yes      disruptive         reset  Reset due to single supervisor
   100       yes      disruptive         reset  Reset due to single supervisor
 
   
 
   
 
   
Images will be upgraded according to following table:
Module       Image         Running-Version             New-Version  Upg-Required
------  ----------  ----------------------  ----------------------  ------------
     1      system             4.1(3)N1(1)             4.2(1)N1(1)           yes
     1   kickstart             4.1(3)N1(1)             4.2(1)N1(1)           yes
     1        bios        v1.3.0(09/08/09)                                    no
   100         fex             4.1(3)N1(1)             4.2(1)N1(1)           yes
 
   

Step 7 Enter the install all kickstart image system image command.

Substeps 5.a., 5.b., and 5.c. describe how to verify the status of the upgrade while it is occurring.

switch-1# install all kickstart bootflash:n5000-uk9-kickstart.4.2.1.N1.1.bin system
bootflash:n5000-uk9.4.2.1.N1.1.bin
 
   
Verifying image bootflash:/n5000-uk9-kickstart.4.2.1.N1.1.bin for boot variable
"kickstart".
[####################] 100% -- SUCCESS
 
   
Verifying image bootflash:/n5000-uk9.4.2.1.N1.1.bin for boot variable "system".
[####################] 100% -- SUCCESS
 
   
Verifying image type.
[####################] 100% -- SUCCESS
 
   
Extracting "system" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin.
[####################] 100% -- SUCCESS
 
   
Extracting "kickstart" version from image bootflash:/n5000-uk9-kickstart.4.2.1.N1.1.bin.
[####################] 100% -- SUCCESS
 
   
Extracting "bios" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin.
[####################] 100% -- SUCCESS
 
   
Extracting "fex" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin.
[####################] 100% -- SUCCESS
 
   
Notifying services about system upgrade.
[####################] 100% -- SUCCESS
 
   
 
   
 
   
Compatibility check is done:
Module  bootable          Impact  Install-type  Reason
------  --------  --------------  ------------  ------
     1       yes      disruptive         reset  Reset due to single supervisor
   100       yes      disruptive         reset  Reset due to single supervisor
 
   
 
   
 
   
Images will be upgraded according to following table:
Module       Image         Running-Version             New-Version  Upg-Required
------  ----------  ----------------------  ----------------------  ------------
     1      system             4.1(3)N1(1)             4.2(1)N1(1)           yes
     1   kickstart             4.1(3)N1(1)             4.2(1)N1(1)           yes
     1        bios        v1.3.0(09/08/09)                                    no
   100         fex             4.1(3)N1(1)             4.2(1)N1(1)           yes
 
   
 
   
Switch will be reloaded for disruptive upgrade.
Do you want to continue with the installation (y/n)?  [n] y
 
   
Install is in progress, please wait.
 
   
Setting boot variables.
[####################] 100% -- SUCCESS
 
   
Performing configuration copy.
[####################] 100% -- SUCCESS
 
   
Module 1: Refreshing compact flash and upgrading bios/loader/bootrom.
Warning: please do not remove or power off the module at this time.
[####################] 100% -- SUCCESS
 
   
Pre-loading modules.
[This step might take upto 20 minutes to complete - please wait.]
[#                   ]   0%2010 Jun 10 18:27:25 N5K1 %$ VDC-1 %$ %SATCTRL-2-SATCTRL_IMAGE:
FEX100 Image update in progress. 
[#####               ]  20%
 
   
[######              ]  25%2010 Jun 10 18:32:54 N5K1 %$ VDC-1 %$ %SATCTRL-2-SATCTRL_IMAGE:
FEX100 Image update complete. Install pending 
[####################] 100% -- SUCCESS
 
   
Finishing the upgrade, switch will reboot in 10 seconds.
switch-1# 
switch-1# 
switch-1#  writing reset reason 31, 
 
   
Broadcast message from root (Thu Jun 10 18:33:16 2010):
 
   
INIT: Sending processes the TERM signal
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "stp" (PID 2843) is forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "netstack" (PID 2782) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "Security Daemon" (PID 2706) is
forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "Cert_enroll Daemon" (PID 2707) is
forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "igmp" (PID 2808) is forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "Radius Daemon" (PID 2806) is
forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "AAA Daemon" (PID 2708) is forced
exit.
 
   
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "vshd" (PID 2636) is forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "vlan_mgr" (PID 2737) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "vdc_mgr" (PID 2681) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "urib" (PID 2718) is forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "u6rib" (PID 2717) is forced exit.
 
   
Jun 10 18:33:17 %TTYD-2-TTYD_ERROR TTYD Error ttyd bad select
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "statsclient" (PID 2684) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "smm" (PID 2637) is forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "res_mgr" (PID 2688) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "licmgr" (PID 2641) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "l3vm" (PID 2715) is forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "icmpv6" (PID 2781) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "eth_dstats" (PID 2700) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "arp" (PID 2780) is forced exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "fs-daemon" (PID 2642) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "ascii-cfg" (PID 2704) is forced
exit.
 
   
Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "adjmgr" (PID 2771) is forced
exit.
 
   
Jun 10 18:33:19 Unexporting directories for NFS kernel daemon...done.
Stopping NFS kernel daemon: rpc.mountd rpc.nfsddone.
Unexporting directories for NFS kernel daemon...
done.
Stopping portmap daemon: portmap. 
Stopping kernel log daemon: klogd.
Sending all processes the TERM signal... done.
Sending all processes the KILL signal... done.
Unmounting remote filesystems... done. 
Deactivating swap...umount: none busy - remounted read-only
done.
Unmounting local filesystems...umount: none busy - remounted read-only
done.
mount: you must specify the filesystem type
Starting reboot command: reboot
Rebooting...
Restarting system.
The primary switch restarts with the new image.
From the primary switch, verify the status of the fabric extender. 
switch-1# show fex 100
 FEX: 100 Description: FEX0100   state: AA Version Mismatch
 FEX version: 4.1(3)N1(1) [Switch version: 4.2(1)N1(1)]
 pinning-mode: static    Max-links: 1
  Fabric port for control traffic: Eth1/37
  Fabric interface state:
    Po5 - Interface Up. State: Active
    Eth1/37 - Interface Up. State: Active
 
   

During the software upgrade on the primary switch, you can view the FEX upgrade progress using the secondary vPC switch (see the bold output):

switch-2#   2010 Jun 10 18:27:25 N5K2 %$ VDC-1 %$ %SATCTRL-2-SATCTRL_IMAGE: FEX100 Image
update in progress. 
 
   
switch-2# 2010 Jun 10 18:32:54 N5K2 %$ VDC-1 %$ %SATCTRL-2-SATCTRL_IMAGE: FEX100 Image
update complete. Install pending 
 
   

Verify the status of the fabric extender from the secondary switch.

switch-2# show fex 100
  FEX: 100 Description: FEX0100   state: Online
  FEX version: 4.1(3)N1(1) [Switch version: 4.1(3)N1(1)]
  Extender Model: N2K-C2148T-1GE,  Extender Serial: JAF1343BHCK
  Part No: 73-12009-06
   pinning-mode: static    Max-links: 1
   Fabric port for control traffic: Eth1/37
  Fabric interface state:
    Po5 - Interface Up. State: Active
    Eth1/37 - Interface Up. State: Active
 
   

Note The Fabric Extender remains online while the primary switch is reloaded.


Step 8 Update the boot variable on the secondary switch to reflect the new image.

switch-2(config)# boot system bootflash:n5000-uk9.4.2.1.N1.1.bin
switch-2(config)# boot kickstart bootflash:n5000-uk9-kickstart.4.2.1.N1.1.bin
Issue the copy running startup-configuration command. 
switch-2(config)# copy running startup-configuration 
[########################################] 100%
 
   

Step 9 From the secondary switch, reload the first Fabric Extender and then the subsequent Fabric Extenders.

switch-2# reload fex 100
WARNING: This command will reboot FEX 100
Do you want to continue? (y/n) [n] y
switch-2# 2010 Jun 10 18:49:54 N5K2 %$ VDC-1 %$ %PFMA-2-FEX_STATUS: Fex 100 is offline
2010 Jun 10 18:49:54 N5K2 %$ VDC-1 %$ %NOHMS-2-NOHMS_ENV_FEX_OFFLINE: FEX-100 Off-line
(Serial Number JAF1343BHCK)
 
   

Note Only the primary switch shows that the Fabric Extender is online because the secondary switch does not have the new image. The secondary switch shows the Fabric Extender is downloading an image.


switch-2# show fex 100
FEX: 100 Description: FEX0100   state: Image Download
  FEX version: 4.2(1)N1(1) [Switch version: 4.1(3)N1(1)]
  Extender Model: N2K-C2148T-1GE,  Extender Serial: JAF1343BHCK
  Part No: 73-12009-06
 pinning-mode: static    Max-links: 1
  Fabric port for control traffic: Eth1/37
  Fabric interface state:
    Po5 - Interface Up. State: Active
    Eth1/37 - Interface Up. State: Active
switch-1# show fex 100
FEX: 100 Description: FEX0100   state: Online
  FEX version: 4.2(1)N1(1) [Switch version: 4.2(1)N1(1)]
  Extender Model: N2K-C2148T-1GE,  Extender Serial: JAF1343BHCK
  Part No: 73-12009-06
 pinning-mode: static    Max-links: 1
  Fabric port for control traffic: Eth1/37
  Fabric interface state:
    Po5 - Interface Up. State: Active
    Eth1/37 - Interface Up. State: Active
 
   

Note Make sure that the first Fabric Extender comes up before reloading the subsequent Fabric Extenders.


When all fabric extenders are loaded, go to the next step.

Step 10 On the secondary switch, enter the reload command.


Note Do not save the configuration to ensure that the Fabric Extender are not lost because the secondary switch does not see the Fabric Extenders online.


switch-2# reload 
WARNING: There is unsaved configuration!!!
WARNING: This command will reboot the system
Do you want to continue? (y/n) [n] y
2010 Jun 10 18:55:52 N5K2 %$ VDC-1 %$ %PFMA-2-PFM_SYSTEM_RESET: Manual system restart from
Command Line Interface writing reset reason 9, 
 
   
INIT: Sending processes the TERM signal
Jun 10 18:55:57 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "statsclient" (PID 2684) is forced
exit.
 
   
Jun 10 18:55:57 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "stp" (PID 2842) is forced exit.
 
   
Jun 10 18:55:57 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "netstack" (PID 2777) is forced
exit.
 
   
Jun 10 18:55:57 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "Security Daemon" (PID 2706) is
forced exit.
 
   
Jun 10 1Jun 10 18:55:57 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "ascii-cfg" (PID 2704) is
forced exit.
 
   
Jun 10 18:55:57 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "adjmgr" (PID 2767) is forced
exit.
 
   
Unexporting directories for NFS kernel daemon...done.
Stopping NFS kernel daemon: rpc.mountd rpc.nfsddone.
Unexporting directories for NFS kernel daemon...
done.
Stopping portmap daemon: portmap. 
Stopping kernel log daemon: klogd.
Sending all processes the TERM signal... done.
Sending all processes the KILL signal... done.
Unmounting remote filesystems... done. 
Deactivating swap...umount: none busy - remounted read-only
done.
Unmounting local filesystems...umount: none busy - remounted read-only
done.
mount: you must specify the filesystem type
Starting reboot command: reboot
Rebooting...
Restarting system.
 
   

Note You should do certain sanity checks to ensure that the system is ready for an ISSU and to understand the impact of an ISSU.



Monitoring the Upgrade Status

Table 1-8 lists the show commands that are used to monitor installation upgrades.

Table 1-8 Monitoring the Upgrade Process 

Command
Definition

show fex

Displays the Fabric Extender status during an ISSU.

show install all failure-reason

Displays the applications that failed during an installation and why the installation failed.

show install all status

Displays a high-level log of the installation.

show system internal log install details

Displays detailed logs of the last installation-related command.

show system internal log install history

Displays detailed logs of the last five installation-related commands, from the oldest to the newest logs.

show tech-support

Displays the system and configuration information that you can provide to the Cisco Technical Assistance Center when reporting a problem.


The following example shows the output from the show install all status command:

There is an on-going installation...
Enter Ctrl-C to go back to the prompt.
 
   
Continuing with installation process, please wait.
The login will be disabled until the installation is completed.
 
   
Performing supervisor state verification. 
SUCCESS
 
   
Supervisor non-disruptive upgrade successful.
 
   
Pre-loading modules.
SUCCESS
 
   
Module 198: Non-disruptive upgrading.
SUCCESS
 
   
Module 199: Non-disruptive upgrading.
SUCCESS
 
   
Install has been successful.  (hit Ctrl-C here)
 
   

The following example shows the output from the show fex command on two vPC peer switches where FEX 198 and FEX 199 are upgraded:

switch-1# show fex
  FEX         FEX           FEX                       FEX               
Number    Description      State            Model            Serial     
------------------------------------------------------------------------
198        FEX0198      Hitless Upg Idle     N2K-C2248TP-1GE   JAF1342ANQP
199        FEX0199                Online     N2K-C2248TP-1GE   JAF1342ANRL
 
   
switch-2# show fex
  FEX         FEX           FEX                       FEX               
Number    Description      State            Model            Serial     
------------------------------------------------------------------------
198        FEX0198       FEX AA Upg Idle     N2K-C2248TP-1GE   JAF1342ANQP
199        FEX0199                Online     N2K-C2248TP-1GE   JAF1342ANRL
 
   

Downgrading from a Higher Release

The procedure to downgrade the switch is identical to a switch upgrade, except that the image files to be loaded are for an earlier release than the image that is currently running on the switch. You can use the show incompatibility system command to ensure that there are no feature incompatibilities between the current release and the target release. Note that downgrades disruptive. Please refer to the Release-specific information section to get information about certain downgrades that are nondisruptive.


Note Before you downgrade to a specific release, check the release notes for the current release installed on the switch, to ensure that your hardware is compatible with the specific release. You must be aware of special caveats before you downgrade the switch software to a Cisco NX-OS 4.0(0)-based release. See the Cisco Nexus 5000 Series Switch and Cisco Nexus 2000 Series Release Notes for details.


Troubleshooting ISSUs and Disruptive Installations

Some common causes for ISSU failure are as follows:

ISSU requirements are not met - bridge assurance is active or the switch is not a leaf node in the STP topology. These problems are described in the commands explained in Pre-ISSU Checks section.

bootflash: does not have enough space to accept the updated image.

Specified system and kickstart are not compatible.

Hardware is installed or removed while the upgrade is in process.

Any power disruption while an upgrade is in progress.

The entire path for the remote server location is not specified accurately.

Related Documentation

Documentation for Cisco Nexus 3000 Series Switches and Cisco Nexus 2000 Series Fabric Extenders is available at the following URL:

http://www.cisco.com/en/US/products/ps9670/tsd_products_support_series_home.html

The following are related Cisco Nexus 3000 Series and Cisco Nexus 2000 Series Fabric Extender documents:

Release Notes

Cisco Nexus 5000 Series and Cisco Nexus 2000 Series Release Notes

Cisco Nexus 5000 Series Switch Release Notes

Configuration Guides

Cisco Nexus 5000 Series Configuration Limits for Cisco NX-OS Release 5.1(3)N1(1)

Cisco Nexus 5000 Series Configuration Limits for Cisco NX-OS Release 5.0(2)N1(1b)

Cisco Nexus 5000 Series Configuration Limits for Cisco NX-OS Release 4.2(1)N1(1) and Release 4.2(1)N2(1)

Cisco Nexus 5000 Series NX-OS Fibre Channel over Ethernet Configuration Guide

Cisco Nexus 5000 Series NX-OS Layer 2 Switching Configuration Guide

Cisco Nexus 5000 Series NX-OS Multicast Routing Configuration Guide

Cisco Nexus 5000 Series NX-OS Quality of Service Configuration Guide

Cisco Nexus 5000 Series NX-OS SAN Switching Configuration Guide

Cisco Nexus 5000 Series NX-OS Security Configuration Guide

Cisco Nexus 5000 Series NX-OS System Management Configuration Guide

Cisco Nexus 5000 Series NX-OS Unicast Routing Configuration Guide

Cisco Nexus 5000 Series Switch NX-OS Software Configuration Guide

Cisco Nexus 5000 Series Fabric Manager Configuration Guide, Release 3.4(1a)

Cisco Nexus 7000 Series NX-OS Fundamentals Configuration Guide, Release 6.x

Cisco Nexus 2000 Series Fabric Extender Software Configuration Guide

Maintain and Operate Guides

Cisco Nexus 5000 Series NX-OS Operations Guide

Installation and Upgrade Guides

Cisco Nexus 5000 Series and Cisco Nexus 5500 Platform Hardware Installation Guide

Cisco Nexus 2000 Series Hardware Installation Guide

Cisco Nexus 5000 Series NX-OS Software Upgrade and Downgrade Guide, Release 4.2(1)N1(1)

Regulatory Compliance and Safety Information for the Cisco Nexus 5000 Series Switches and Cisco Nexus 2000 Series Fabric Extenders

Licensing Guide

Cisco NX-OS Licensing Guide

Command References

Cisco Nexus 5000 Series NX-OS FabricPath Command Reference

Cisco Nexus 5000 Series NX-OS Fabric Extender Command Reference

Cisco Nexus 5000 Series NX-OS Fibre Channel Command Reference

Cisco Nexus 5000 Series NX-OS Fundamentals Command Reference

Cisco Nexus 5000 Series NX-OS Layer 2 Interfaces Command Reference

Cisco Nexus 5000 Series NX-OS Multicast Routing Command Reference

Cisco Nexus 5000 Series NX-OS QoS Command Reference

Cisco Nexus 5000 Series NX-OS Security Command Reference

Cisco Nexus 5000 Series NX-OS System Management Command Reference

Cisco Nexus 5000 Series NX-OS TrustSec Command Reference

Cisco Nexus 5000 Series NX-OS Unicast Routing Command Reference

Cisco Nexus 5000 Series NX-OS vPC Command Reference

Technical References

Cisco Nexus 5000 Series and Cisco Nexus 2000 Series Fabric Extender MIBs Reference

Error and System Messages

Nexus 5000 Series NX-OS System Message Reference

Troubleshooting Guide

Cisco Nexus 5000 Troubleshooting Guide

Obtaining Documentation and Submitting a Service Request

For information on obtaining documentation, submitting a service request, and gathering additional information, see the monthly What's New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation:

http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html

Subscribe to the What's New in Cisco Product Documentation as an RSS feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free service. Cisco currently supports RSS Version 2.0.