Catalyst 9600 - Migration to Quad-Supervisor Stackwise Virtual setup

Introduction

The objective of this document is to help configure two Catalyst 9600 switches running IOS-XE versions 16.11.X or 16.12.X  to Quad-supervisor Stackwise Virtual setup on IOS-XE version 17.X.

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

Cisco Stackwise Virtual

High Availability using Quad Supervisor redundancy

Components Used

This document is not restricted to specific software and hardware versions. Quad-Supervisor Stackwise Virtual setup can be configured on any Catalyst 9600 chassis running IOS XE version 17.2.X or higher. The example outputs in this document are based on these software and hardware versions:

• Catalyst 9606 chassis running C9600-SUP-1
• IOS-XE version 17.2.1

The information in this document was created from the devices in a specific lab environment. If your network is live, ensure that you understand the potential impact of any command.

Background Information

Cisco StackWise Virtual is a network system virtualization technology that pairs two switches into one virtual switch. Switches in Cisco StackWise Virtual setup simplify operational efficiency with single control and management plane, scaled system bandwidth with distributed forwarding plane, and assist in building resilient networks using Cisco recommended network design.

Here are different high availability configuration options available on a Catalyst 9600 switch.

Stand-alone / Redundant Supervisor High Availability

Single chassis setup with no Stackwise Virtual configuration. This configuration utilizes  dual-supervisor in one chassis for high availbility. On Catalyst 9600, this is supported IOS-XE version 16.11 onwards.

2-Supervisor Stackwise Virtual

Stackwise Virtual setup is formed between two Catalyst 9600 chassis with single supervisor in each chassis. On Catalyst 9600, this is supported IOS-XE version16.12 onwards.

Quad-SUP Stackwise Virtual

Stackwise Virtual setup is formed between two Catalyst 9600 chassis with dual supervisors in each chassis. On Catalyst 9600, this is supported IOS-XE version 17.2 onwards. This is the configuration this document will focus on.

Terminology

Stackwise-A - Active supervisor of Stackwise Virtual setup.

Stackwise-S - Standby supervisor of Stackwise Virtual setup.

ICA - In-Chassis Active supervisor engine.

ICS - In-Chassis Standby supervisor engine.

Configure

Migration to Quad-Supervisor Stackwise Virtual depends on the current configuration on the C9600 switches. Follow the right procedure below depending on existing configuration.

1. Configure two stand-alone Catalyst 9600 chassis to Quad-Supervisor Stackwise Virtual setup

Note: The steps below are to be followed only if currently both C9600 switches are running as stand-alone with dual supervisors in each chassis and they are not configured for Stackwise Virtual.

1. If not already running on IOS-XE Version17.2.X , upgrade both switches individually to IOS-XE Version17.2.X in INSTALL mode.

2. Ensure supervisor in slot 3 Is ACTIVE in both chassis before moving ahead with Stackwise Virtual conversion. This step is important.

3. Configure Stackwise Virtual between both chassis and reload. Use steps listed here.

Once complete, two C9600 switches will be available in Quad-supervisor Stackwise Virtual setup.

2. Migrate from 2-Supervisor Stackwise Virtual setup to Quad-Supervisor Stackwise Virtual setup

The options below are to be followed only if two C9606 chassis are pre-configured for Stackwise Virtual using single supervisor engine in each chassis.  There is more than one way to go about this depending on the feasibility of having downtime and availability of spare chassis to upgrade the In-Chassis Standby (ICS) supervisors.

2.1 Hitless migration when a spare C9600 chassis is available

Note: Follow these steps if both chassis are pre configured for 2-Supervisor Stackwise Virtual with single supervisor in each chassis and the secondary In-Chassis Standby supervisor engine (ICS) is not installed yet. This step requires the ICS to already be running IOS-XE Version 17.2 (or higher) or a spare chassis to upgrade ICS to IOS-XE Version 17.2 (or higher) in a separate step.

1. If ICS is not already on IOS-XE Version 17.2 (or higher), use the spare C9600 chassis to upgrade both ICS to IOS-XE Version 17.2.X (same as ICA) in INSTALL mode.

2. Upgrade the 2-Supervisor Stackwise Virtual setup to IOS-XE Version 17.2 (or higher) in INSTALL mode, if not already done.

3. Once the 2-Supervisor Stackwise comes up on IOS-XE Version 17.2 (or higher), insert the ICS in both chassis. ICS will boot and become part of Quad-Sup. 

Once complete, two C9600 switches will be available in Quad-supervisor Stackwise Virtual setup.

2.2 Hitless migration when a spare C9600 chassis is unavailable

Note: Follow these steps if both C9600 chassis are already configured for 2-Supervisor Stackwise Virtual with ICS in reset mode (on IOS-XE Version 16.12) or ICS is not installed yet and there is no spare chassis to upgrade ICS to IOS-XE Version 17.x.

1. If inserted, pull out the ICS supervisors from both chassis.

2. Upgrade the 2-Supervisor Stackwise Virtual setup to IOS-XE Version 17.2 (or higher) in INSTALL mode, if not already done.

3. Software auto-upgrade feature is enabled by default. If this was removed, enable the feature. 

C9600-SVL-Active(config)# software auto-upgrade enable

C9600-SVL-Active# show run all | in software auto-upgrade
no software auto-upgrade source url
software auto-upgrade enable

4. Insert the ICS supervisors and establish a console session. Immediately after insertion Press Ctrl+C on keyboard till bootup is interrupted and a rommon prompt is seen. This step is very important.

Last reset cause:PowerOn
C9600-SUP-1 platform with 16777216 Kbytes of main memory

Preparing to autoboot. [Press Ctrl-C to interrupt] 5 (interrupted)
rommon 1 >

Note: Perform steps 5 to 7 only if the ICS supervisor module was previously used in a Stackwise Virtual setup. If not, move to Step 8.

5. Look for any Stackwise-Virtual rommon variables. These variables starts with D_STACK. For eg:

D_STACK_DISTR_STACK_LINK2=""
D_STACK_DAD="Fo1/0/13,Fo1/0/15,"
D_STACK_MODE="aggregation"
D_STACK_DOMAIN_NUM="255"
D_STACK_DISTR_STACK_LINK1="Fo1/0/10,Fo1/0/15,Fo1/0/16,Fo1/0/17,Fo1/0/3,Fo1/0/6,"

 6. Unset all the variables above.

rommon 1 > unset D_STACK_DAD
rommon 1 > unset D_STACK_DISTR_STACK_LINK1
rommon 1 > unset D_STACK_DISTR_STACK_LINK2
rommon 1 > unset D_STACK_DOMAIN_NUM
rommon 1 > unset D_STACK_MODE

7. Look for variable "SWITCH_NUMBER=1". If the switch number is 2, set the variable to 1. If it is already 1, move to the next step.

rommon 1 > set
<snip>
SWITCH_NUMBER=2
<snip>

rommon 2 > SWITCH_NUMBER=1
rommon 3 

 8. Set this variable to manually boot the supervisor.

rommon 3 > MANUAL_BOOT=YES
rommon 4 >

 9. Manually boot the ICS supervisor in bundle mode on IOS-XE version 17.x using a USB stick. Do not change boot variable in rommon. Just boot it manually from rommon as below.

rommon 4 > boot usbflash0:cat9k_iosxe.17.02.01.SPA.bin

 10. The supervisor will reset as it gets converted from stand-alone to stackwise mode. It will again fall back into rommon.

 11. Unset manual boot to enable auto-boot.

rommon 1 > unset MANUAL_BOOT

 12. Again, manually boot the ICS supervisor on IOS-XE Version 17.x (like step 9) using a USB stick. Do not change boot variable in rommon. Just boot it manually from rommon. This will boot the ICS in BUNDLE mode.

 13. Software auto-upgrade will automatically take care of upgrading the ICS supervisor on IOS-XE Version 17.x in INSTALL mode and will reload both the ICS supervisors to come up in Route Processor Redundancy (RPR) mode.  On IOS-XE version 17.2, ICS on Stackwise Virtual Active switch will be upgraded automatically through auto-upgrade (see step 3). However, ICS on Standby switch will need to be manually upgraded by issuing “install auto-upgrade” on active switch’s console (as below) after ICS in the active chassis has finished booting up in install mode.

C9600-SVL-Active# install autoupgrade

Once complete, two C9600 switches will be available in Quad-supervisor Stackwise Virtual setup.

2.3 Migrate with downtime

Note: Follow these steps if both C9600 switches are already configured for 2-Supervisor Stackwise Virtual setup with single supervisor in each chassis (SUP #1 and SUP #2), ICS is not installed yet (SUP #3 and SUP #4) and there is no spare chassis to upgrade ICS to 17.x code. This method involves downtime.

1. Upgrade the 2-Supervisor Stackwise Virtual to IOS-XE Version 17.x in INSTALL mode and ensure that Stackwise Virtual is up after upgrade.

2. Power down one Stackwise-Virtual chassis – say SWITCH-2.  Traffic going through SWITCH-2 will start going through SWITCH-1 if downstream network is connected via a multi chassis etherchannel. This failover will involve a small hit to the traffic.

3. Pull out the supervisor (SUP #2) which was in Stackwise Virtual mode from SWITCH-2  and keep it safe.

4. Unseat the line cards from SWITCH-2 but let them be in the chassis. This step will avoid any impact on downstream devices.

Note: With line cards unseated, there is a chance that system may power down due to insufficient cooling if kept in this state for too long. Please execute next steps (step 5 onwards) as quickly as possible.

  5. Insert the “new” supervisor engines in this chassis (SUP #3 and SUP #4). Power them on and upgrade them to IOS-XE Version 17.x (in stand-alone / redundant supervisor high availability mode) using Install commands.

  6. Pull both the supervisors off Switch-2 (SUP #3 and SUP #4) and install the original supervisor (SUP #2) from step 3 and insert back all the line cards.

  7. Switch 2 supervisor will boot up as Standby. Wait until supervisor becomes Stackwise Virtual standby, all line cards are up and traffic re-converges.

  8. Insert the new ICS supervisors (SUP #3 and SUP #4), one in each chassis and let them boot up. 

Once complete, two C9600 switches will be available in Quad-supervisor Stackwise Virtual setup.