Replacement of Nexus 93180YC-EX Leaf Switch - CPAR

Introduction

This document describes the steps required to replace a faulty Leaf switch (Nexus 93180YC-EX) in an Ultra-M setup.

This procedure applies for an Openstack environment using NEWTON version where ESC does not manage Cisco Prime Access Registrar (CPAR) and CPAR is installed directly on the VM deployed on Openstack.

Background Information

Ultra-M is a pre-packaged and validated virtualized mobile packet core solution designed to simplify the deployment of VNFs. The servers that are part of the Ultra-M setup are connected to three different types of switches :

• Catalyst Switch
• Leaf Switch
• Spine Switch

The network topology of a Ultra-M setup is shown in the image:

Note: The Network topology is only a representation, the connections between the switches might slightly vary, it depends upon the solution deployed.

This document is intended for the Cisco personnel who are familiar with Cisco Ultra-M setup and Nexus Switch.

Abbreviations

Workflow of the MoP

This image shows the high level workflow of the replacement procedure.

Leaf Switch in the UltraM Setup

In an UltraM setup, the Leaf switch handles these network:

• Openstack Network - Handles Tenant, Internal API, Storage network, Storage management network.
• StarOS VNF Network - Handles DI-Network and Service networks.
• Uplink Network - Uplink connection to Spine Switches.  

Every rack in an UltraM setup has a pair of Leaf switches. The networks that are handled by the leaf switch have redundancy across the leaf switches of the rack so that the networks are not impacted by the replacement of a leaf switch. 

Prerequisite

Step 1. Take a backup of the configuration file from the Leaf switch through FTP/SFTP before it proceedes with the switch replacement. 

 AL03-N93180-POD2-leaf1# copy running-config sftp: 
 Enter destination filename: [AL03-N93180-POD2-leaf1-running-config] backup-leaf-1-cfg
 Enter vrf (If no input, current vrf 'default' is considered): management 
 Enter hostname for the sftp server: 10.10.10.10
 Enter username: admin 

The authenticity of host '10.10.10.10 (10.10.10.10)' can't be established.
ECDSA key fingerprint is SHA256:dnHSXQ6fyYX5v8DU87oqVkCoA8CSPW7CpT+o2qilNI4.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.10.10.10' (ECDSA) to the list of known hosts. 
admin@10.10.10.10's password: 
Connected to 10.10.10.10.
sftp> put /var/tmp/vsh/backup-leaf-1-cfg backup-leaf-1-cfg 
Uploading /var/tmp/vsh/backup-leaf-1-cfg to /root/backup-leaf-1-cfg
 /var/tmp/vsh/backup-leaf-1-cfg 0% 0 0.0KB/s --:-- ETA /var/tmp/vsh/backup-leaf-1-cfg 100% 8588 8.4KB/s 00:00 
sftp> exit 
Copy complete, now saving to disk (please wait)...
Copy complete.

Step  2. Check the current software version that runs in the switch and make a note of it.

AL03-N93180-POD2-leaf1# show version 
<snip>
Software
 BIOS: version 07.59
 NXOS: version 7.0(3)I7(3)
 BIOS compile time: 08/26/2016
 NXOS image file is: bootflash:///nxos.7.0.3.I7.3.bin
 NXOS compile time: 2/12/2018 13:00:00 [02/12/2018 19:13:48]

Step 3. Check the current license entitlement.

AL03-N93180-POD2-leaf1# show license usage 
Feature Ins Lic Status Expiry Date Comments
 Count
--------------------------------------------------------------------------------
N9K_LIC_1G No - Unused -
VPN_FABRIC No - Unused -
FCOE_NPV_PKG No - Unused -
SECURITY_PKG No 0 Unused -
N9K_UPG_EX_10G No - Unused -
TP_SERVICES_PKG No - Unused -
NXOS_ADVANTAGE_GF No - Unused -
NXOS_ADVANTAGE_M4 No - Unused -
NXOS_ADVANTAGE_XF No - Unused -
NXOS_ESSENTIALS_GF No - Unused -
NXOS_ESSENTIALS_M4 No - Unused -
NXOS_ESSENTIALS_XF No - Unused -
SAN_ENTERPRISE_PKG No - Unused -
PORT_ACTIVATION_PKG No 0 Unused -
NETWORK_SERVICES_PKG No - Unused -
NXOS_ADVANTAGE_M8-16 No - Unused -
NXOS_ESSENTIALS_M8-16 No - Unused -
FC_PORT_ACTIVATION_PKG No 0 Unused -
LAN_ENTERPRISE_SERVICES_PKG No - Unused -
-------------------------------------------------------------------------------- 

Step 4. Make a note of the physical cables connected to the switch and the port status.

AL03-N93180-POD2-leaf1# show int status

Step 5. Check the current status of the port channels, BGP peers and the BFD neighbors.

AL03-N93180-POD2-leaf1# show port-channel summary
AL03-N93180-POD2-leaf1# show ip bgp summary vrf all
AL03-N93180-POD2-leaf1# show ipv6 bgp summary vrf all
AL03-N93180-POD2-leaf1# show bfd neighbor vrf all
AL03-N93180-POD2-leaf1# show bfd ipv6 neighbor vrf all

Switch Replacement procedure

Step 1. Install the new switch in the rack and connect the cables to the switch as noted. The steps for switch installation can be found in the link below:

      Cisco Nexus 93180YC-EX NX-OS Mode Hardware Installation Guide

Step 2. Do the inital IP configuration to enable access to the switch through the Management IP address for further configuration. The steps for performing the Initial IP settings is in this link: 

      Connecting the Switch to the Network

Step 3. Check the software version of the Nexus switch and upgrade/downgrade to the previous software version as per the given link:

      Cisco Nexus 9000 Series NX-OS Software Upgrade and Downgrade Guide

Step 4. Check the license level of the new Nexus switch and confirm it is as per the old license settings.

AL03-N93180-POD2-leaf1# show license usage 
Feature Ins Lic Status Expiry Date Comments
Count
--------------------------------------------------------------------------------
N9K_LIC_1G No - Unused -
VPN_FABRIC No - Unused -
FCOE_NPV_PKG No - Unused -
SECURITY_PKG No 0 Unused -
N9K_UPG_EX_10G No - Unused -
TP_SERVICES_PKG No - Unused -
NXOS_ADVANTAGE_GF No - Unused -
NXOS_ADVANTAGE_M4 No - Unused -
NXOS_ADVANTAGE_XF No - Unused -
NXOS_ESSENTIALS_GF No - Unused -
NXOS_ESSENTIALS_M4 No - Unused -
NXOS_ESSENTIALS_XF No - Unused -
SAN_ENTERPRISE_PKG No - Unused -
PORT_ACTIVATION_PKG No 0 Unused -
NETWORK_SERVICES_PKG No - Unused -
NXOS_ADVANTAGE_M8-16 No - Unused -
NXOS_ESSENTIALS_M8-16 No - Unused -
FC_PORT_ACTIVATION_PKG No 0 Unused -
LAN_ENTERPRISE_SERVICES_PKG No - Unused -
-------------------------------------------------------------------------------- 

Step 5. Transfer the configuration backup to the new switch.

server-backup$ sftp admin@10.10.10.10
The authenticity of host '10.10.10.10 (10.10.10.10)' can't be established.
RSA key fingerprint is SHA256:fnbUmd2mL5yE94zxrRoKAlvYfQbheXJfQox7m3XfpIU.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.10.10.10' (RSA) to the list of known hosts.
User Access Verification
Password: 
Password: 
Connected to 10.10.10.10.
sftp> ls 
20170607_193430_poap_15489_1.log 20170607_193430_poap_15489_2.log 20170607_193430_poap_15489_init.log backup-leaf-1-cfg
sftp> put backup-leaf-1-cfg
Uploading backup-spine-cfg-2 to /backup-leaf-1-cfg
backup-leaf-1-cfg 100% 33KB 23.5KB/s 00:01 
sftp> bye

AL03-N93180-POD2-leaf1# copy bootflash:///backup-leaf-1-cfg startup-config
Copy progress 100% 33KB
Copy complete, now saving to disk (please wait)...
Copy complete.

Step 6. Reload the switch after loading the backup configuration.

 AL03-N93180-POD2-leaf1# reload

Step 7. Verify the port status,  port channels, BGP peers and the BFD neighbors.

AL03-N93180-POD2-leaf1# show int status
AL03-N93180-POD2-leaf1# show port-channel summary
AL03-N93180-POD2-leaf1# show ip bgp summary vrf all
AL03-N93180-POD2-leaf1# show ipv6 bgp summary vrf all
AL03-N93180-POD2-leaf1# show bfd neighbor vrf all
AL03-N93180-POD2-leaf1# show bfd ipv6 neighbor vrf all