Replace Nexus 93180YC-EX Leaf Switch - CPS

Introduction

This document describes the steps required in order to replace a faulty Leaf switch (Nexus 93180YC-EX) in an Ultra-M setup that hosts StarOS Virtual Network Functions (VNFs).

Background Information

Ultra-M is a pre-packaged and validated virtualized mobile packet core solution that is designed in order to simplify the deployment of VNFs. The servers that are part of the Ultra-M setup are connected to three different types of switches:

• Catalyst Switch
• Leaf Switch
• Spine Switch

The network topology of an Ultra-M setup is as shown in this image.

UltraM Network Topology

Note: The Network topology is only a representation, the connections between the switches might slightly vary and it depends on the solution deployed. This document is intended for the Cisco personnel who are familiar with Cisco Ultra-M setup and Nexus Switch.

Abbreviations

Workflow of the MoP

Highlevel Workflow of the replacement procedure  

Leaf Switch in the Ultra-M Setup

In an Ultra-M setup, the Leaf switch handles these networks:

• OpenStack Networks - Handling Tenant, Internal API, Storage Network, Storage Management Network 
• StarOS VNF networks - Handling DI-Network and Service Networks
• Uplink Network - Uplink Connection to Spine Switches  

Every rack in an Ultra-M setup has a pair of Leaf Switches. The networks that are handled by the leaf switch have redundancy across the 

the leaf switches of the rack. So, the networks should not be impacted by the replacement of a leaf switch. 

Prerequisites

1. Take a backup of the configuration file from the Leaf switch with the use of ftp/sftp before you proceed with the switch replacement.

POD1-leaf1# copy running-config sftp:
Enter destination filename: [POD1-leaf1-running-config] backup-leaf-1-cfg
Enter vrf (If no input, current vrf 'default' is considered): management
Enter hostname for the sftp server: 10.10.10.10
Enter username: admin

The authenticity of host '10.10.10.10 (10.10.10.10)' can't be established.
RSA key fingerprint is SHA256:fnbUmd2mL5yE94zxrRoKAlvYfQbheXJfQox7m3XfpIU.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.10.10.10' (RSA) to the list of known hosts.
User Access Verification
Password: 
Connected to 10.10.10.10
sftp> put /var/tmp/vsh/backup-leaf-1-cfg backup-leaf-1-cfg
Uploading /var/tmp/vsh/backup-leaf-1-cfg to /backup-leaf-1-cfg
/var/tmp/vsh/backup-leaf-1-cfg 100% 33KB 33.2KB/s 00:00 
sftp> exit 
Copy complete, now saving to disk (please wait)...
Copy complete.
POD1-leaf1#

2. Check the current software version that runs in the switch and make a note of it.

POD1-leaf1# show version 
<snip>
Software
 BIOS: version 07.59
 NXOS: version 7.0(3)I7(3)
 BIOS compile time: 08/26/2016
 NXOS image file is: bootflash:///nxos.7.0.3.I7.3.bin
 NXOS compile time: 2/12/2018 13:00:00 [02/12/2018 19:13:48]

3. Check the current license entitlement.

 POD1-leaf1 # show license usage 
Feature Ins Lic Status Expiry Date Comments
 Count
--------------------------------------------------------------------------------
N9K_LIC_1G No - Unused -
VPN_FABRIC No - Unused -
FCOE_NPV_PKG No - Unused -
SECURITY_PKG No 0 Unused -
N9K_UPG_EX_10G No - Unused -
TP_SERVICES_PKG No - Unused -
NXOS_ADVANTAGE_GF No - Unused -
NXOS_ADVANTAGE_M4 No - Unused -
NXOS_ADVANTAGE_XF No - Unused -
NXOS_ESSENTIALS_GF No - Unused -
NXOS_ESSENTIALS_M4 No - Unused -
NXOS_ESSENTIALS_XF No - Unused -
SAN_ENTERPRISE_PKG No - Unused -
PORT_ACTIVATION_PKG No 0 Unused -
NETWORK_SERVICES_PKG No - Unused -
NXOS_ADVANTAGE_M8-16 No - Unused -
NXOS_ESSENTIALS_M8-16 No - Unused -
FC_PORT_ACTIVATION_PKG No 0 Unused -
LAN_ENTERPRISE_SERVICES_PKG Yes - In use Never -
-------------------------------------------------------------------------------- 

4. Make a note of the physical cables connected to the switch and the port status.

 POD1-leaf1  # show int status 

5. Check the current status of the port channels, Border Gateway Protocol (BGP) peers and the Bidirectional Forwarding Detection (BFD) neighbors.

 POD1-leaf1  #  show port-channel summary
 POD1-leaf1  #  show ip bgp summary vrf all
 POD1-leaf1  #  show ipv6 bgp summary vrf all
 POD1-leaf1  #  show bfd neighbor vrf all
 POD1-leaf1  #  show bfd ipv6 neighbor vrf all

Switch Replacement Procedure

1. Install the new switch in the rack and connect the cables to the switch as noted. The steps for switch installation can be found here: Cisco Nexus 93180YC-EX NX-OS Mode Hardware Installation Guide

2. Do the initial IP configuration in order to enable access to the switch through the Management IP address for further configuration. The steps in order to perform the initial IP settings is: Connecting the Switch to the Network

3. Check the software version of the Nexus switch and upgrade/downgrade to the previous software version as per this link: Cisco Nexus 9000 Series NX-OS Software Upgrade and Downgrade Guide

4. Check the license level of the new Nexus switch and confirm it is as per the old license settings.  

 POD1-leaf1  # show license usage  
 Feature Ins Lic Status Expiry Date Comments 
 Count 
 -------------------------------------------------------------------------------- 
 N9K_LIC_1G No - Unused - 
 VPN_FABRIC No - Unused - 
 FCOE_NPV_PKG No - Unused - 
 SECURITY_PKG No 0 Unused - 
 N9K_UPG_EX_10G No - Unused - 
 TP_SERVICES_PKG No - Unused - 
 NXOS_ADVANTAGE_GF No - Unused - 
 NXOS_ADVANTAGE_M4 No - Unused - 
 NXOS_ADVANTAGE_XF No - Unused - 
 NXOS_ESSENTIALS_GF No - Unused - 
 NXOS_ESSENTIALS_M4 No - Unused - 
 NXOS_ESSENTIALS_XF No - Unused - 
 SAN_ENTERPRISE_PKG No - Unused - 
 PORT_ACTIVATION_PKG No 0 Unused - 
 NETWORK_SERVICES_PKG No - Unused - 
 NXOS_ADVANTAGE_M8-16 No - Unused - 
 NXOS_ESSENTIALS_M8-16 No - Unused - 
 FC_PORT_ACTIVATION_PKG No 0 Unused - 
 LAN_ENTERPRISE_SERVICES_PKG Yes - In use Never - 
 --------------------------------------------------------------------------------  

5. Transfer the configuration backup to the new switch.

server-backup$ sftp admin@10.10.10.10
The authenticity of host '10.10.10.10 (10.10.10.10)' can't be established.
RSA key fingerprint is SHA256:fnbUmd2mL5yE94zxrRoKAlvYfQbheXJfQox7m3XfpIU.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.10.10.10' (RSA) to the list of known hosts.
User Access Verification
Password: 
Password: 
Connected to 10.10.10.10.
sftp> ls 
20170607_193430_poap_15489_1.log 20170607_193430_poap_15489_2.log 20170607_193430_poap_15489_init.log backup-leaf-1-cfg
sftp> put backup-leaf-1-cfg
Uploading backup-spine-cfg-2 to /backup-leaf-1-cfg
backup-leaf-1-cfg 100% 33KB 23.5KB/s 00:01 
sftp> bye

 POD1-leaf1 # copy bootflash:///backup-leaf-1-cfg startup-config
Copy progress 100% 33KB
Copy complete, now saving to disk (please wait)...
Copy complete.

6. Reload the switch after you load the backup configuration.

  POD1-leaf1  # reload 

 7. Verify the port status, port channels, BGP peers and the BFD neighbors.

 POD1-leaf1# show int status
POD1-leaf1  #  show port-channel summary
 POD1-leaf1  #  show ip bgp summary vrf all
 POD1-leaf1  #  show ipv6 bgp summary vrf all
 POD1-leaf1  #  show bfd neighbor vrf all
 POD1-leaf1  #  show bfd ipv6 neighbor vrf all