Contents

Introduction

Background Information

Abbreviations

Workflow of the MoP

Leaf Switch in the Ultra-M Setup

Prerequisites

Switch Replacement Procedure

Introduction

This document describes the steps required in order to replace a faulty Leaf switch (Nexus 93180YC-EX) in an Ultra-M setup that hosts StarOS Virtual Network Functions (VNFs).

Background Information

Ultra-M is a pre-packaged and validated virtualized mobile packet core solution that is designed in order to simplify the deployment of VNFs. The servers that are part of the Ultra-M setup are connected to three different types of switches:

- Catalyst Switch

- Leaf Switch

- Spine Switch

The network topology of an Ultra-M setup is as shown in this image.

UltraM Network Topology

Note: The Network topology is only a representation, the connections between the switches might slightly vary and it depends on the solution deployed. This document is intended for the Cisco personnel who are familiar with Cisco Ultra-M setup and Nexus Switch.

Abbreviations

VNF

Virtual Network Function

API

Application Programming Interface

MOP

Method of Procedure

DI

Distributed Instance

FTP

File Transfer Protocol

SFTP

Secure File Transfer Protocol

BGP

Border Gateway Protocol

BFD

Bidirectional Forwarding Detection

Workflow of the MoP

Highlevel Workflow of the replacement procedure

Leaf Switch in the Ultra-M Setup

In an Ultra-M setup, the Leaf switch handles these networks:

- OpenStack Networks - Handling Tenant, Internal API, Storage Network, Storage Management Network

- StarOS VNF networks - Handling DI-Network and Service Networks

- Uplink Network - Uplink Connection to Spine Switches

Every rack in an Ultra-M setup has a pair of Leaf Switches. The networks that are handled by the leaf switch have redundancy across the

the leaf switches of the rack. So, the networks should not be impacted by the replacement of a leaf switch.

Prerequisites

1. Take a backup of the configuration file from the Leaf switch with the use of ftp/sftp before you proceed with the switch replacement.

POD1-leaf1# copy running-config sftp:

Enter destination filename: [POD1-leaf1-running-config] backup-leaf-1-cfg

Enter vrf (If no input, current vrf 'default' is considered): management

Enter hostname for the sftp server: 10.10.10.10

Enter username: admin

The authenticity of host '10.10.10.10 (10.10.10.10)' can't be established.

RSA key fingerprint is SHA256:fnbUmd2mL5yE94zxrRoKAlvYfQbheXJfQox7m3XfpIU.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added '10.10.10.10' (RSA) to the list of known hosts.

User Access Verification

Password:

Connected to 10.10.10.10

sftp> put /var/tmp/vsh/backup-leaf-1-cfg backup-leaf-1-cfg

Uploading /var/tmp/vsh/backup-leaf-1-cfg to /backup-leaf-1-cfg

/var/tmp/vsh/backup-leaf-1-cfg 100% 33KB 33.2KB/s 00:00

sftp> exit

Copy complete, now saving to disk (please wait)...

Copy complete.

POD1-leaf1#

2. Check the current software version that runs in the switch and make a note of it.

POD1-leaf1# show version

<snip>

Software

BIOS: version 07.59

NXOS: version 7.0(3)I7(3)

BIOS compile time: 08/26/2016

NXOS image file is: bootflash:///nxos.7.0.3.I7.3.bin

NXOS compile time: 2/12/2018 13:00:00 [02/12/2018 19:13:48]

3. Check the current license entitlement.

POD1-leaf1 # show license usage

Feature Ins Lic Status Expiry Date Comments

Count

--------------------------------------------------------------------------------

N9K_LIC_1G No - Unused -

VPN_FABRIC No - Unused -

FCOE_NPV_PKG No - Unused -

SECURITY_PKG No 0 Unused -

N9K_UPG_EX_10G No - Unused -

TP_SERVICES_PKG No - Unused -

NXOS_ADVANTAGE_GF No - Unused -

NXOS_ADVANTAGE_M4 No - Unused -

NXOS_ADVANTAGE_XF No - Unused -

NXOS_ESSENTIALS_GF No - Unused -

NXOS_ESSENTIALS_M4 No - Unused -

NXOS_ESSENTIALS_XF No - Unused -

SAN_ENTERPRISE_PKG No - Unused -

PORT_ACTIVATION_PKG No 0 Unused -

NETWORK_SERVICES_PKG No - Unused -

NXOS_ADVANTAGE_M8-16 No - Unused -

NXOS_ESSENTIALS_M8-16 No - Unused -

FC_PORT_ACTIVATION_PKG No 0 Unused -

LAN_ENTERPRISE_SERVICES_PKG Yes - In use Never -

--------------------------------------------------------------------------------

4. Make a note of the physical cables connected to the switch and the port status.

POD1-leaf1  # show int status

5. Check the current status of the port channels, Border Gateway Protocol (BGP) peers and the Bidirectional Forwarding Detection (BFD) neighbors.

POD1-leaf1  #  show port-channel summary

POD1-leaf1  #  show ip bgp summary vrf all

POD1-leaf1  #  show ipv6 bgp summary vrf all

POD1-leaf1  #  show bfd neighbor vrf all

POD1-leaf1  #  show bfd ipv6 neighbor vrf all

Switch Replacement Procedure

1. Install the new switch in the rack and connect the cables to the switch as noted. The steps for switch installation can be found here: Cisco Nexus 93180YC-EX NX-OS Mode Hardware Installation Guide

2. Do the initial IP configuration in order to enable access to the switch through the Management IP address for further configuration. The steps in order to perform the initial IP settings is: Connecting the Switch to the Network

3. Check the software version of the Nexus switch and upgrade/downgrade to the previous software version as per this link: Cisco Nexus 9000 Series NX-OS Software Upgrade and Downgrade Guide

4. Check the license level of the new Nexus switch and confirm it is as per the old license settings.

POD1-leaf1  # show license usage

Feature Ins Lic Status Expiry Date Comments

Count

--------------------------------------------------------------------------------

N9K_LIC_1G No - Unused -

VPN_FABRIC No - Unused -

FCOE_NPV_PKG No - Unused -

SECURITY_PKG No 0 Unused -

N9K_UPG_EX_10G No - Unused -

TP_SERVICES_PKG No - Unused -

NXOS_ADVANTAGE_GF No - Unused -

NXOS_ADVANTAGE_M4 No - Unused -

NXOS_ADVANTAGE_XF No - Unused -

NXOS_ESSENTIALS_GF No - Unused -

NXOS_ESSENTIALS_M4 No - Unused -

NXOS_ESSENTIALS_XF No - Unused -

SAN_ENTERPRISE_PKG No - Unused -

PORT_ACTIVATION_PKG No 0 Unused -

NETWORK_SERVICES_PKG No - Unused -

NXOS_ADVANTAGE_M8-16 No - Unused -

NXOS_ESSENTIALS_M8-16 No - Unused -

FC_PORT_ACTIVATION_PKG No 0 Unused -

LAN_ENTERPRISE_SERVICES_PKG Yes - In use Never -

--------------------------------------------------------------------------------

5. Transfer the configuration backup to the new switch.

server-backup$ sftp admin@10.10.10.10

The authenticity of host '10.10.10.10 (10.10.10.10)' can't be established.

RSA key fingerprint is SHA256:fnbUmd2mL5yE94zxrRoKAlvYfQbheXJfQox7m3XfpIU.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added '10.10.10.10' (RSA) to the list of known hosts.

User Access Verification

Password:

Password:

Connected to 10.10.10.10.

sftp> ls

20170607_193430_poap_15489_1.log 20170607_193430_poap_15489_2.log 20170607_193430_poap_15489_init.log backup-leaf-1-cfg

sftp> put backup-leaf-1-cfg

Uploading backup-spine-cfg-2 to /backup-leaf-1-cfg

backup-leaf-1-cfg 100% 33KB 23.5KB/s 00:01

sftp> bye

POD1-leaf1 # copy bootflash:///backup-leaf-1-cfg startup-config

Copy progress 100% 33KB

Copy complete, now saving to disk (please wait)...

Copy complete.

6. Reload the switch after you load the backup configuration.

POD1-leaf1  # reload

7. Verify the port status, port channels, BGP peers and the BFD neighbors.

POD1-leaf1# show int status

POD1-leaf1  #  show port-channel summary

POD1-leaf1  #  show ip bgp summary vrf all

POD1-leaf1  #  show ipv6 bgp summary vrf all

POD1-leaf1  #  show bfd neighbor vrf all

POD1-leaf1  #  show bfd ipv6 neighbor vrf all