PDF(290.5 KB) View with Adobe Reader on a variety of devices
ePub(469.6 KB) View in various apps on iPhone, iPad, Android, Sony Reader, or Windows Phone
Mobi (Kindle)(246.7 KB) View on Kindle device or Kindle app on multiple devices
Updated:August 30, 2018
The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
This document describes the steps required to replace a faulty Spine switch (Nexus 9236C) in an Ultra-M setup.
This procedure applies for an Openstack environment through NEWTON version where ESC does not manage Cisco Prime Access Registrar (CPAR) and CPAR is installed directly on the VM deployed on Openstack.
Ultra-M is a pre-packaged and validated virtualized mobile packet core solution designed to simplify the deployment of VNFs. The servers that are part of the Ultra-M setup are connected to three different types of switches :
This image shows the network topology of a Ultra-M setup:
Note: The Network topology is only a representation, the connections between the switches might slightly vary, it depends upon the solution deployed.
This document is intended for the Cisco personnel who are familiar with Cisco Ultra-M setup and Catalyst Switch operations.
Virtual Network Function
Nexus 9236C Switch as Spine
Method of Procedure
Local Area Network
File Transfer Protocol
Trivial File Transfer Protocol
Cisco Integrated Management Controller
Workflow of the MoP
This image shows the high level workflow of Replacement Procedure.
Spine Switch in UltraM Setup
Take a backup of the configuration file from the Spine switch through FTP/TFTP before it proceedes with the switch replacement.
AL03-N9236c-POD2-spine1# copy running-config sftp: Enter destination filename: [AL03-N9236c-POD2-spine1-running-config] backup-spine-cfg-2 Enter vrf (If no input, current vrf 'default' is considered): management Enter hostname for the sftp server: 10.1.1.1 Enter username: root
The authenticity of host '10.1.1.1 (10.1.1.1)' can't be established. ECDSA key fingerprint is SHA256:dnHSXQ6fyYX5v8DU87oqVkCoA8CSPW7CpT+o2qilNI4. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '10.1.1.1' (ECDSA) to the list of known hosts. firstname.lastname@example.org's password: Connected to 10.1.1.1. sftp> put /var/tmp/vsh/backup-spine-cfg-2 backup-spine-cfg-2 Uploading /var/tmp/vsh/backup-spine-cfg-2 to /root/backup-spine-cfg-2 /var/tmp/vsh/backup-spine-cfg-2 0% 0 0.0KB/s --:-- ETA /var/tmp/vsh/backup-spine-cfg-2 100% 11KB 11.2KB/s 00:00 sftp> exit Copy complete, now saving to disk (please wait)... Copy complete.
Step 1. Check the current software version that runs in the switch and make a note of it.
AL03-N9236c-POD2-spine1# show ver Cisco Nexus Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac Copyright (C) 2002-2018, Cisco and/or its affiliates. All rights reserved. The copyrights to certain works contained in this software are owned by other third parties and used and distributed under their own licenses, such as open source. This software is provided "as is," and unless otherwise stated, there is no warranty, express or implied, including but not limited to warranties of merchantability and fitness for a particular purpose. Certain components of this software are licensed under the GNU General Public License (GPL) version 2.0 or GNU General Public License (GPL) version 3.0 or the GNU Lesser General Public License (LGPL) Version 2.1 or Lesser General Public License (LGPL) Version 2.0. A copy of each such license is available at http://www.opensource.org/licenses/gpl-2.0.php and http://opensource.org/licenses/gpl-3.0.html and http://www.opensource.org/licenses/lgpl-2.1.php and http://www.gnu.org/licenses/old-licenses/library.txt.
Software BIOS: version 07.59 NXOS: version 7.0(3)I7(3) BIOS compile time: 08/26/2016 NXOS image file is: bootflash:///nxos.7.0.3.I7.3.bin NXOS compile time: 2/12/2018 13:00:00 [02/12/2018 19:13:48]
Hardware cisco Nexus9000 C9236C chassis Intel(R) Xeon(R) CPU @ 1.80GHz with 16400560 kB of memory. Processor Board ID FDO211613CA
Last reset at 705173 usecs after Fri Mar 2 17:56:31 2018 Reason: Reset due to upgrade System version: 7.0(3)I5(2) Service:
plugin --More-- Core Plugin, Ethernet Plugin
Step 2. Check the current license usage.
AL03-N9236c-POD2-spine1# show license uasgesage Feature Ins Lic Status Expiry Date Comments Count -------------------------------------------------------------------------------- N9K_LIC_1G No - Unused - VPN_FABRIC No - Unused - FCOE_NPV_PKG No - Unused - SECURITY_PKG No 0 Unused - N9K_UPG_EX_10G No - Unused - TP_SERVICES_PKG No - Unused - NXOS_ADVANTAGE_GF No - Unused - NXOS_ADVANTAGE_M4 No - Unused - NXOS_ADVANTAGE_XF No - Unused - NXOS_ESSENTIALS_GF No - Unused - NXOS_ESSENTIALS_M4 No - Unused - NXOS_ESSENTIALS_XF No - Unused - SAN_ENTERPRISE_PKG No - Unused - PORT_ACTIVATION_PKG No 0 Unused - NETWORK_SERVICES_PKG No - Unused - NXOS_ADVANTAGE_M8-16 No - Unused - NXOS_ESSENTIALS_M8-16 No - Unused - FC_PORT_ACTIVATION_PKG No 0 Unused - LAN_ENTERPRISE_SERVICES_PKG Yes - In use Never - --------------------------------------------------------------------------------
Step 3. Ensure that the physical cables connected to the switch port-channel, Vlan and also the port status are good as expected.
AL03-N9236c-POD2-spine1# show interface status | grep connected mgmt0 -- connected routed full 100 -- Eth1/1 Connected to leaf1 connected trunk full 100G QSFP-100G-AOC5M Eth1/2 Connected to leaf1 connected trunk full 100G QSFP-100G-AOC5M Eth1/4 Connected to leaf2 connected trunk full 100G QSFP-100G-AOC5M Eth1/5 Connected to leaf2 connected trunk full 100G QSFP-100G-AOC5M Eth1/7 Connected to Leaf3 connected trunk full 100G QSFP-100G-AOC5M Eth1/8 Connected to Leaf3 connected trunk full 100G QSFP-100G-AOC5M Eth1/10 connect leaf4 connected trunk full 100G QSFP-100G-AOC5M Eth1/11 connect leaf4 connected trunk full 100G QSFP-100G-AOC5M Eth1/21/1 Connected to NMNET connected 101 full 10G QSFP-40G-SR4 Eth1/29 Connected to AL04- connected trunk full 100G QSFP-100G-AOC5M Eth1/30 Connected to AL04- connected trunk full 100G QSFP-100G-AOC5M Eth1/36/1 connected to ARG | connected routed full 10G QSFP-40G-SR4 Eth1/36/2 connected to ARG | disabled routed auto auto QSFP-40G-SR4 Eth1/36/3 connected to ARG | disabled routed auto auto QSFP-40G-SR4 Eth1/36/4 connected to ARG | disabled routed auto auto QSFP-40G-SR4 Po10 -- connected routed full 10G -- Po10.3101 -- connected routed full 10G -- Po10.3109 -- connected routed full 10G -- Po18 -- connected trunk full 100G -- Po21 -- connected trunk full 100G -- Po22 -- connected trunk full 100G -- Po25 -- connected trunk full 100G -- Po26 -- connected trunk full 100G -- Lo1 -- connected routed auto auto -- Vlan101 -- connected routed auto auto -- Vlan1025 vPCRF 1 and vPCRF connected routed auto auto -- Vlan1026 Replication for vP connected routed auto auto -- Vlan1031 External Traffic v connected routed auto auto -- Vlan1032 External External connected routed auto auto -- Vlan1033 External Traffic connected routed auto auto -- Vlan1034 External Traffic v connected routed auto auto -- Vlan1035 External Traffic v connected routed auto auto -- Vlan1036 External Traffic v connected routed auto auto -- Vlan1041 External Traffic A connected routed auto auto -- Vlan1042 External Traffic A connected routed auto auto -- Vlan1043 External Traffic A connected routed auto auto -- Vlan2601 Global Interchassi connected routed auto auto -- Vlan2602 NMNET Interchassis connected routed auto auto --
Step 4. Ensure that BGP neighbors are up.
AL03-N9236c-POD2-spine1# show ip bgp summary vrf all BGP summary information for VRF NMNET, address family IPv4 Unicast BGP router identifier 10.225.247.201, local AS number 65361 BGP table version is 2, IPv4 Unicast config peers 2, capable peers 1 0 network entries and 0 paths using 0 bytes of memory BGP attribute entries [0/0], BGP AS path entries [0/0] BGP community entries [0/0], BGP clusterlist entries [0/0]
BGP summary information for VRF default, address family IPv4 Unicast BGP router identifier 10.0.200.1, local AS number 65361 BGP table version is 161, IPv4 Unicast config peers 2, capable peers 2 18 network entries and 29 paths using 5700 bytes of memory BGP attribute entries [3/480], BGP AS path entries [1/6] BGP community entries [0/0], BGP clusterlist entries [0/0]
Step 3. Transfer the configuration backup to the new switch and copy that configuration to the startup-config.
server-backup$ sftp email@example.com The authenticity of host '10.1.1.1 (10.1.1.1)' can't be established. RSA key fingerprint is SHA256:fnbUmd2mL5yE94zxrRoKAlvYfQbheXJfQox7m3XfpIU. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '10.1.1.1' (RSA) to the list of known hosts. User Access Verification Password: Password: Connected to 10.1.1.1. sftp> ls 20170607_193430_poap_15489_1.log 20170607_193430_poap_15489_2.log 20170607_193430_poap_15489_init.log backup-spine-cfg-2 backup-spine.cfg backup_run_02152018 bios_daemon.dbg cfg-july25th cfg-july25th-spine1 flash: flash:cfg-aug8th-ybattina-afterNSO flash:cfg-jul28th home l3-cfg-aug10th license_FDO211406K1_16.lic lost+found nxos.7.0.3.I4.4.bin nxos.7.0.3.I5.2.bin nxos.7.0.3.I7.3.bin platform-sdk.cmd scripts spine-config-bkp starat virtual-instance vlan.dat 100% 33KB 21.7KB/s 00:01 sftp> put backup-spine-cfg-2 Uploading backup-spine-cfg-2 to /backup-spine-cfg-2 backup-spine-cfg-2 100% 33KB 23.5KB/s 00:01 sftp> bye
AL03-N9236c-POD2-spine1# copy bootflash:///backup-spine-cfg-2 startup-config Copy progress 100% 33KB Copy complete, now saving to disk (please wait)... Copy complete.
Step 4. Reload the switch after loading the backup configuration.
Verification of Replaced Spine Switch
After the switch is available for access, perform the health checks through these commands on the spine switch to verify that replaced switch is available in expected state.
AL03-N9236c-POD2-spine1# show port-channel summary
AL03-N9236c-POD2-spine1# show ip bgp summary vrf all
AL03-N9236c-POD2-spine1# show ipv6 bgp summary vrf all
AL03-N9236c-POD2-spine1# show bfd neighbor vrf all
AL03-N9236c-POD2-spine1# show bfd ipv6 neighbor vrf all