Contents

Introduction

Prerequisites

Requirements

Components Used

Background Information

Switch Boot Up Sequence

Switch Prompt

Recovery Procedure

Introduction

This document describes how to recover a Multilayer Datacenter Switch (MDS) from Boot Loader Prompt and Boot Mode.

Prerequisites

Requirements

Cisco recommends that you have a knowledge of these topics:

- MDS9000 CLI configuration commands

- How to set up a console connection to MDS switch.

- How to set up or connect MDS switch to a network via its management port.

- Familiarity with TFTP.

Components Used

Information in this document is based on the software requirement release 6.2(9) and later. This document is using an MDS9148S switch, switchname tempest that supports 12 to 48 line-rate 16 Gbps Fibre Channel ports.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Background Information

In order to function correctly, MDS switches require a valid non-corrupted Kickstart and System image to boot up completely. Invalid, corrupted, or missing Kickstart images prevent a switch to boot up and stops at the Boot Loader Prompt. A switch with a valid Kickstart image, but an invalid or missing System Image, ends up in Boot Mode.

Switch Boot Up Sequence

BIOS > Loader > Kickstart Image > System Image

1. The Basic Input Output System (BIOS) loads the Loader.

2. The Loader loads the Kickstart Image into RAM and starts the Kickstart Image.

3. The Kickstart Image loads and starts the System Image.

4. The System Image reads the Startup Configuration file.

Switch Prompt

Phase

Prompt

Description

BIOS

Non Bootable

BIOS begins the power on self test and memory test.

Loader

loader>

Boot loader uncompresses loaded software to boot a kickstart image.

Ctrl + C (Control Key and C Key) to enter boot loader prompt.

Kickstart

switch(boot)#

When the boot loader phase is over, press Ctrl + ] (Control key and Right Bracket key) to enter the boot mode.

System

log in:

The system image loads the startup configuration file and returns a switch to log in prompt.

Recovery Procedure

From switch console session.

loader>

Step 1. Download the switch’s Kickstart and System images, and put them into the TFTP server directory.

In this example, TFTP server IP address is 10.104.204.100.

Step 2. Configure the switch management IP address, subnet mask, and its default gateway.

Check config options in loader prompt using help option

loader> help

help    - print online help

boot     - Boot image

cmdline - <arguments>

dir     -   List Directory

help    - print online help

network -   Set network communication settings

reload  - Reboot the board.

serial  - Set serial console settings

show    - Show loader configuration

loader> net --ip=10.66.78.80 --nm=255.255.255.128 --gw=10.66.78.4

In this example:

- The switch interface mgmt0 IP address is 10.66.78.80  255.255.255.128

- The switch default gateway IP address is 10.66.78.4

OR

The IP assignment synatx in loader prompt varies slightly for switch HW models.

Example listed here in case of T&V-series Fabric switches.

loader> set ip 10.66.78.80 255.255.255.128

loader> set gw 10.66.78.4

Note: Step 2 config required if you are booting the switch for the first time (for the RMA switch) or the switch has been write erase boot.

Step 3. Boot the Kickstart image which is located in the TFTP server directory, in order to bring the switch to the boot mode.

loader> boot tftp://10.104.204.100/m9100-s5ek9-kickstart-mz.8.3.1.bin

eTSEC2: Speed 1000, full duplex

Using eTSEC2 device

TFTP from server 10.104.204.100; our IP address is 10.66.78.80; sending through gateway 10.66.78.4

Filename 'm9100-s5ek9-kickstart-mz.8.3.1.bin'.

Load address: 0x8000000

Loading:         *#################################################################

#################################################################

#################################################################

#################################################################

#################################################################

#################################################################

#################################################################

#################################################################

#################################################################

#################################################################

#################################################################

#################################################################

#################################################################

#################################################################

#################################################################

#################################################################

#######

done

Bytes transferred = 26027008 (18d2400 hex)

NBI at 08000000 size 26027008

Booting image at addr 0x00800000 ...

Memory <- <0x0 0x0 0x1 0x0> (4096MB)

ethernet0: local-mac-address <- 10:05:ca:b6:c5:a0

ethernet1: local-mac-address <- 00:e0:0c:00:01:fd

ethernet2: local-mac-address <- 00:e0:0c:00:02:fd

CPU clock-frequency <- 0x3f941f80 (1067MHz)

CPU timebase-frequency <- 0x3f941f8 (67MHz)

CPU bus-frequency <- 0x1fca0fc0 (533MHz)

zImage starting: loaded at 0x00800000 (sp: 0x7fedc620)

Allocating 0x620d88 bytes for kernel ...

gunzipping (0x00000000 <- 0x00817000:0x00de3838)...done 0x5bc060 bytes

Using loader supplied ramdisk at 0x2800000-0x3de8e00

initrd head: 0x1f8b0808

Linux/PowerPC load: rw root=/dev/ram0 rdbase=0x7000000 card_index=9046 maxcpus=2 ip=off ramdisk_size=262144 noquiet obfl_type_ide=1 kgdboc=ttyS0,9600,B console=ttyS0,9600n8nn loader_ver="02.01.17" card_index=9046 quiet bootdev=mgmt0 server_ip=64.104.204.100 ksimg=m9100-s5ek9-kickstart-mz.8.3.1.bin isanimg=  Finalizing device tree... flat tree at 0xdf0140

Ísetup_arch: bootmem

mpc85xx_ds_setup_arch()

arch: exit

[    0.060378] Host controller irq 26

[    0.103390] Assign root port irq 26

[    0.144046] Host controller irq 24

[    0.193691] Assign root port irq 24

[    0.810630] physmap-flash physmap-flash.0: Could not reserve memory region

[    1.084751] Enabling all PCI devices

INIT: Checking all filesystems..r.r.r.retval=[1]

r done

WARNING: Image sync is going to be disabled after a loader netboot.

Loading system software

No system image is specified

/etc/init.d/functions: li

INIT: Sending processes the KILL signal

Cisco Nexus Operating System (NX-OS) Software

TAC support: http://www.cisco.com/tac

Copyright (c) 2002-2018, Cisco Systems, Inc. All rights reserved.

The copyrights to certain works contained in this software are

owned by other third parties and used and distributed under

license. Certain components of this software are licensed under

the GNU General Public License (GPL) version 2.0 or the GNU

Lesser General Public License (LGPL) Version 2.1. A copy of each

such license is available at

http://www.opensource.org/licenses/gpl-2.0.php and

http://www.opensource.org/licenses/lgpl-2.1.php

switch(boot)#

Step 4. Recover the switch from the boot mode.

1. Check IP config in switch boot prompt.

switch(boot)# sh ip

ip routing is disabled

Management Interface: ip address 10.66.78.80/25

ip default-gateway 10.66.78.4

If IP Config not found, then configure the interface management IP address and default gateway IP address again as shown.

switch(boot)# config t

Enter configuration commands, one per line. End with CNTL/Z.

switch(boot)(config)# interface mgmt 0

switch(boot)(config-if)# ip address 10.66.78.80 255.255.255.128

switch(boot)(config-if)# no shut

switch(boot)(config-if)# exit

switch(boot)(config)# ip default-gateway 10.66.78.4

switch(boot)(config)# exit

2. Test IP connectivity between TFTP server and the switch (ping response is not available in loader prompt).

Ping the switch management IP address from the TFTP server.

C:\Users\mpandika> ping 10.66.78.80

Pinging 10.66.78.80 with 32 bytes of data:

Reply from 10.66.78.80: bytes=32 time=2ms TTL=60

Reply from 10.66.78.80: bytes=32 time=3ms TTL=60

Reply from 10.66.78.80: bytes=32 time=3ms TTL=60

Reply from 10.66.78.80: bytes=32 time=3ms TTL=60

Ping statistics for 10.66.78.80:

Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

Approximate round trip times in milli-seconds:

Minimum = 2ms, Maximum = 3ms, Average = 2ms

C:\Users\mpandika>

3. Download the Kickstart and System images from the TFTP server to the switch’s bootflash.

switch(boot)# copy tftp://10.104.204.100/m9100-s5ek9-kickstart-mz.8.3.1.bin bootflash:

Trying to connect to tftp server......

Connection to server Established. Copying Started.....

TFTP get operation was successful

Copy complete, now saving to disk (please wait)...

switch(boot)#

switch(boot)#copy tftp://10.104.204.100/m9100-s5ek9-mz.8.3.1.bin bootflash:

Trying to connect to tftp server......

Connection to server Established. Copying Started.....

TFTP get operation was successful

Copy complete, now saving to disk (please wait)...

switch(boot)#

4. Verify the Kickstart and System images are successfully downloaded to the switch’s bootflash.

switch(boot)# dir bootflash:

4096   Dec 22 2002 01:38:31  .partner/

297  Nov 09 2017 07:18:54  MDS201711091818530370.lic

297  Nov 09 2017 07:28:42  MDS201711091828418770.lic

4096  Jul 23 2018 14:44:25  lost+found/

26027008  Aug 08 2018 02:55:31  m9100-s5ek9-kickstart-mz.8.3.1.bin

109635731  Aug 08 2018 02:53:38  m9100-s5ek9-mz.8.3.1.bin

3430222  Feb 01 2000 02:31:30  moonlake.1.f.xsvf

23467  Jul 30 2018 12:40:56  mts.log

347607  Jul 28 2017 04:46:31  portchannel

4096  Dec 22 2002 02:38:54  scripts/

31812  Jun 30 2018 05:07:27  snmp.pcap

Usage for bootflash: filesystem

1264746496 bytes used

2106781696 bytes free

3371528192 bytes total

switch(boot)#

5. Load the System image.

switch(boot)# load bootflash:m9100-s5ek9-mz.8.3.1.bin

Uncompressing system image: bootflash:/m9100-s5ek9-mz.8.3.1.bin

CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC

Load plugins that defined in image conf: /isan/plugin_img/img.conf

No Patching support on this platform

Loading plugin 0: core_plugin...

No Patching support on this platform

Enter pboot_chk_compatibility

num srgs 1

0: swid-core-s5ek9m, swid-core-s5ek9m

num srgs 1

0: swid-sup-mini-ks, swid-sup-mini-ks

INIT: Switching to runlevel: 3

INIT: Sending processes the TERM signal

switch(boot)#

INIT:

2018 Aug  8 03:13:03  %SYSLOG-2-SYSTEM_MSG : Syslogs wont be logged into logflash until logflash is online

2018 Aug  8 03:13:06  %KERN-0-SYSTEM_MSG: [    1.084751] Enabling all PCI devices - kernel

2018 Aug  8 03:14:13 switch %CARDCLIENT-2-REG: OK

System is coming up ... Please wait ...

System is coming up ... Please wait ...

2018 Aug  8 03:14:22 switch %ASCII-CFG-2-CONFIG_REPLAY_STATUS: Bootstrap Replay Started.

System is coming up ... Please wait ...

User Access Verification

10.66.78.80 login: 2018 Aug  8 03:14:41 switch %ASCII-CFG-2-CONFIG_REPLAY_STATUS: Bootstrap Replay Done.

2018 Aug  8 03:14:44 switch %PMON-SLOT1-2-PMON_CRIT_INFO: Port Monitor Critical Information: Config download success .

2018 Aug  8 03:14:45 switch %ASCII-CFG-2-CONFIG_REPLAY_STATUS: Ascii Replay Started.

2018 Aug  8 03:14:45 switch %PMON-SLOT1-2-PMON_CRIT_INFO: Port Monitor Critical Information: Policy (slowdrain) activation is successfull .

2018 Aug  8 03:14:46 switch %PLATFORM-2-PS_OK: Power supply 1 ok(Serial number QCS18242L8D)

2018 Aug  8 03:14:46 switch %PLATFORM-2-PS_FANOK: Fan in Power supply 1 ok

2018 Aug  8 03:14:46 switch %PLATFORM-2-PS_FAIL: Power supply 2 failed or shut down(Serial number QCS18242L0H)

2018 Aug  8 03:14:46 switch %PLATFORM-2-FAN_OK: Fan module ok

2018 Aug  8 03:14:46 switch %PLATFORM-2-FAN_OK: Fan module ok

2018 Aug  8 03:14:46 switch %PLATFORM-2-FAN_OK: Fan module ok

2018 Aug  8 03:14:46 switch %PLATFORM-2-FAN_OK: Fan module ok

2018 Aug  8 14:15:35 tempest %ASCII-CFG-2-CONFIG_REPLAY_STATUS: Ascii Replay Done.

User Access Verification

tempest login:

Note: Switch was previously configured with switchname tempest.

6. Log in to the switch.

tempest login: admin

Password:

Last login: Wed Aug  8 14:20:01 on pts/0

Cisco Nexus Operating System (NX-OS) Software

TAC support: http://www.cisco.com/tac

Copyright (c) 2002-2018, Cisco Systems, Inc. All rights reserved.

The copyrights to certain works contained in this software are

owned by other third parties and used and distributed under

license. Certain components of this software are licensed under

the GNU General Public License (GPL) version 2.0 or the GNU

Lesser General Public License (LGPL) Version 2.1. A copy of each

such license is available at

http://www.opensource.org/licenses/gpl-2.0.php and

http://www.opensource.org/licenses/lgpl-2.1.php

tempest#

7. Verify the boot variables.

tempest# show boot

Current Boot Variables:

kickstart variable = bootflash:/m9100-s5ek9-kickstart-mz.8.1.1.bin

system variable = bootflash:/m9100-s5ek9-mz.8.1.1.bin

Boot POAP Disabled

No module boot variable set

Boot Variables on next reload:

kickstart variable = bootflash:/m9100-s5ek9-kickstart-mz.8.1.1.bin

system variable = bootflash:/m9100-s5ek9-mz.8.1.1.bin

Boot POAP Disabled

No module boot variable set

tempest#

8. Set the boot variables to the correct image.

tempest# show version

Cisco Nexus Operating System (NX-OS) Software

TAC support: http://www.cisco.com/tac

Documents: http://www.cisco.com/en/US/products/ps9372/tsd_products_support_series_home.html

Copyright (c) 2002-2018, Cisco Systems, Inc. All rights reserved.

The copyrights to certain works contained herein are owned by

other third parties and are used and distributed under license.

Some parts of this software are covered under the GNU Public

License. A copy of the license is available at

http://www.gnu.org/licenses/gpl.html.

Software

BIOS:      version 2.1.17

loader:    version N/A

kickstart: version 8.3(1)

system:    version 8.3(1)

BIOS compile time:       01/08/14

system image file is:    bootflash:///m9100-s5ek9-mz.8.3.1.bin

system compile time:     7/30/2018 12:00:00 [07/12/2018 19:32:46]

Hardware

cisco MDS 9148S 16G 48 FC (1 Slot) Chassis ("2/4/8/16 Gbps FC/Supervisor")

Motorola, e500v2 with 4088556 kB of memory.

Processor Board ID JAE18250A8C

Device name: tempest

bootflash:    4018896 kB

Kernel uptime is 0 day(s), 0 hour(s), 47 minute(s), 44 second(s)

Last reset at 362296 usecs after Wed Aug  8 13:02:31 2018

Reason: Reset Requested by CLI command reload

System version: 8.1(1)

Service:

plugin

Core Plugin

tempest# config t

Enter configuration commands, one per line.  End with CNTL/Z.

tempest(config)# boot kickstart bootflash:m9100-s5ek9-kickstart-mz.8.3.1.bin

Performing image verification and compatibility check, please wait....

tempest(config)# boot system bootflash:m9100-s5ek9-mz.8.3.1.bin

Performing image verification and compatibility check, please wait....

tempest(config)#

9. Verify the set Boot Variables.

tempest# show boot

Current Boot Variables:

kickstart variable = bootflash:/m9100-s5ek9-kickstart-mz.8.3.1.bin

system variable = bootflash:/m9100-s5ek9-mz.8.3.1.bin

Boot POAP Disabled

No module boot variable set

Boot Variables on next reload:

kickstart variable = bootflash:/m9100-s5ek9-kickstart-mz.8.1.1.bin

system variable = bootflash:/m9100-s5ek9-mz.8.1.1.bin

Boot POAP Disabled

No module boot variable set

10. Save the switch’s running-config.

tempest# copy running-config startup-config

[########################################] 100%

Copy complete.

tempest#

11. Verify the set Boot Variables post copy running-config startup-config.

tempest# show boot

Current Boot Variables:

kickstart variable = bootflash:/m9100-s5ek9-kickstart-mz.8.3.1.bin

system variable = bootflash:/m9100-s5ek9-mz.8.3.1.bin

Boot POAP Disabled

No module boot variable set

Boot Variables on next reload:

kickstart variable = bootflash:/m9100-s5ek9-kickstart-mz.8.3.1.bin

system variable = bootflash:/m9100-s5ek9-mz.8.3.1.bin

Boot POAP Disabled

No module boot variable set