Cisco MDS 9000 Family Configuration Guide, Release 1.3 (from Release 1.3(1) through Release 1.3(6)) - Managing Modules [Cisco MDS 9000 NX-OS and SAN-OS Software]

Table Of Contents

Managing Modules

About Modules

Supervisor Modules

Switching Modules

Services Modules

Verifying the Status of a Module

Checking the State of a Module

Connecting to a Module

Reloading Modules

Reloading the Switch

Power Cycling Modules

Reloading Switching Modules

Preserving Module Configuration

Purging Module Configuration

Powering Off Switching Modules

Identifying Module LEDs

EPLD Configuration

Upgrading EPLD Images

Displaying EPLD Versions

Default Settings

Managing Modules

This chapter describes how to manage switching and services modules (also known as line cards) and provides information on monitoring module states. This chapter includes the following sections:

•About Modules

•Verifying the Status of a Module

•Checking the State of a Module

•Connecting to a Module

•Reloading Modules

•Preserving Module Configuration

•Purging Module Configuration

•Powering Off Switching Modules

•Identifying Module LEDs

•EPLD Configuration

•Default Settings

About Modules

Table 7-1 describes the supervisor module options for switches in the Cisco MDS 9000 Family.

Table 7-1 Supervisor Module Options

Product

No. of Supervisor Modules

Supervisor Module Slot No.

Switching Module Features

Cisco MDS 9216

One module (includes 16 Fibre Channel ports)

1

2-slot chassis allows one optional switching module in the other slot.

Cisco MDS 9509

Two modules

5 and 6

9-slot chassis allows any switching module in the other seven slots.

Cisco MDS 9506

Two modules

5 and 6

6-slot chassis allows any switching module in the other four slots.

Supervisor Modules

Supervisor modules are automatically powered up and started with the switch.

•Cisco MDS 9200 Series switches have one supervisor module that includes an integrated 16-port switching module.

•Cisco MDS 9216i Switches have one supervisor module that includes an intergrated switching module with 14 Fibre Channel ports and 2 Gigabit Ethernet ports.

•Cisco MDS 9500 Series switches have two supervisor modules—one in slot 5 (sup-1) and one in slot 6 (sup-2). See Table 7-2. When the switch powers up and both supervisor modules come up together, the active module is the one that comes up first. The standby module constantly monitors the active module. If the active module fails, the standby module takes over without any impact to user traffic.
Table 7-2 Supervisor Module Terms and Usage
Module Terms

Fixed or Relative

Usage
module-5 and module-6
fixed usage

module-5 always refers to the supervisor module in slot 5 and
module-6 always refers to the supervisor module in slot 6.
sup-1 and sup-2
fixed usage

sup-1 always refers to the supervisor module in slot 5 and
sup-2 always refers to the supervisor module in slot 6.
sup-active and sup-standby
relative usage

sup-active refers to the active supervisor module—relative to the slot that contains the active supervisor module.

sup-standby refers to the standby supervisor module—relative to the slot that contains the standby supervisor module.
sup-local and sup-remote
relative usage

If you are logged into the active supervisor, sup-local refers to the active supervisor module and sup-remote refers to the standby supervisor module.

If you are logged into the standby supervisor, sup-local refers to the standby supervisor module (the one you are logged into.) There is no sup-remote available from the standby supervisor module (you cannot access a filesystem on the active sup).
Switching Modules

Cisco MDS 9000 Family switches support any switching module in any non-supervisor slot. These modules obtain their image from the supervisor module.

Services Modules

Cisco MDS 9000 Family switches support any services module in any non-supervisor slot.

Refer to the Cisco MDS 9000 Family SAN Volume Controller Configuration Guide for more information on CSMs.

Verifying the Status of a Module

Before you begin configuring the switch, you need to ensure that the modules in the chassis are functioning as designed. To verify the status of a module at any time, issue the show module command (see the "Configuring Fibre Channel Interfaces" section). The interfaces in each module are ready to be configured when the ok status is displayed in the show module command output. A sample output of the show module command follows:
switch# show module
Mod  Ports  Module-Type                     Model              Status
---  -----  ------------------------------- ------------------ ------------
2    8      IP Storage Services Module      DS-X9308-SMIP      ok
4    0      Caching Services Module                            ok
5    0      Supervisor/Fabric-1             DS-X9530-SF1-K9    active *
6    0      Supervisor/Fabric-1             DS-X9530-SF1-K9    ha-standby
8    0      Caching Services Module         DS-X9560-SMAP      ok
9    32     1/2 Gbps FC Module              DS-X9032           ok
Mod  Sw           Hw      World-Wide-Name(s) (WWN)
---  -----------  ------  --------------------------------------------------
2    1.3(0.106a)  0.206   20:41:00:05:30:00:00:00 to 20:48:00:05:30:00:00:00
5    1.3(0.106a)  0.602   --
6    1.3(0.106a)) 0.602   -- <--------------- New running version in module 6 
8    1.3(0.106a)  0.702   --
9    1.3(0.106a)  0.3     22:01:00:05:30:00:00:00 to 22:20:00:05:30:00:00:00
Mod  MAC-Address(es)                         Serial-Num
---  --------------------------------------  ----------
2    00-05-30-00-9d-d2 to 00-05-30-00-9d-de  JAB064605a2
5    00-05-30-00-64-be to 00-05-30-00-64-c2
6    00-d0-97-38-b3-f9 to 00-d0-97-38-b3-fd  JAB06350B1R
8    00-05-30-01-37-7a to 00-05-30-01-37-fe  JAB072705ja
9    00-05-30-00-2d-e2 to 00-05-30-00-2d-e6  JAB06280ae9
* this terminal session
The Status column in the output should display an ok status for switching modules and an active or standby (or HA-standby) status for supervisor modules. If the status is either ok or active, you can continue with your configuration.

Note A standby supervisor module reflects the HA-standby status if the HA switchover mechanism is enabled (see the "HA Switchover Characteristics" section). If the warm switchover mechanism is enabled, the standby supervisor module reflects the standby status.

The states through which a switching module progresses is discussed in the "Checking the State of a Module" section.

Checking the State of a Module

If your chassis has more than one switching module (also known as line card), you can check the progress by issuing the show module command several times and viewing the Status column each time.

The switching module goes through a testing and an initializing stage before displaying an ok status. Table 7-3 describes the possible states in which a module can exist.
Table 7-3 Module States
show module Status Output

Description
powered up
The hardware has electrical power. When the hardware is powered up, the software begins booting.
testing
The switching module has established connection with the supervisor supervisor and the switching module is performing bootup diagnostics.
initializing
The diagnostics have completed successfully and the configuration is being downloaded.
failure
The switch detects a switching module failure upon initialization and automatically attempts to power-cycle the module three times. After the third attempt it continues to display a failed state.
ok
The switch is ready to be configured.
power-denied
The switch detects insufficient power for a switching module to power up (see the "Displaying Environment Information" section).
active
This module is the active supervisor module and the switch is ready to be configured.
HA-standby
The HA switchover mechanism is enabled on the standby supervisor module (see the "HA Switchover Characteristics" section).
standby
The warm switchover mechanism is enabled on the standby supervisor module (see the "HA Switchover Characteristics" section).
Connecting to a Module

At any time, you can connect to any module using the attach module command. Once you are at the module prompt, you can obtain further details about the module using module-specific commands in EXEC mode.

To attach to a module, follow these steps:
Command

Purpose
Step 1
switch# attach module 6
switch(standby)# 
Provides direct access to the specified module (in this example, the standby supervisor module is in slot 6).
Step 2
switch(standby)# dir bootflash:
root     14502912  Jan 13 12:23:52 1980 kickstart_image1
admin    14424576  Jan 14 06:47:29 1980 kickstart_image2
admin    14469632  Jan 14 01:29:16 1980 kickstart_image3
root     14490112  Jan 08 07:25:50 1980 kickstart_image4
root        12288  Jan 16 15:49:24 1980 lost+found/
admin    14466048  Jan 14 02:40:16 1980 kickstart_image5
admin    24206675  Jan 14 02:57:03 1980 m9500-sf1ek.bin
root     19084510  Jan 13 12:23:28 1980 system_image1
admin    19066505  Jan 14 06:45:16 1980 system_image2
admin    18960567  Jan 14 01:25:21 1980 system_image5
           Usage for bootflash: filesystem
                  158516224 bytes total used
                     102400 bytes free
                  167255040 bytes available
Provides the available space information for the standby supervisor module.

Note Type exit to exit the module-specific prompt.

Tip If you are not accessing the switch from a console terminal, this is the only way to access the standby supervisor module.
You can also use the attach module command as follows:

•To view the standby supervisor module information, although you cannot configure the standby supervisor module using this command.

•To view the switching module portion of the Cisco MDS 9216 supervisor module which resides in slot 1.

Reloading Modules

You can reload the entire switch, reset specific modules in the switch, or reload the image on specific modules in the switch.

Reloading the Switch

To reload the switch, issue the reload command without any options. When you issue this command, you reboot the switch (see "Software Images").

Note If you need to issue the reload command, be sure to save the running configuration using the copy running-config startup-config command.

Power Cycling Modules

To power cycle any module, follow these steps:

Step 1 Identify the module that needs to be reset.

Step 2 Issue the reload module command to reset the identified module. This command merely power cycles the selected module.
switch# reload module number
Where number indicates the slot in which the identified module resides. For example:
switch# reload module 2
Reloading Switching Modules

Switching modules automatically download their images from the supervisor module and do not need a forced download. This procedure is provided for reference should a need arise.

To replace the image on a switching module, follow these steps:

Step 1 Identify the switching module that requires the new image.

Step 2 Issue the reload module number force-dnld command to update the image on the switching module.
switch# reload module number force-dnld
Where number indicates the slot in which the identified module resides. In this example, the identified module resides in slot 9.
switch# reload module 9 force-dnld...
Jan  1 00:00:46 switch %LC-2-MSG:SLOT9 LOG_LC-2-IMG_DNLD_COMPLETE: COMPLETED
downloading of linecard image. Download successful...
Preserving Module Configuration

Issue the copy running-config startup-config command from EXEC mode to save the new configuration into nonvolatile storage. Once this command is issued, the running and the startup copies of the configuration are identical.

Table 7-4 displays various scenarios when module configurations are preserved or lost.

.

Table 7-4 Switching Module Configuration Status

Scenario

Consequence

A particular switching module is removed and the copy running-config startup-config command is issued again.

The configured module information is lost.

A particular switching module is removed and the same switching module is replaced before the copy running-config startup-config command is issued again.

The configured module information is preserved.

A particular switching module is removed and replaced with the same type switching module, and a reload module number command is issued.

The configured module information is preserved.

A particular switching module is removed and replaced with a different type of switching module. For example, a 16-port switching module is replaced with a 32-port switching module.

Sample scenario:

1. The switch currently has a 16-port switching module and the startup and running configuration files are the same.

2. You replace the 16-port switching module in the switch with a 32-port switching module.

3. Next, you remove the 32-port switching module and replace it with the same 16-port switching module referred to in Step 1.

4. You reload the switch.

The configured module information is lost from the running configuration. The default configuration is applied.

The configured module information remains in startup configuration until a copy running-config startup-config command is issued again.

Sample response:

1. The switch uses the 16-port switching module and the present configuration is saved in nonvolatile storage.

2. The factory default configuration is applied.

3. The factory default configuration is applied.

4. The configuration saved in nonvolatile storage referred to in Step 1 is applied.

Purging Module Configuration

Issue the purge module slot running-config command from EXEC mode to delete the configuration in a specific module. Once this command is issued, the running configuration is cleared for the specified slot. This command does not work on supervisor modules or on any slot that currently has a module. This command only works on an empty slot (where the specified module once resided).

The purge module command clears the configuration for any module that previously existed in a slot and has since been removed. While the module was in that slot, some parts of the configuration may have been stored in the running configuration and cannot be reused (for example, IP addresses), unless it is cleared from the running configuration.

For example, suppose you create an IP storage configuration with an IPS module in slot 3 in Switch A. This module uses IP address 10.1.5.500. You decide to remove this IPS module and move it to Switch B, and you no longer need the IP address10.1.5.500. If you try to configure this unused IP address, you will receive an error message that prevents you from proceeding with the configuration. In this case, you need to issue the purge module 3 running-config command to clear the old configuration in Switch A before proceeding with using this IP address.

Powering Off Switching Modules

By default, all switching modules are configured to be in the power up state.

To power off a module, follow these steps:
Command

Purpose
Step 1
switch# config t
Enters configuration mode.
Step 2
switch(config)# poweroff module 1
switch(config)# 
Powers off the specified module (switching module 1) in the switch.
switch(config)# no poweroff module 1
switch(config)# 
Powers up the specified module (switching module 1) in the switch.
Identifying Module LEDs

Table 7-5 to Table 7-8 describe the LED location, type, and status for supervisor and switching modules used in Cisco MDS 9000 Family switches.

Table 7-5 Module LEDs on a Cisco MDS 9200 Series Switch

Module

LED Type

Status

Description

Fixed switching module

Status

Green

•All chassis environmental monitors (power supply, fan, temperature sensor, clock, and chassis EEPROM) are reporting OK.

•Sufficient power is available for all modules

Orange

•Any one of the chassis environmental monitors (power supply, fan, temperature sensor, clock, and chassis EEPROM) failed.

•Sufficient power is not available for all modules.

•Incompatible power supplies are installed.

•The redundant clock failed.

Red

•The diagnostic test failed.

•The module is not operational because a fault occurred during the initialization sequence.

•A temperature condition occurred. (A major threshold was exceeded during environmental monitoring.)

Optional switching module

System

Green

All diagnostics pass. The module is operational (normal initialization sequence).

Orange

•The module is booting or running diagnostics (normal initialization sequence).

•An over temperature condition occurred. (A minor threshold was exceeded during environmental monitoring.)

Red

•The diagnostic test failed. The module is not operational because a fault occurred during the initialization sequence.

•An over temperature condition occurred. (A major threshold was exceeded during environmental monitoring.)

Table 7-6 Supervisor Module LEDs on a Cisco MDS 9500 Series Switch

LED Type

Status

Description

Status

Green

All diagnostics pass. The module is online.

Orange

•The module is booting or running diagnostics (normal initialization sequence).

•The module is not online.

•An over temperature condition has occurred. (A minor threshold has been exceeded during environmental monitoring.)

Red

•The diagnostic test failed. The module is not operational because a fault occurred during the initialization sequence.

•An over temperature condition has occurred. (A major threshold has been exceeded during environmental monitoring.)

System¹

Green

All chassis environmental monitors (power supply, fan, temperature sensor, clock, and chassis EEPROM) are reporting OK.

Orange

•Any one of the environmental monitors (power supply, fan, temperature sensor, clock, and chassis EEPROM) has failed.

•Incompatible power supplies are installed.

•The redundant clock has failed.

Red

The temperature of the supervisor module major threshold has been exceeded.

Active

Green

The supervisor module is operational and active.

Orange

The supervisor module is in standby mode.

Pwr Mgmt1

Green

Sufficient power is available for all modules.

Orange

Sufficient power is not available for all modules.

¹The System and Pwr Mgmt LEDs on a redundant supervisor module are synchronized to the active supervisor module.

Table 7-7 Ethernet Interface LEDs on a Cisco MDS 9200 Series Switch

Module

LED Type

Status

Description

Ethernet (mgmt 0)

Activity

Flashing green

Traffic is passing through the interface.

Link

Solid green

The link is functioning.

Off

The link is down.

Table 7-8 Switching Module LEDs

LED Type

Status

Description

Status

Green

All diagnostics pass. The module is operational (normal initialization sequence).

Orange

•The module is booting or running diagnostics (normal initialization sequence).

•An over temperature condition occurred. (A minor threshold was exceeded during environmental monitoring.)

Red

•The diagnostic test failed. The module is not operational because a fault occurred during the initialization sequence.

•An over temperature condition occurred. (A major threshold was exceeded during environmental monitoring.)

Speed

On

2 Gbps mode.

Off

1 Gbps mode and beacon mode disabled.

Flashing green

Beacon mode enabled. See the "Identifying the Beacon LEDs" section.

Link

Solid green

Link is up.

Solid yellow

Disabled by software.

Flashing yellow

Fault is detected.

Off

Link is down.

EPLD Configuration

Switches and directors in the Cisco MDS 9000 Family contain several electrical programmable logical devices (EPLDs) that provide hardware functionalities in all modules. Starting with Cisco MDS SAN-OS Release 1.2, EPLD image upgrades are periodically provided to include enhanced hardware functionality or to resolve known issues.

Tip Refer to the Cisco MDS SAN-OS Release Notes to verify if the EPLD has changed for the Cisco SAN-OS image version being used.

EPLDs can be upgraded or downgraded using CLI commands. When EPLDs are being upgraded or downgraded, the following guidelines and observations apply:

•You can individually update each module that is online. The EPLD update is only disruptive to the module being upgraded.

•If you interrupt an upgrade, the module must be upgraded again.

•The upgrade or downgrade can only be executed from the active supervisor module. While the active supervisor module cannot be updated, you can update the other modules individually.

•In Cisco MDS 9100 Series fabric switches, be sure to specify one (1) as the module number.

•Cisco MDS 9216 Switches do not support EPLD upgrades.

Caution Do not insert or remove any modules while an EPLD upgrade or downgrade is in progress.

Upgrading EPLD Images

Issue the install module number epld url command on the active supervisor module to upgrade EPLD images for a module.
switch# install module 2 epld scp://user@10.6.16.22/m9000-epld-version.img
The authenticity of host '10.6.16.22' can't be established.
RSA1 key fingerprint is 55:2e:1f:0b:18:76:24:02:c2:3b:62:dc:9b:6b:7f:b7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.6.16.22' (RSA1) to the list of known hosts.
user@10.6.16.22's password:
epld.img             100% |*****************************|  1269 KB    00:00
Module Number                                      2
EPLD                            Curr Ver     New Ver
----------------------------------------------------
Power Manager								0x06
XBUS IO								0x07        0x08
UD Flow Control								0x05
PCI ASIC I/F								0x05        0x05
Module 2 will be powered down now!!
Do you want to continue (y/n) ? y
\ <------------------------------------------------------------progress twirl
Module 2 EPLD upgrade is successful
If you forcefully upgrade a module that is not online, all EPLDs are forcefully upgraded. If the module is not present in the switch, an error is returned. If the module is present, the command process continues. To update a module that is not online but is present in the chassis, use the same command. The switch software prompts you to continue after reporting the module state. When you confirm your intention to continue, the upgrade continues,
switch# install module 2 epld scp://user@10.6.16.22/m9000-epld-version.img
Module 2 is not online, Do you want to continue (y/n) ? y
cchetty@171.69.16.22's password:
epld.img             100% |*****************************|  1269 KB    00:00
\ <------------------------------------------------------------progress twirl
Module 2 EPLD upgrade is successful
Note Switches in the Cisco MDS 9100 Series do not support a forced EPLD upgrade. When you upgrade the EPLD module on these switches, you receive the following message:
Data traffic on the switch will stop now!!
Do you want to continue (y/n) ?

Displaying EPLD Versions

Use the show version module number epld command to view all current EPLD versions on a specified module (see Example 7-1).

Example 7-1 Displays Current EPLD Versions for a Specified Module
switch# show version module 2 epld
Module Number                      2
EPLD Device                  Version
------------------------------------
Power Manager								0x06
XBUS IO								0x07
UD Flow Control									0x05
PCI ASIC I/F									0x05
Use the show version epld url command to view the available EPLD versions (see Example 7-2).

Example 7-2 Displays Available EPLD Versions
switch# show version epld scp://user@10.6.16.22/m9000-epld-version.img
user@10.6.16.22's password:
Module Name                     EPLD Device         Version
-----------------------------------------------------------
MDS 9500 Supervisor 1								XBUS 1 IO					0x09
								XBUS 2 IO					0x0c
								UD Flow Control					0x05
								PCI ASIC I/F					0x04
1/2 Gbps FC Module (16 port)								XBUS IO					0x08
								PCI ASIC I/F					0x05
1/2 Gbps FC Module (32 port)								XBUS IO					0x07
								PCI ASIC I/F					0x05
Advanced Services Module								XBUS IO					0x07
								UD Flow Control					0x05
								PCI Bridge					0x04
IP Storage Services Module								XBUS IO					0x02
								UD Flow Control					0x05
								PCI ASIC I/F					0x05
								Services MOdule I/F					0x12
								IPS DB I/F					0x08
MDS 9100 Series Fabric Switch								XBUS IO					0x03
Default Settings

Table 7-9 lists the default settings for the supervisor module.

Table 7-9 Default Supervisor Module Settings

Parameters

Default

Administrative connection

Serial connection.

Global switch information

•No value for system name.

•No value for system contact.

•No value for location.

System clock

No value for system clock time.

In-band (VSAN 1) interface

IP address, subnet mask, and broadcast address assigned to the VSAN are set to 0.0.0.0.