This chapter describes the Cisco SFS 7008P switch features and contains these sections:
•Cisco SFS 7008P Chassis
Cisco SFS 7008P Chassis
The Cisco SFS 7008P features are described in this section. See Figure 1-1 for the front panel of the Cisco SFS 7008P switch (bezel removed). See Figure 1-2 for the rear panel of the Cisco SFS 7008P switch.
Figure 1-1 Cisco SFS 7008P Switch—Front View without Bezel
Figure 1-2 Cisco SFS 7008P Switch—Rear View
Chassis Slot Numbering
Slot numbers are printed on the chassis for easy identification. The Cisco SFS 7008P chassis slot numbers are from top to bottom. See Figure 1-1 and Figure 1-3 for the location of the slot numbers on the chassis.
Figure 1-3 Cisco SFS 7008P Switch Slot Numbers—Rear View
The Cisco SFS 7008P chassis dimensions are as follows:
•Height: 6 RU
•Width: 17.04 inches (43.3 cm)
•Depth: 23.7 inches (60.2 cm)
•Weight: maximum 106 pounds (48.1 kg) (full chassis configuration with 4x copper Line Interface Modules)
The Cisco SFS 7008P chassis is designed to be installed in standard, 19-inch (48.26-cm) equipment racks that meet ANSI/EIA 310-D, EIC 60297, and ETS 300-119 standards.
•96 ports of 10 Gbps 4x Copper InfiniBand.
•Two 10/100 Ethernet RJ-45 Management-Ethernet ports for out-of-band management.
•Two RJ-45 console ports used to configure and monitor the Cisco SFS 7008P switch.
Note All cabling is on the rear of the chassis.
LEDs are located on both the front and back of the chassis. See the "Using the Front Panel LEDs" section on page 5-1 and "Using the Back Panel LEDs" section on page 5-9 for more information.
The Cisco SFS 7008P chassis has two AC-DC bulk power supply modules. The characteristics of the power supplies are as follows:
•Only one power supply, in either of the two slots, is required to power the system. However, a filler plate is required for any unused bay.
•The power supplies are 48V, 1200W
•The power supplies are 1 RU in height
•The power supplies provide regulated +48V DC to all other modules in the system.
•Each power supply has self contained fans for cooling.
Note See the "Installing a New Power Supply Module" section on page 3-6 for installation instructions.
The fan trays are redundant, hot-swappable cooling units. The characteristics of the fan trays are as follows:
•Only one fan tray, in either of the two slots, is required to cool the system. However, a filler plate is required for any unused bay.
•The replacement of any one fan tray does not disrupt the operation of the device in any way and can be successfully completed without removing the device from a rack or disconnecting any cables.
•The switch software will set the fan speed according to the room temperature.
•The fan trays use DC voltage regulators to control fan speed and contain a hardware fan control circuit, which controls and monitors the fan speed.
•The fan tray defaults to full speed after a power-on or reset occurs.
•If you have two fans and one fan fails, the redundant fan goes to full speed.
The fan trays do not have guards. Do not remove a fan tray completely from the bay until the blades have stopped moving (approximately 15 seconds).
See the "Installing a Fan Module" section on page 3-10 for installation instructions.
Management Interface Modules
The management interface module front panel is shown in Figure 1-4.
Figure 1-4 Management Interface Module
See the "Installing a Management Interface Module" section on page 3-24 for installation instructions.
The Cisco SFS 7008P switch supports redundant, hot-swappable management interface modules. Each module is paired to one of the fabric controller core modules.
The management interface modules provide the following features:
•RS-232 serial console port.
•10/100 Ethernet management port.
•System real time clock (RTC).
•Intra-system Ethernet switch, which provides a system configuration management data path to all fabric controllers.
•LED indicators on the front side of the chassis for the power supply, fans, and over-all system status. (See the "Management Interface Module LEDs" section on page 5-10).
Understanding the Core Slot to Management Interface Module Connection
Each core slot in the chassis is connected to one port on the management interface module. Controllers in core slots use a management interface module to connect to other fabric modules and to communicate with the outside network through Ethernet or serial ports. The controller (a fabric interface module inserted in the core slot) selects the management interface module to which it will connect.
The pairing of management interface module to core slot is as follows:
•The management interface module in slot 15 is connected to the fabric controller in the core slot 11.
•The management interface module in slot 16 is connected to the fabric controller in the core slot 12.
For example, removing the fabric controller from the core slot 11 would cause both the fabric controller card and its paired management interface module in slot 15 to fail over. The new primary controller would be in slot 12 and the primary management interface would be in slot 16.
See the "Installing a Fabric Controller" section on page 3-15 for fabric controller installation instructions.
The Cisco SFS 7008P switch contains six fabric control modules.
The fabric controllers perform the following roles within the system:
•Acts as a system controller
•Acts as a switch fabric spine
•Acts as switch fabric end nodes
•Stores redundant vital product data (VPD) from the chassis ID module to allow the chassis to continue functioning (and the VPD information to be recovered) if a chassis ID module fails.
Each fabric controller runs the management software and contains nonvolatile storage.
The behavior and responsibility of each fabric controller is determined by the type of slot into which it is inserted. Each fabric controller can be installed in either a node slot or a core slot (see "Node and Core Slots" on page 9).
Master/Slave and Standby Roles
When the software on a fabric controller module detects that a module is inserted in a core slot, it arbitrates for system mastership and runs.
•One fabric controller in the core slot acts as the master
•One fabric controller in the core slot acts as the standby to the master
•Fabric controllers in node slots act as slaves
Note The fabric controllers in the node slots do not operate in an active/standby configuration.
Node and Core Slots
The fabric controllers can occupy either a node slot or a core slot in the chassis.
The Cisco SFS 7008P switch has four node slots. The fabric controllers installed in the node slots provide fabric leaf switching, not controller functionality.
The Cisco SFS 7008P switch has two core slots. The chassis must have at least one fabric module in a core slot paired with a management interface module to function correctly.
The subnet manager runs on the fabric controller installed in the core slot that has arbitrated to be the master. The core slot module also acts as the switch fabric spine, interconnecting the end-nodes. Each core slot is connected to one or two ports on the management interface modules.
Line Interface Modules
See the "Installing a LIM Module" section on page 3-21 for installation instructions.
The Cisco SFS 7008P switch has hot-swappable Line Interface Modules (LIMs), which are installed in the rear of the chassis. The LIMs provide the physical InfiniBand port interfaces and are 4x over copper. The number of LIMs that you use in your Cisco SFS 7008P switch depends on your particular configuration.
Chassis ID Module
The chassis ID module (see Figure 1-5) is hot-swappable and is located on the rear of the chassis (see Figure 1-2).
Figure 1-5 Chassis ID Module
Upon initial boot-up of the Cisco SFS 7008P switch, the Chassis ID is read by the system, and the unique information is populated to the rest of the chassis.
The Chassis ID module contains model and serial number information and logs certain error information. See the "Vital Product Data Storage" section.
This section describes the system architecture for the Cisco SFS 7008P switch and contains these sections:
•Hot Standby Capability
Hot Standby Capability
On the Cisco SFS 7008P switch, the inactive controller card is always in the hot-standby state, which means that if the active controller card fails, then the card in the hot-standby state can assume control. With the hot-standby feature, you do not have to reboot the switch or reset any cards in the chassis if a switch failover occurs.
Note You can access the hot standby controller card with a connection to the serial console port. A CLI session to the standby controller provides read-only access to the controller. You can view but not configure the card.
Because the hot standby feature keeps both controller cards up, you can enter the show card command to identify the card on which your CLI session runs. In the command output, an asterisk identifies the controller card to which you are connected. In the example that follows, the CLI session applies to the controller card in slot 11 (the active card), as identified by the asterisk next to the slot number (line 11).
admin oper admin oper oper
slot type type status status code
11* controllerFabric12x controllerFabric12x up up normal
12 controllerFabric12x controllerFabric12x up up standby
The oper code field of the command output identifies the controller card as the active card (normal) or the hot standby card (standby). By default, when you power on a Cisco SFS 7008P switch, the card in slot 11 becomes the active card and the card in slot 12 becomes the standby card. The master and standby controllers automatically synchronize state and configuration information.
The Cisco SFS 7008P switch is an enterprise-class managed 960 Gbps, 6U InfiniBand (IB) switch that is capable of supporting up to 96 4x (4-channel) ports or 32 12x (12-channel) ports.
No failure on any single InfiniBand port will result in interruptions in service for any of the other ports.
For redundancy, InfiniBand HCAs can be dual-connected to a redundant pair of Cisco SFS 7008P switches.
In an InfiniBand fabric that includes more than one Cisco SFS 7008P switch: if the subnet manager on the Cisco SFS 7008P that is acting as the master fails, another subnet manager will take over within seconds. All necessary state information is kept in a synchronized state.
The architecture of the Cisco SFS 7008P switch offers flexibility in that the switch fabric cards are generic and will assume the role of fabric controller or fabric node depending on the slot into which they are installed. Because the switch fabric card can be either a fabric controller or fabric node, you need to stock only one type of fabric card.
The Cisco SFS 7008P switch features the following hot-swappable and redundant components:
•Power supply modules
•Line interface modules (LIMs) (redundant connection from the host channel adapters (HCA))
•Fabric controllers (in redundant core or node slots)
•Management interface module
The Cisco SFS 7008P switch also features a hot-swappable chassis ID module.
The Cisco SFS 7008P switch provides a nonblocking switch element architecture with full bidirectional bandwidth for the switch chassis.
The Cisco SFS 7008P switch has port-to-port latency of less than 600 ns.
This section describes the administrative features of the Cisco SFS 7008P switch:
•Vital Product Data Storage
The Cisco SFS 7008P switch maintains the correct time regardless of power conditions or connectivity.
The memory supports the following:
•Up to three stored system images (not including the recovery image)
•Up to one week of log files at normal verbosity
•Up to one day of log files at maximum verbosity
Vital Product Data Storage
Vital product data (VPD) storage is stored in nonvolatile memory in the chassis ID module (see the "Chassis ID Module" section) and the fabric controller that is installed in the core slot (see the "Fabric Controllers" section). The VPD is available electronically. Some VPD information can be accessed via CLI, the Java GUI, or the Web GUI.
The following VPD can be recovered by returning a field replaceable unit (FRU) to the original manufacturer:
•Manufacturing part number
•Final test date
•OEM part number
The Cisco SFS 7008P switch runs the following tests to determine the operational status:
•The power-on self-test (POST) is performed on all system components during power-on to determine operational readiness.
•Redundant components' operational status is ensured periodically during normal operation, including the logic required to perform the transition from the faulted/primary to the redundant component. This test can detect if the redundant components are showing an abnormal status.
See the "Diagnostic Tests" section on page 6-1 for more detailed information.