Cisco UCS 6200 Series Fabric Interconnect Hardware Installation Guide
Product Overview
Downloads: This chapterpdf (PDF - 1.92MB) The complete bookPDF (PDF - 8.94MB) | The complete bookePub (ePub - 6.59MB) | The complete bookMobi (Mobi - 12.75MB) | Feedback

Product Overview

Product Overview

This chapter includes the following sections:

Interconnect Features

A Cisco UCS 6200 series fabric interconnect is a top-of-rack fabric interconnect that provides Ethernet and Fibre Channel to all servers in the UCS system. Servers connect to the fabric interconnect, and it connects to the LAN or SAN.

This family of fabric interconnects connect UCS servers to 10 Gigabit Ethernet 1, 2, 4, and 8 Gbps Fibre Channel networks, and provides consolidated I/O connectivity to both production Ethernet LANs and Fibre Channel SANs in a cost-effective, high-performance, low-latency environment.

The Cisco UCS 6200 series has the following characteristics:

  • The UCS 6248 UP fabric interconnect is a one-rack-unit (1 RU), 10-Gigabit Ethernet and FCoE device that offers up to 960-Gbps throughput and up to 48 ports. It has 32 1- or 10-Gbps fixed small form-factor pluggable plus (SFP+) ports and one expansion slot. The Cisco UCS 6248 UP has 32 ports on the base system and can be upgraded with one expansion module providing an additional 16 ports.
  • The UCS 6296 UP fabric interconnect is a two-rack-unit (2 RU), 10-Gigabit Ethernet and FCoE device that offers up to 1920-Gbps throughput and up to 96 ports. It has 48 1- or 10-Gbps fixed SFP+ ports and three expansion slots. The Cisco UCS 6248 UP has 48 ports on the base system and can be upgraded with three expansion modules providing an additional 48 ports.
  • Universal port functionality allows Ethernet or Fibre Channel over Ethernet (FCoE) physical port types to be selected in software.
  • Slots on the back of the switch for an expansion module. You can use the UCS EIGUP module.
  • Slots on the front of the chassis for hot swap-capable power supplies.
  • Slots on the front of the chassis for fan modules.
  • One USB port at the front of the switch.

Cisco UCS 6248 UP Fabric Interconnect Chassis

The Cisco UCS 6248 UP chassis is 1 RU, 1.72 inches tall, 17.3 inches wide and 30.0 inches deep. It mounts in a standard 19-inch rack (the Cisco R Series rack is an ideal choice). The chassis has two power supplies and two fan modules on the front of the chassis, and it has network ports on the rear of the chassis and has one USB port (usb1:) at the front. This external USB flash memory is installed in a supervisor module used for storing image files, configuration files, and other miscellaneous files. You can create directories on external flash memory and navigate through these directories. You can also create and access files. The usb1: port usage on the Cisco UCS 6248 is the same as that on Cisco NX-OS devices.

Thirty-two fixed 10-Gigabit Ethernet ports and an expansion modules slot supporting 16 ports are at the rear of the switch. The airflow is front to back.

Figure 1. Cisco UCS 6248 UP Front View

1

Management (10/100/1000) port, console port, and USB port

3

Two power supplies (AC shown)

2

Two fan modules

   

The rear of the Cisco UCS 6248 UP chassis has 32 fixed 10-Gigabit, Fiber Channel over Ethernet-capable SFP+ Ethernet ports and 1 slot for an optional expansion module.

Figure 2. Cisco UCS 6248 UP Rear View

1

32 fixed 10-Gigabit Ethernet ports

2

16 Expansion module ports

Figure 3. Cisco UCS 6248 UP Switch Rear View Close-up

1

System status LED and beaconing LED/button

3

16 port Expansion module

2

32 fixed 10-Gigabit Ethernet ports

   

The front connector ports are in a 2x2 stacked RJ-45 jack.

Figure 4. Connector Ports and LED Closeup

1

Beaconing LED/button

5

Network management port

2

System Status LED

6

Console port

3

UCS cross connect port L1

7

USB port

4

UCS cross connect port L2

   
Table 1 Ethernet LED Descriptions

LED

Status

Description

Left

Off

No link

Solid green

Physical link

Right

Off

No activity

Blinking green

Activity

Cisco UCS 6296 UP Fabric Interconnect Chassis

The Cisco UCS 6296 UP chassis is 2 RU or 3.47 inches (8.8 cm) tall, 17.3 inches (43.9 cm) wide, and 29.5 inches (74.9 cm) deep. It is designed to be mounted in a standard 19-inch wide rack. The front of the switch has a USB port, four Ethernet and ports (two cross-connect ports, one management port, and one console port), two power supplies, and four fan modules.

Figure 5. Front View of the Cisco UCS 6296 UP

1

USB port

4

System status LED

2

Management and console ports (two RJ-45 Ethernet connector ports on the left, a RJ-45 network management connector on the upper right, and a console connector on the lower right)

5

Two power supplies

3

Identifier LED

6

Four fan modules

The management and console ports are in a 2 x 2 stacked RJ-45 jack.

Figure 6. Management and Console Ports Closeup

1

Beaconing LED/button

5

Network management port

2

System Status LED

6

Console port

3

UCS cross connect port L1

7

USB port

4

UCS cross connect port L2

   

The rear of the Cisco UCS 6296 UP chassis has 48 fixed 10-Gigabit Ethernet data ports on the bottom and three slots for optional expansion modules on top.

Figure 7. Rear View of the Cisco UCS 6296 UP

1

Expansion modules, shown here with three 16-port Universal GEM2 modules

3

System status LED

2

Identifier LED

4

48 fixed 1- and 10-Gigabit Ethernet ports

UCS Unified Port Expansion Module

Expansion modules allow Cisco UCS 6200 Series Fabric Interconnect to be configured as cost-effective 10-Gigabit Ethernet fabric interconnects and as I/O consolidation platforms with native Fibre Channel connectivity. The Cisco UCS 6248 UP has one slot for an optional expansion module. The Cisco UCS 6296 has three slots for an optional expansion module. Only the 16-port UCS E16UP hot-pluggable expansion module using the small form factor pluggable plus (SFP+) interface is supported.

Figure 8. UCS E16UP GEM

Cisco UCS 6248 UP Port Numbering

Each individual port is numbered, and groups of ports are numbered based on their function. The ports are numbered top to bottom and left to right.

There are 32 to 48 ports on the Cisco UCS 6248 UP, depending on whether an expansion module is installed. Each individual port is numbered, and groups of ports are numbered based on their function. The ports are numbered from top to bottom and left to right.

Figure 9. Port Numbering of the Cisco UCS 6248UP with an Expansion Module

1

Slot 1, 32 fixed ports

2

Slot 2, 16 port Expansion module (UCS E16UP)

Cisco UCS 6296 UP Port Numbering

Each port on the Cisco UCS 6296 UP is numbered, and groups of ports are numbered based on their function. The ports are numbered top to bottom and left to right. The 48 fixed ports support 8-, 4-, 2-, or 1-Gbps Fibre Channel transceivers and 1- or 10-Gigabit Ethernet transceivers.

Figure 10. Port Numbering of Fixed Ports and Universal Port Expansion Modules

0

Port numbering for the 48 fixed ports in slot 0

2

Port numbering for the 16-port GEM in slot 2

1

Port numbering for the 16-port GEM in slot 1

3

Port numbering for the 16-port GEM in slot 3

Cabling Considerations for Fabric Port Channels

When you configure the links between the Cisco UCS 2200 Series FEX and a Cisco UCS 6200 series fabric interconnect in fabric port channel mode, the available VIF namespace on the adapter varies depending on where the FEX uplinks are connected to the fabric interconnect ports.

Inside the 6248 fabric interconnect there are six sets of eight contiguous ports, with each set of ports managed by a single chip. When uplinks are connected such that all of the uplinks from an FEX are connected to a set of ports managed by a single chip, Cisco UCS Manager maximizes the number of VIFs used in service profiles deployed on the blades in the chassis. If uplink connections from an IOM are distributed across ports managed by separate chips, the VIF count is decreased.

Figure 11. Port Groups for Fabric Port Channels


Caution


Adding a second link to a fabric port channel port group is disruptive and will automatically increase the available amount of VIF namespace from 63 to 118. Adding further links is not disruptive and the VIF namespace stays at 118.



Caution


Linking a chassis to two fabric port channel port groups is disruptive and does not affect the VIF namespace unless it is manually acknowledged. The VIF namespace is then automatically set to the smaller size fabric port channel port group usage (either 63 or 118 VIFs) of the two groups.


For high availability cluster mode applications, symmetric cabling configurations are strongly recommended. If the cabling is asymmetric, the maximum number of VIFs available is the smaller of the two cabling configurations.

For more information on the maximum number of VIFs for your Cisco UCS environment, see the configuration limits document for your hardware and software configuration.

Power Supplies

Table 2 Power Supply Models

Cisco PID

Fabric Interconnect

Source

Wattage

UCS-PSU-6248UP-AC

Cisco UCS 6248 UP

110 VAC

390

UCS-PSU-6248UP-DC

Cisco UCS 6248 UP

-48 VDC

390

UCS-PSU-6296UP-AC

Cisco UCS 6296 UP

110 VAC

680

The fabric interconnect uses a front-end power supply. The chassis has slots for two power supplies. Two power supplies can be used for redundancy, but the fabric interconnect is fully functional with one power supply. Power supplies have two LEDs: one for power status and one for failure condition.

Figure 12. AC Power Supply for the Cisco UCS 6248 UP Fabric Interconnect

1

Fault LED

2

Power LED

Figure 13. DC Power Supply for the Cisco UCS 6248 UP Fabric Interconnect

1

Amber LED indicates a failure condition.

2

Green LED indicates the input power status.

Figure 14. Power Supply for the Cisco UCS 6296 UP Switch

1

Failure (top) and Power (bottom) LEDs

3

Release lever

2

Handle

   
Table 3 Power Supply LED Descriptions

Power Supply Condition

Power LED Status

Fail LED Status

No AC power to all power supplies.

Off

Off

Power supply failure, including over voltage, over current, over temperature, and fan failure.

Off

On

Power supply warning events where the power supply continues to operate. These events include high temperature, high power, and slow fan.

Off

1 Hz Blinking

AC present, 3.3 voltage standby (VSB) on, and the power supply unit is off.

1 Hz blinking

Off

Power supply on and OK.

On

Off

If one power supply is installed in the chassis, but the other power supply slot is empty, a blank filler panel (N10-S1BLKP=) should be used to cover the empty slot.

Fan Module

The Cisco UCS 6248 UP fabric interconnect has slots for two fan modules (also called fan trays). Each fan module houses four fans. The combination of four fans per module and two modules provides the chassis with eight fans.

Figure 15. Cisco UCS 6248 UP Fan Module (UCS-FAN-6248UP=)

1

Fan Tray Status LED

The Cisco UCS 6296 UP switch has four fan modules with one fan each.

Figure 16. Cisco UCS 6296 UP Fan Module

1

Captive screw

3

Handle

2

Status LED

   

The bicolor fan module LED indicates fan tray health. Green indicates normal operation, while amber indicates a fan failure.

LED Descriptions

The fabric interconnect has three chassis status LEDs.

Table 4 LEDs for the Cisco UCS Fabric Interconnect

LED

Location

Function

Color

Status

Description

System Status LED

Front

Power/Health

Green

Solid On

System is on and operating normally

Off

Switch is powered off

Amber

On

Fault condition

Fan Tray Status

Fan trays (front)

Fan tray health indicator

(multi color)

Green

On

Fan tray is operating normally

Amber

On

Fan failure has occurred within the fan tray

PSU Status

Power supply

(front)

PSU Health (multi color)

Green

OFF

No AC power to power supply

Solid On

Power supply on and OK

Amber

Solid On

Power supply failures, overvoltage, overcurrent, overtemperature

1 Hz

blinking

AC present, 3.3 VSB on, PSU is off

OFF

Operating normally

Beaconing

Front of chassis

Identify selected chassis

Blue

Solid on

Chassis is selected

Off

Chassis is not selected

Port Level LEDs

There are port activity LEDs on the chassis and on the expansion modules.

Table 5 Port LEDs

Color

Description

Green (blinking)

Link is up, enabled, and active.

Amber

Link is administratively disabled.

Amber (blinking)

POST or operational error.

Off

Link is down.

Supported Transceivers

The fabric interconnect supports SFP+ Ethernet transceivers, SFP transceivers, and SFP Fibre Channel transceivers. Specifications for these transceivers are at this URL:

http:/​/​www.cisco.com/​en/​US/​docs/​interfaces_modules/​transceiver_modules/​installation/​note/​78_​15160.html

SFP+ Transceivers

The high bandwidth of 10-Gigabit Ethernet poses challenges to transmissions that are met by the transceiver and cabling options supported by the Cisco UCS 6200 platform.

The enhanced SFP+ 10-Gigabit Ethernet transceiver module is a bidirectional device with a transmitter and receiver in the same physical package. It has a 20-pin connector on the electrical interface and duplex LC connector on the optical interface.

Table 6 Supported Transceivers

Cisco SFP

Description

FET-10G

10G SFP+ Fabric extender transceiver module

SFP-10G-SR

10GBASE-SR SFP+ module (multimode fiber [MMF])

SFP-10G-LR

10GBASE-LR SFP+ module (single-mode fiber [SMF])

SFP-H10GB-CU1M

10GBASE-CU SFP+ cable 1 m (Twinax cable)

SFP-H10GB-CU3M

10GBASE-CU SFP+ cable 3 m (Twinax cable)

SFP-H10GB-CU5M

10GBASE-CU SFP+ cable 5 m (Twinax cable)

SFP-H10GB-ACU7M

10GBASE-CU SFP+ cable 7 m (Twinax cable)

SFP-H10GB-ACU10M

10GBASE-CU SFP+ cable 10 m (Twinax cable)

GLC-T

1000BASE-T SFP

GLC-SX-MM

GE SFP, LC connector SX transceiver (MMF)

GLC-LH-SM

GE SFP, LC connector LX/LH transceiver (SMF)

SFP-GE-T

1000BASE-T SFP, extended temperature range

SFP-GE-S

GE SFP, LC connector SX transceiver (MMF), extended temperature range and digital optical monitoring (DOM)

SFP-GE-L

GE SFP, LC connector LX/LH transceiver (SMF), extended temperature range and DOM

DS-SFP-FC4G-SW

4-Gbps Fibre Channel SW SFP, LC (for FC configured Unified ports)

DS-SFP-FC4G-LW

4-Gbps Fibre Channel LW SFP, LC (for FC configured Unified ports)

DS-SFP-FC8G-SW

8-Gbps Fibre Channel SW SFP+, LC(for FC configured Unified ports)

DS-SFP-FC8G-LW

8-Gbps Fibre Channel LW SFP+, LC (for FC configured Unified ports)


Note


The maximum length of fiber optic runs is limited to 300 meters. This is imposed by our use of 802.3X/802.1Qbb Priority Pauses. SFP-10G-LR is supported between fabric interconnect and FEX, but the 300 m limit still applies.


SFP Transceivers

The SFP 1 -Gigabit Ethernet transceiver module is a bidirectional device with a transmitter and receiver in the same physical package.

Table 7 Supported SFP Optical Transceivers

Model

Description

GLC-T

1-Gigabit Ethernet copper SFP module

GLC-SX-MM

1-Gigabit Ethernet—short range (550m max) SFP module

GLC-LH-SM

1-Gigabit Ethernet—long range (10km) SFP module


Note


The maximum length of fiber optic runs is limited to 300 meters. This is imposed by our use of 802.3X/802.1Qbb Priority Pauses. SFP-10G-LR is supported between fabric interconnect and I/O Module, but the 300 m limit still applies.


SFP+ Copper Cables

Copper cables are available for use with the 10-Gigabit Ethernet SFP+ module.

Table 8 Cables Used with the 10-Gbps Ethernet SFP+ Transceivers

Model

Description

SFP-H10GB-CU1M

10GBASE-CU SFP+ 1-meter cable

SFP-H10GB-CU3M

10GBASE-CU SFP+ 3-meter cable

SFP-H10GB-CU5M

10GBASE-CU SFP+ 5-meter cable

SFP-H10GB-ACU7M=

10GBASE-CU SFP+ 7-meter cable

SFP-H10GB-ACU10M=

10GBASE-CU SFP+ 10-meter cable

On the Cisco UCS 6200 platforms, you can use an innovative Twinax copper cable that connects to standard SFP+ connectors for in-rack use, and on optical cable for longer cable runs.

For in-rack or adjacent-rack cabling, the Cisco UCS 6200 platform supports SFP+ direct-attach 10-Gigabit Ethernet copper, which integrates transceivers with Twinax cables into an energy efficient, low-cost, and low-latency solution. SFP+ direct-attach 10-Gigabit Twinax copper cables use only 0.1 watts of power per transceiver and introduce only approximately 0.25 microsecond of latency per link.

For longer cable runs, the Cisco UCS 6200 platform supports multimode, short-reach optical SFP+ transceivers. These optical transceivers use approximately 1 W per transceiver and have a latency of less than 0.1 microsecond.

Table 9 Supported Cables

Connector (Media)

Cable

Distance

Power (each side)

Transceiver Latency (Link)

Standard

SFP+ CU copper

Twinax

5 m

~ 0.1 W

~ 0.1 microseconds

SFF 8431

SFP+ ACU copper

Active Twinax

7 m/

10 m

~ 0.5 W

~ 6.8 nanoseconds

SFF 8461

SFP+ SR MMF and SR

MM OM2

MM OM3

82 m/

300 m

1 W

~ 0 microseconds

IEEE 802.3ae

SFP Fibre Channel Transceivers

The Cisco UCS 6200 series fabric interconnects support multimode 850 nm 4 Gbps and 8 Gbps SFPs with 150 m reach.

Table 10 SFP Fiber Channel Transceivers

Model

Description

Cisco DS-SFP-FC4G-SW

4 Gbps Fibre Channel-SW SFP, LC (for FC configured Unified ports)

Cisco DS-SFP-FC4G-LW

4 Gbps Fibre Channel-LW SFP, LC, (10 km reach) (for FC configured Unified ports)

Cisco DS-SFP-FC8G-SW

8-Gbps Fibre Channel SW SFP+, LC (for FC configured Unified ports)

Cisco DS-SFP-FC8G-LW

8-Gbps Fibre Channel LW SFP+, LC (for FC configured Unified ports)


Note


The maximum length of fiber optic runs from the fabric interconnect to a chassis is limited to 300 meters. This restriction is imposed by the use of 802.3X/802.1Qbb Priority Pauses.