Configuration of Fibre Channel Interfaces
This chapter describes how to configure the Fibre Channel interfaces.
This chapter includes the following topics:
Information About Fibre Channel Interfaces
This section includes the following topics:
Generations of Modules and Switches
Cisco MDS 9000 Family hardware modules and switches are categorized into generations based on the time of introduction, capabilities, features, and compatibilities:
- Generation 1—Modules and switches with a maximum port speed of 2 Gbps.
- Generation 2—Modules and switches with a maximum port speed of 4 Gbps.
- Generation 3—Modules and switches with a maximum port speed of 8 Gbps.
- Generation 4—Modules with a maximum port speed of 8-Gbps or 10-Gbps.
The Cisco MDS 9500 Series switches, Cisco MDS 9222i, Cisco MDS 9216A, and Cisco MDS 9216i switches support the Generation 2 modules. Each module or switch can have one or more ports in port groups that share common resources such as bandwidth and buffer credits.
In addition to supporting Generation 2 modules, the Cisco MDS 9500 Series switches and the Cisco MDS 9222i switch support the Generation 3 modules. Similar to Generation 2, each Generation 3 or Generation 4 module can have one or more ports in port groups that share common resources such as bandwidth and buffer credits.
Generation 3 modules are supported on the Cisco MDS 9506 and 9509 switches with Supervisor-2 modules. The MDS 9513 Director supports 4/44-port Host-Optimized Fibre Channel switching module with either Fabric 1 or Fabric 2 modules, but requires Fabric 2 module for support of the 48-port and the 24-port 8-Gbps Fibre Channel switching modules. The MDS 9222i switch supports the 4/44-port Host-Optimized Fibre Channel switching module.
The Cisco 9500 Series switches support the following Generation 4 modules: the 48-port 8-Gbps Advanced Fibre Channel switching module (DS-X9248-256K9) and the 32-port 8-Gbps Advanced Fibre Channel module (DS-X9232-256K9). Cisco MDS NX-OS Release 6.x or higher is required to support the Generation 4 modules.
Table 1 identifies the Generation 2, Generation 3, and Generation 4 modules, as well as the Fabric switches.
Part Number |
Product Name and Description |
---|---|
Generation 4 Modules |
|
DS-X9248-256K9 |
48-port 8-Gbps Advanced Fibre Channel switching module. |
DS-X9232-256K9 |
32-port 8-Gbps Advanced Fibre Channel switching module. |
DS-X9530-SF2A-K9 |
Supervisor-2A module for Cisco MDS 9500 Series switches. |
DS-13SLT-FAB3 |
Fabric 3 module that enables the 32-port and the 48-port 8-Gbps Advanced Fibre Channel switching module to use the full 96-Gbps or 256-Gbps backplane crossbar bandwidth. |
Generation 3 Modules |
|
DS-X9248-96K9 |
48-port 8-Gbps Fibre Channel switching module. |
DS-X9224-96K9 |
24-port 8-Gbps Fibre Channel switching module. |
DS-X9248-48K9 |
4/44-port 8-Gbps Host-Optimized Fibre Channel switching module |
DS-13SLT-FAB2 |
Fabric 2 module that enables the 24-port and the 48-port 8-Gbps Fibre Channel switching module to use the full 96-Gbps backplane bandwidth with any-to-any connectivity. |
Generation 3 Fabric Switch |
|
DS-C9148-K9 |
Cisco MDS 9148 Fabric switch. 48-port 8-Gbps Fabric switch. |
Generation 2 Modules |
|
DS-X9148 |
48-port 4-Gbps Fibre Channel switching module. |
DS-X9124 |
24-port 4-Gbps Fibre Channel switching module. |
DS-X9304-18K9 |
18-port 4-Gbps Fibre Channel switching module with 4-Gigabit Ethernet ports. |
DS-X9112 |
12-port 4-Gbps Fibre Channel switching module. |
DS-X9704 |
4-port 10-Gbps Fibre Channel switching module. |
DS-X9530-SF2-K9 |
Supervisor-2 module for Cisco MDS 9500 Series switches. |
Generation 2 Fabric Switches |
|
DS-C9134-K9 |
Cisco MDS 9134 Fabric switch. 32-port 4-Gbps Fabric switch with 2 additional 10-Gbps ports. |
DS-C9124-K9 |
Cisco MDS 9124 Fabric switch. 24-port 4-Gbps Fabric switch. |
DS-C9222i-K9 |
Cisco MDS 9222i Multiservice Modular switch. 18-port 4-Gbps switch with 4-Gigabit Ethernet IP storage services ports, and a modular expansion slot to host Cisco MDS 9000 Family switching and services modules. |
Note |
Generation 2 Fibre Channel switching modules are not supported on the Cisco MDS 9216 switch; however, they are supported by both the Supervisor-1 module and the Supervisor-2 module. |
For detailed information about the installation and specifications for these modules and switches, refer to the hardware installation guide for your switch.
Port Groups
Each module or switch can have one or more ports in port groups that share common resources such as bandwidth and buffer credits. Port groups are defined by the hardware consisting of sequential ports. For example, ports 1 through 6, ports 7 through 12, ports 13 through 18, ports 19 through 24, ports 25 through 30, 31 through 36, and ports 37 through 42, 43 through 48 are the port groups on the 48-port 8-Gbps Advanced Fibre Channel switching modules.
Table 1 shows the bandwidth and number of ports per port group for the Generation 2, Generation 3, and Generation 4 Fibre Channel modules, and Generation 2 and Generation 3 Fabric switches.
Part Number | Product Name/ Description | Number of Ports Per Port Group | Bandwidth Per Port Group (Gbps) | Maximum Bandwidth Per Port (Gbps) |
---|---|---|---|---|
Generation 4 Modules | ||||
DS-X9248-256K9 | 48-port 8-Gbps Advanced Fibre Channel switching module. | 6 | 32.4 or 12.8 | 8 or 10 Gbps—depending on the configuration |
DS-X9232-256K9 | 32-port 8-Gbps Advanced Fibre Channel switching module. | 4 | 32.43 or 12.84 | 8 or 10 Gbps—depending on the configuration |
Generation 3 Modules | ||||
DS-X9248-96K9 | 48-port 8-Gbps Fibre Channel switching module | 6 | 12.8 | 8 Gbps |
DS-X9224-96K9 | 24-port 8-Gbps Fibre Channel switching module | 3 | 12.8 | 8 Gbps |
DS-X9248-48K9 | 4/44-port 8-Gbps Host-Optimized Fibre Channel switching module | 12 | 12.8 | 8/4 Gbps5 |
Generation 3 Fabric Switches | ||||
DS-C9148-K9 (Cisco MDS 9148 Fabric switch) | 48-port 8-Gbps Fabric switch | 4 | 32 | 8 Gbps |
Generation 2 Modules | ||||
DS-X9148 | 48-port 4-Gbps Fibre Channel switching module | 12 | 12.8 | 4 Gbps |
DS-X9124 | 24-port 4-Gbps Fibre Channel switching module | 6 | 12.8 | 4 Gbps |
DS-X9304-18K9 (MSM-18/4 Multiservice module) |
18-port 4-Gbps Fibre Channel switching module with 4-Gigabit Ethernet ports | 6 | 12.8 | 4 Gbps |
DS-X9112 | 12-port 4-Gbps Fibre Channel switching module | 3 | 12.8 | 4 Gbps |
DS-X9704 | 4-port 10-Gbps Fibre Channel switching module | 1 | 10 | 10 Gbps |
Generation 2 Fabric Switches | ||||
DS-C9134-K9 (Cisco MDS 9134 Fabric switch) | 32-port 4-Gbps Fabric switch | 4 | 16 | 4 Gbps |
2-port 10-Gbps Fabric switch | 1 | 10 | 10 Gbps | |
DS-C9124K9 (Cisco MDS 9124 Fabric switch) | 24-port 4-Gbps Fabric switch | 4 | 16 | 4 Gbps |
DS-C9222i-K9 (Cisco MDS 9222i Multiservice Modular switch) |
18-port 4-Gbps, 4 Gigabit Ethernet ports and a modular expansion slot. | 6 | 12.8 | 4 Gbps |
Port Rate Modes
In Generation 2, Generation 3, and Generation 4 modules, you can configure the port rate modes. The port rate mode configuration is used to determine the bandwidth allocation for ports in a port group. Two port rate modes are supported:
Note |
In Generation 1 modules, you cannot configure the port rate modes. The mode is determined implicitly based on the port mode and line card type. |
Note |
Port rate modes are not supported on the Cisco Fabric Switch for HP c-Class Blade System, and the Cisco Fabric Switch for IBM Blade Center. |
Port Rate Mode Support on Generation 2, Generation 3, and Generation 4 Modules and Switches shows the modules that support dedicated, shared, and the default rate modes.
Part Number | Product Name/Description | Supports Dedicated Rate Mode | Supports Shared Rate Mode | Default Speed Mode and Rate Mode on All Ports |
---|---|---|---|---|
Generation 4 Modules | ||||
DS-X9248-256K9 | 48-port 8-Gbps Advanced Fibre Channel switching module | Yes | Yes | Auto, Shared |
DS-X9232-256K9 | 32-port 8-Gbps Advanced Fibre Channel switching module | Yes | Yes7 | Auto, Shared |
Generation 3 Modules | ||||
DS-X9248-96K9 | 48-Port 8-Gbps Fibre Channel switching module | Yes | Yes | Auto, Shared |
DS-X9224-96K9 | 24-Port 8-Gbps Fibre Channel switching module | Yes | Yes9 | Auto, Shared |
DS-X9248-48K9 | 4/44-Port 8-Gbps Host-Optimized Fibre Channel switching module | Yes | Yes10 | Auto Max 4 Gbps, Shared |
Generation 3 Fabric Switches | ||||
DS-C9148-K9 (Cisco MDS 9148 Fabric switch) |
48-port 8-Gbps Fabric switch | Yes | No | Auto, Dedicated |
Generation 2 Modules | ||||
DS-X9148 | 48-port 4-Gbps Fibre Channel switching module 11 | Yes | Yes | Auto, Shared |
DS-X9124 | 24-port 4-Gbps Fibre Channel switching module | Yes | Yes | Auto, Shared |
DS-X9304-18K9 (MSM-18/4 Multiservice module) |
18-port 4-Gbps Fibre Channel switching module with 4-Gigabit Ethernet ports | Yes | Yes | Auto, Shared |
DS-X9112 | 12-port 4-Gbps Fibre Channel switching module | Yes | No | Auto, Dedicated |
DS-X9704 | 4-port 10-Gbps Fibre Channel switching module | Yes | No | Auto, Dedicated |
Generation 2 Fabric Switches | ||||
DS-C9134-K9 (Cisco MDS 9134 Fabric switch) |
32-port 4-Gbps Fabric switch | Yes | Yes | Auto, Shared |
2-port 10-Gbps Fabric switch | Yes | No | Auto, Dedicated | |
DS-C9124-K9 (Cisco MDS 9124 Fabric switch) |
24-port 4-Gbps Fabric switch12 | Yes | No | Auto, Dedicated |
DS-C9222i-K9 (Cisco MDS 9222i Multiservice Modular switch) |
18-port 4-Gbps Fibre Channel switch with 4-Gigabit Ethernet IP storage services ports, and a modular expansion slot to host Cisco MDS 9000 Family Switching and Services Modules | Yes | Yes | Auto, Shared |
Dedicated Rate Mode
When port rate mode is configured as dedicated, a port is allocated required fabric bandwidth and related resources to sustain line rate traffic at the maximum operating speed configured for the port. In this mode, ports do not use local buffering and all receive buffers are allocated from a global buffer pool (see the “Buffer Pools” section on page 60-2 ).
Table 1 shows the bandwidth provided by the various port speed configurations on the 8-Gbps Advanced Fibre Channel switching modules.
Configured Speed |
Reserved Bandwidth |
---|---|
Auto |
8 Gbps |
8-Gbps |
|
Auto with 4-Gbps maximum |
4 Gbps |
4-Gbps |
|
Auto with 2-Gbps maximum |
2 Gbps |
2-Gbps |
|
1-Gbps |
1 Gbps |
Table 2 shows the bandwidth provided by the various port speed configurations on the 8-Gbps Fibre Channel switching modules.
Configured Speed |
Reserved Bandwidth |
---|---|
Auto |
8 Gbps |
8-Gbps |
|
Auto with 4-Gbps maximum |
4 Gbps |
4-Gbps |
|
Auto with 2-Gbps maximum |
2 Gbps |
2-Gbps |
|
1-Gbps |
1 Gbps |
Table 3 shows the amount of bandwidth reserved for a configured port speed on 4-Gbps switching modules.
Configured Speed |
Reserved Bandwidth |
---|---|
Auto |
4 Gbps |
4-Gbps |
|
Auto with 2-Gbps maximum |
2 Gbps |
2-Gbps |
|
1-Gbps |
1 Gbps |
Note |
The 4-Port 10-Gbps Fibre Channel module ports in auto mode only support auto speed mode at 10 Gbps. |
Shared Rate Mode
When port rate mode is configured as shared, multiple ports within a port group share data paths to the switch fabric so that fabric bandwidth and related resources are shared. Often, the available bandwidth to the switch fabric may be less than the negotiated operating speed of a port. Ports in this mode use local buffering for the BB_credit buffers.
All ports in 8-Gbps Advanced Fibre Channel switching modules where bandwidth is shared support 1-Gbps, 2-Gbps, 4-Gbps, or 8 Gbps traffic. However, it is possible to configure one or more ports in a port group to operate in dedicated rate mode with 1-Gbps, 2-Gbps, 4-Gbps, or 8 Gbps operating speed.
All ports in 4-Gbps Fibre Channel switching modules where bandwidth is shared support 1-Gbps, 2-Gbps, or 4-Gbps traffic. However, it is possible to configure one or more ports in a port group to operate in dedicated rate mode with 1-Gbps, 2-Gbps, or 4-Gbps operating speed.
All ports in the 32-Port or 48-Port 8-Gbps Advanced Fibre Channel modules where bandwidth is shared support 1-Gbps, 2-Gbps, 4-Gbps, or 8-Gbps traffic in a maximum or 32 or 48 ports.
All ports in the 48-Port and 24-Port 8-Gbps Fibre Channel switching modules where bandwidth is shared support 1-Gbps, 2-Gbps, 4-Gbps, or 8-Gbps traffic.
In the 4/44-Port 8-Gbps Host-Optimized Fibre Channel switching module, all the ports where bandwidth is shared support 1-Gbps, 2-Gbps, 4-Gbps in a maximum of 44 ports, or 8 Gbps in a maximum of 4 ports.
Dedicated Rate Mode Configurations for the 8-Gbps Modules
Table 1 shows the maximum possible dedicated rate mode configuration scenarios for the Generation 4 Fibre Channel modules.
Part Number |
Product Name/ Description |
Dedicated Bandwidth per Port |
Maximum Allowed Ports That Can Come Up |
Ports in Shared Mode |
---|---|---|---|---|
DS-X9248-256K9 |
48-port 8-Gbps Advanced Fibre Channel switching module |
10 Gbps |
24 Ports |
All the remaining ports are 8 Gbps shared. |
8 Gbps |
32 Ports |
|||
4 Gbps |
48 Ports |
|||
2 Gbps |
48 Ports |
|||
1 Gbps |
48 Ports |
|||
DS-X9232-256K9 |
32-port 8-Gbps Advanced Fibre Channel switching module |
10 Gbps |
24 Ports |
All the remaining ports are 8 Gbps shared. |
8 Gbps |
32 Ports |
|||
4 Gbps |
32 Ports |
|||
2 Gbps |
32 Ports |
|||
1 Gbps |
32 Ports |
Table 2 shows the maximum possible dedicated rate mode configuration scenarios for the Generation 3 Fibre Channel modules.
Part Number |
Product Name/ Description |
Dedicated Bandwidth per Port |
Maximum Allowed Ports That Can Come Up |
Ports in Shared Mode |
---|---|---|---|---|
DS-X9224-96K9 |
24-port 8-Gbps Fibre Channel switching module |
8 Gbps |
8 Ports |
All the remaining ports are 8 Gbps shared. |
4 Gbps |
24 Ports |
|||
DS-X9248-96K9 |
48-port 8-Gbps Fibre Channel switching module |
8 Gbps |
8 Ports |
All the remaining ports are 8 Gbps shared. |
4 Gbps |
24 Ports |
|||
2 Gbps |
48 Ports |
|||
DS-X9248-48K9 |
4/44-port 8-Gbps Host-Optimized Fibre Channel switching module |
8 Gbps |
4 Ports |
All the remaining ports are 4 Gbps shared (8 Gbps of bandwidth can be provided only to one port per port group in dedicated or shared rate mode). |
4 Gbps |
12 Ports |
|||
2 Gbps |
24 Ports |
|||
1 Gbps |
48 Ports |
Port Speed
The port speed on an interface, combined with the rate mode, determines the amount of shared resources available to the ports in the port group on a 48-port, 24-port 4-Gbps, or any 8-Gbps Fibre Channel switching module. Especially in the case of dedicated rate mode, the port group resources are reserved even though the bandwidth is not used. For example, on Generation 2 modules, if an interface is configured for autosensing (auto) and dedicated rate mode, then 4 Gbps of bandwidth is reserved even though the maximum operating speed is 2 Gbps. For the same interface, if autosensing with a maximum speed of 2 Gbps (auto max 2000) is configured, then only 2 Gbps of bandwidth is reserved and the unused 2 Gbps is shared with the other interface in the port group.
Note |
The Generation 2, 4-port 10-Gbps switching module supports 10-Gbps traffic only. |
- On Generation 2, 4-Gbps modules, setting the port speed to auto enables autosensing, which negotiates to a maximum speed of 4 Gbps.
- On Generation 3, 8-Gbps modules, setting the port speed to auto enables autosensing, which negotiates to a maximum speed of 8 Gbps.
- On Generation 4, 8-Gbps modules, setting the port speed to auto enables autosensing, which negotiates to a maximum speed of 8 Gbps.
Dynamic Bandwidth Management
On port switching modules where bandwidth is shared, the bandwidth available to each port within a port group can be configured based on the port rate mode and speed configurations. Within a port group, some ports can be configured in dedicated rate mode while others operate in shared mode.
Ports configured in dedicated rate mode are allocated the required bandwidth to sustain a line rate of traffic at the maximum configured operating speed, and ports configured in shared mode share the available remaining bandwidth within the port group. Bandwidth allocation among the shared mode ports is based on the operational speed of the ports. For example, if four ports operating at speeds 1 Gbps, 1 Gbps, 2 Gbps, and 4 Gbps share bandwidth of 8 Gbps, the ratio of allocation would be 1:1:2:4.
Unutilized bandwidth from the dedicated ports is shared among only the shared ports in a port group as per the ratio of the configured operating speed. A port cannot be brought up unless the reserved bandwidth is quaranteed for the shared ports. For dedicated ports, configured bandwidth is taken into consideration while calculating available bandwidth for the port group. This behavior can be changed using bandwidth fairness by using the rate-mode bandwidth-fairness module number command.
For example, consider a 48-port 8-Gbps module. This module has 6 ports per port group with 12.8 Gbps bandwidth. Ports 3 to 6 are configured at 4 Gbps. If the first port is configured at 8 Gbps dedicated rate mode, and the second port is configured at 4-Gbps dedicated rate mode, then no other ports can be configured at 4 Gbps or 8 Gbps because the left over bandwidth of 0.8 Gbps (12.8-(8+4)) cannot meet the required 0.96 Gbps for the remaining four ports. A minimum of 0.24 Gbps reserved bandwidth is required for the for the rest of the four ports. However, if the two ports (for example, 5 and 6) are taken out of service (which is not same as shutdown), required reserved bandwidth for the two ports (3 and 4) is 0.48 and port 2 can be configured at 4 Gbps in dedicated rate mode. This behavior can be overridden by the bandwidth fairness command in which case reserved bandwidth is not enforced. Once the port is up, ports 3 and 4 can share the unutilized bandwidth from ports 1 and 2.
Bandwidth Reservation: 48-Port 96-Gbps Fibre Channel Module
RateMode Configuration Macro |
Description |
---|---|
Dedicated 4 Gbps on the first port of each group and the remaining ports 8 Gbps shared |
Allocates a rate mode of 4 Gbps on the first port of each group and the remaining ports share 8 Gbps depending on the operational speed of the ports. |
Dedicated 8 Gbps on the first port of each group and the remaining ports 8 Gbps shared |
Allocates a rate mode of 8 Gbps on the first port of each group and the remaining ports share 8 Gbps depending on the operational speed of the ports. |
Shared 8 Gbps on all ports (initial and default settings) |
Allocates a rate mode of 8 Gbps on all the available ports. This is the default setting. |
Bandwidth Reservation: 48-Port 48-Gbps Fibre Channel Module
RateMode Configuration Macro |
Description |
---|---|
Dedicated 2 Gbps on the first port of each group and the remaining ports 4 Gbps shared |
Allocates a rate mode of 2 Gbps on the first port of each group and the remaining ports share 4 Gbps depending on the operational speed of the ports. |
Dedicated 8 Gbps on the first port of each group and the remaining ports 4 Gbps shared |
Allocates a rate mode of 8 Gbps on the first port of each group and the remaining ports share 4 Gbps depending on the operational speed of the ports. |
Shared auto with maximum of 4 Gbps on all ports (initial and default settings) |
Allocates a maximum rate mode of 4 Gbps on all the available ports. This is the default setting. |
Bandwidth Reservation: 24-Port 48-Gbps Fibre Channel Module
RateMode Configuration Macro |
Description |
---|---|
Dedicated 8 Gbps on the first port of each group and the remaining ports 8 Gbps shared |
Allocates a rate mode of 8Gbps on the first port of each group and the remaining ports share 8 Gbps depending on the operational speed of the ports. |
Shared Auto on all ports (initial and default settings) |
Allocates a rate mode of 8 Gbps on all the available ports. This is the default setting. |
Bandwidth Reservation: 48-Port 256-Gbps Advanced Fibre Channel Module
RateMode Configuration Macro |
Description |
---|---|
Dedicated 8 Gbps on the first 4 ports in each 6-port port group and the remaining ports 8 Gbps shared |
Allocates a rate mode of 8 Gbps on the first 4 ports in each 6-port port group and the remaining ports share 8 Gbps depending on the operational speed of the ports. |
Dedicated 8 Gbps on the first port of each group and the remaining ports 8 Gbps shared |
Allocates a rate mode of 8 Gbps on the first port of each group and the remaining ports share 8 Gbps depending on the operational speed of the ports. |
Shared 8 Gbps on all ports |
Allocates a rate mode of 8 Gbps on all the available ports. This is the default setting. |
Dedicated 4 Gbps on all ports |
Allocates a rate mode of 4 Gbps on all the available ports. |
Dedicated 10 Gbps on following ports:
|
Allocates a rate mode of 10 Gbps on all the available ports. |
Bandwidth Reservation: 32-Port 256-Gbps Advanced Fibre Channel Module
RateMode Configuration Macro |
Description |
---|---|
Dedicated 8 Gbps on all ports–initial and default settings |
Allocates a rate mode of 8 Gbps on all the available ports. |
Shared 8 Gbps on all ports–initial and default settings |
Allocates a rate mode of shared 8 Gbps on all the available ports. |
Dedicated 10 Gbps on following ports:
|
Allocates a rate mode of 10Gbps on the following ports. |
Out-of-Service Interfaces
On supported modules and fabric switches, you might need to allocate all the shared resources for one or more interfaces to another interface in the port group or module. You can take interfaces out of service to release shared resources that are needed for dedicated bandwidth. When an interface is taken out of service, all shared resources are released and made available to the other interface in the port group or module. These shared resources include bandwidth for the shared mode port, rate mode, BB_credits, and extended BB_credits. All shared resource configurations are returned to their default values when the interface is brought back into service. Corresponding resources must be made available in order for the port to be successfully returned to service.
Caution |
If you need to bring an interface back into service, you might disrupt traffic if you need to release shared resources from other interfaces in the same port group. |
Oversubscription Ratio Restrictions
The 48-port and 24-port 4-Gbps, and all 8-Gbps Fibre Channel switching modules support oversubscription on switches with shared rate mode configurations. By default, all 48-port and 24-port 4-Gbps, and 8-Gbps Fibre Channel switching modules have restrictions on oversubscription ratios enabled. As of Cisco SAN-OS Release 3.1(1) and NX-OS Release 4.1(1), you can disable restrictions on oversubscription ratios.
Table 1 describes the bandwidth allocation for oversubscribed interfaces configured in shared mode on the 4-Gbps and 8-Gbps modules.
Switching Module |
Configured Speed |
Reserved Bandwidth (Gbps) |
Maximum Bandwidth (Gbps) |
|
---|---|---|---|---|
Ratios enabled | Ratios disabled | |||
24-Port 8-Gbps Fibre Channel Module | Auto 8 Gbps | 0.8 | 0.8 | 8 |
Auto Max 4 Gbps | 0.4 | 0.4 | 4 | |
Auto Max 2 Gbps | 0.2 | 0.2 | 2 | |
4/44-Port 8-Gbps Host-Optimized Fibre Channel Module | 8 Gbps | 0.87 | 0.16 | 8 |
Auto Max 4 Gbps | 0.436 | 0.08 | 4 | |
Auto Max 2 Gbps | 0.218 | 0.04 | 2 | |
1 Gbps | 0.109 | 0.02 | 1 | |
48-port 4-Gbps Fibre Channel switching module | Auto 4 Gbps | 0.8 | 0.09 | 4 |
Auto Max 2 Gbps | 0.4 | 0.045 | 2 | |
1 Gbps | 0.2 | 0.0225 | 1 | |
24-port 4-Gbps Fibre Channel switching module | Auto 4 Gbps | 1 | 0.27 | 4 |
Auto Max 2 Gbps | 0.5 | 0.135 | 2 | |
1 Gbps | 0.25 | 0.067 | 1 |
All ports in the 48-port and 24-port 4-Gbps modules can be configured to operate at 4 Gbps in shared mode even if other ports in the port group are configured in dedicated mode, regardless of available bandwidth. However, when oversubscription ratio restrictions are enabled, you may not have all shared 4-Gbps module ports operating at 4 Gbps.
All ports in the 48-port, 32-Port, and 24-port 8-Gbps modules can be configured to operate at 8 Gbps in shared mode even if other ports in the port group are configured in dedicated mode, regardless of available bandwidth. However, when oversubscription ratio restrictions are enabled you may not have all shared 8-Gbps module ports operating at 8 Gbps.
On the 48-port, 32-Port, and 24-port 8-Gbps modules, if you have configured one 8-Gbps dedicated port in one port group, no other ports in the same port group can be configured to operate at 8-Gbps dedicated mode. You can have any number of 8-Gbps shared and 4-Gbps dedicated or shared ports. On the 4/44-port 8-Gbps module, only one port per port group can be configured in 8-Gbps dedicated or shared mode.
In the following example, a 24-port 4-Gbps module has oversubscription ratios enabled and three dedicated ports in one port group operating at 4-Gbps. No other ports in the same port group can be configured to operate at 4 Gbps.
switch# show port-resources module 8
Module 8
Available dedicated buffers are 5478
Port-Group 1
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 0.8 Gbps
Allocated dedicated bandwidth is 12.0 Gbps
--------------------------------------------------------------------
Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode
Buffers (Gbps)
--------------------------------------------------------------------
fc8/1 16 4.0 dedicated
fc8/2 16 4.0 dedicated
fc8/3 16 4.0 dedicated
fc8/4 (out-of-service)
fc8/5 (out-of-service)
fc8/6 (out-of-service)
For dedicated ports, oversubscription ratio restrictions do not apply to the shared pool in port groups. So if oversubscription ratio restrictions are disabled, and you have configured three 4-Gbps dedicated ports in one port group, then you can configure all other ports in the same port group to operate at a shared rate of 4 Gbps.
In the following example, a 48-port module has a group of six ports, four dedicated ports are operating at 8 Gbps, and the two shared ports are also operating at 8 Gbps:
switch# show port-resources module 5
Module 5
Available dedicated buffers for global buffer #0 [port-group 1] are 3970
Available dedicated buffers for global buffer #1 [port-group 2] are 3970
Available dedicated buffers for global buffer #2 [port-group 3] are 3970
Available dedicated buffers for global buffer #3 [port-group 4] are 3970
Available dedicated buffers for global buffer #4 [port-group 5] are 3058
Available dedicated buffers for global buffer #5 [port-group 6] are 3058
Available dedicated buffers for global buffer #6 [port-group 7] are 3970
Available dedicated buffers for global buffer #7 [port-group 8] are 3970
Port-Group 1
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
--------------------------------------------------------------------
Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode
Buffers (Gbps)
--------------------------------------------------------------------
fc5/1 32 8.0 shared
fc5/2 32 8.0 shared
fc5/3 32 8.0 shared
fc5/4 32 8.0 shared
Port-Group 2
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
--------------------------------------------------------------------
Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode
Buffers (Gbps)
--------------------------------------------------------------------
fc5/5 32 8.0 shared
fc5/6 32 8.0 shared
fc5/7 32 8.0 shared
fc5/8 32 8.0 shared
Port-Group 3
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
--------------------------------------------------------------------
Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode
Buffers (Gbps)
--------------------------------------------------------------------
fc5/9 32 8.0 shared
fc5/10 32 8.0 shared
fc5/11 32 8.0 shared
fc5/12 32 8.0 shared
Port-Group 4
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
--------------------------------------------------------------------
Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode
Buffers (Gbps)
--------------------------------------------------------------------
fc5/13 32 8.0 shared
fc5/14 32 8.0 shared
fc5/15 32 8.0 shared
fc5/16 32 8.0 shared
Port-Group 5
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 16.4 Gbps
Allocated dedicated bandwidth is 16.0 Gbps
--------------------------------------------------------------------
Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode
Buffers (Gbps)
--------------------------------------------------------------------
fc5/17 32 8.0 shared
fc5/18 32 8.0 shared
fc5/19 500 8.0 dedicated
fc5/20 500 8.0 dedicated
Port-Group 6
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 16.4 Gbps
Allocated dedicated bandwidth is 16.0 Gbps
--------------------------------------------------------------------
Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode
Buffers (Gbps)
--------------------------------------------------------------------
fc5/21 500 8.0 dedicated
fc5/22 500 8.0 dedicated
fc5/23 32 8.0 shared
fc5/24 32 8.0 shared
Port-Group 7
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
--------------------------------------------------------------------
Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode
Buffers (Gbps)
--------------------------------------------------------------------
fc5/25 32 8.0 shared
fc5/26 32 8.0 shared
fc5/27 32 8.0 shared
fc5/28 32 8.0 shared
Port-Group 8
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
--------------------------------------------------------------------
Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode
Buffers (Gbps)
--------------------------------------------------------------------
fc5/29 32 8.0 shared
fc5/30 32 8.0 shared
fc5/31 32 8.0 shared
fc5/32 32 8.0 shared
Isola-13# show port-resources module 13
Module 13
Available dedicated buffers for global buffer #0 [port-group 1] are 3880
Available dedicated buffers for global buffer #1 [port-group 2] are 3880
Available dedicated buffers for global buffer #2 [port-group 3] are 3880
Available dedicated buffers for global buffer #3 [port-group 4] are 3056
Available dedicated buffers for global buffer #4 [port-group 5] are 3880
Available dedicated buffers for global buffer #5 [port-group 6] are 3880
Available dedicated buffers for global buffer #6 [port-group 7] are 3880
Available dedicated buffers for global buffer #7 [port-group 8] are 3880
Port-Group 1
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
--------------------------------------------------------------------
Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode
Buffers (Gbps)
--------------------------------------------------------------------
fc13/1 32 8.0 shared
fc13/2 32 8.0 shared
fc13/3 32 8.0 shared
fc13/4 32 8.0 shared
fc13/5 32 8.0 shared
fc13/6 32 8.0 shared
Port-Group 2
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
--------------------------------------------------------------------
Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode
Buffers (Gbps)
--------------------------------------------------------------------
fc13/7 32 8.0 shared
fc13/8 32 8.0 shared
fc13/9 32 8.0 shared
fc13/10 32 8.0 shared
fc13/11 32 8.0 shared
fc13/12 32 8.0 shared
Port-Group 3
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
--------------------------------------------------------------------
Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode
Buffers (Gbps)
--------------------------------------------------------------------
fc13/13 32 8.0 shared
fc13/14 32 8.0 shared
fc13/15 32 8.0 shared
fc13/16 32 8.0 shared
fc13/17 32 8.0 shared
fc13/18 32 8.0 shared
Port-Group 4
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 0.4 Gbps
Allocated dedicated bandwidth is 32.0 Gbps
--------------------------------------------------------------------
Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode
Buffers (Gbps)
--------------------------------------------------------------------
fc13/19 250 8.0 dedicated
fc13/20 250 8.0 dedicated
fc13/21 250 8.0 dedicated
fc13/22 250 8.0 dedicated
fc13/23 32 8.0 shared
fc13/24 32 8.0 shared
Port-Group 5
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
--------------------------------------------------------------------
Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode
Buffers (Gbps)
--------------------------------------------------------------------
fc13/25 32 8.0 shared
fc13/26 32 8.0 shared
fc13/27 32 8.0 shared
fc13/28 32 8.0 shared
fc13/29 32 8.0 shared
fc13/30 32 8.0 shared
Port-Group 6
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
--------------------------------------------------------------------
Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode
Buffers (Gbps)
--------------------------------------------------------------------
fc13/31 32 8.0 shared
fc13/32 32 8.0 shared
fc13/33 32 8.0 shared
fc13/34 32 8.0 shared
fc13/35 32 8.0 shared
fc13/36 32 8.0 shared
Port-Group 7
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
--------------------------------------------------------------------
Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode
Buffers (Gbps)
--------------------------------------------------------------------
fc13/37 32 8.0 shared
fc13/38 32 8.0 shared
fc13/39 32 8.0 shared
fc13/40 32 8.0 shared
fc13/41 32 8.0 shared
fc13/42 32 8.0 shared
Port-Group 8
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
--------------------------------------------------------------------
Interfaces in the Port-Group B2B Credit Bandwidth Rate Mode
Buffers (Gbps)
--------------------------------------------------------------------
fc13/43 32 8.0 shared
fc13/44 32 8.0 shared
fc13/45 32 8.0 shared
fc13/46 32 8.0 shared
fc13/47 32 8.0 shared
fc13/48 32 8.0 shared
...
When disabling restrictions on oversubscription ratios, all ports in shared mode on 48-port and 24-port 4-Gbps or any 8-Gbps Fibre Channel switching modules must be shut down. When applying restrictions on oversubscription ratios, you must take shared ports out of service.
Note |
When restrictions on oversubscription ratios are disabled, the bandwidth allocation among the shared ports is proportionate to the configured speed. If the configured speed is auto on Generation 2 modules, then bandwidth is allocated assuming a speed of 4 Gbps. For example, if you have three shared ports configured at 1, 2, and 4 Gbps, then the allocated bandwidth ratio is 1:2:4. As of Cisco SAN-OS Release 3.0 and NX-OS Release 4.1(1) or when restrictions on oversubscription ratios are enabled, the port bandwidths are allocated in equal proportions, regardless of port speed, so, the bandwidth allocation for the same three ports mentioned in the example would be 1:1:1. |
Bandwidth Fairness
This feature improves fairness of bandwidth allocation among all ports and provides better throughput average to individual data streams. Bandwidth fairness can be configured per module.
As of Cisco SAN-OS Release 3.1(2), all 48-port and 24-port 4-Gbps Fibre Channel switching modules, as well as 18-port Fibre Channel/4-port Gigabit Ethernet Multiservice modules, have bandwidth fairness enabled by default. As of Cisco NX-OS Release 4.1(1), all the 8-Gbps Fibre Channel switching modules have bandwidth fairness enabled by default.
Caution |
When you disable or enable bandwidth fairness, the change does not take effect until you reload the module. |
Use the show module bandwidth-fairness command to check whether ports in a module are operating with bandwidth fairness enabled or disabled.
switch# show module 2 bandwidth-fairness
Module 2 bandwidth-fairness is enabled
Note |
This feature is supported only on the 48-port and 24-port 4-Gbps modules, the 8-Gbps modules, and the 18/4-port Multiservice Module (MSM). |
Upgrade or Downgrade Scenario
When you are upgrading from a release earlier than Cisco SAN-OS Release 3.1(2), all modules operate with bandwidth fairness disabled until the next module reload. After the upgrade, any new module that is inserted has bandwidth fairness enabled.
When you are downgrading to a release earlier than Cisco SAN-OS Release 3.1(2), all modules keep operating in the same bandwidth fairness configuration prior to the downgrade. After the downgrade, any new module that is inserted has bandwidth fairness disabled.
Note |
After the downgrade, any insertion of a module or module reload will have bandwidth fairness disabled. |
Guidelines and Limitations
This section includes the following topics:
Combining Generation 1, Generation 2, Generation 3, and Generation 4 Modules
Cisco MDS NX-OS Release 6.x and later supports combining Generation 1, Generation 2, Generation 3, and Generation 4 modules and switches with the following considerations:
-
MDS NX-OS Release 4.1(1) and later features are not supported on the following Generation 1 switches and modules:
- Supervisor 1 module
- 4-Port IP Storage Services module
- 8-Port IP Storage Services module
- MDS 9216 switch
- MDS 9216A switch
- MDS 9020 switch
- MDS 9120 switch
- MDS 9140 switch
- Supervisor-1 modules must be upgraded to Supervisor-2 modules on the MDS 9506 and MDS 9509 Directors.
- IPS-4 and IPS-8 modules must be upgraded to the MSM-18/4 Multiservice modules.
- Fabric 1 modules must be upgraded to Fabric 2 modules on the MDS 9513 Director to use the 48-port or the 24-port 8-Gbps module.
- Fabric 2 modules must be upgraded to Fabric 3 modules on the MDS 9513 Director to get the maximum backplane bandwidth of 256 Gbps.
- Cisco Fabric Manager Release 4.x supports MDS SAN-OS Release 3.x and NX-OS 4.x in mixed mode through Interswitch Link (ISL) connectivity.
Note |
When a Cisco or another vendor switch port is connected to a Generation 1 module port (ISL connection), the receive buffer-to-buffer credits of the port connected to the Generation 1 module port must not exceed 255. |
Local Switching Limitations
All ports in the module must be in shared mode.
- Use the switchport ratemode shared command to ensure that all the ports in the module are in shared mode.
- No E ports are allowed in the module because E ports must be in dedicated mode.
Port Index Limitations
Cisco MDS 9000 switches allocate index identifiers for the ports on the modules. These port indexes cannot be configured. You can combine Generation 1, Generation 2, Generation 3, and Generation 4 switching modules, with either Supervisor-1 modules or Supervisor-2 modules. However, combining switching modules and supervisor modules has the following port index limitations:
- Supervisor-1 modules only support a maximum of 252 port indexes, regardless of the type of switching modules.
- Supervisor-2 modules support a maximum of 1020 port indexes when all switching modules in the chassis are Generation 2 or Generation 3.
- Supervisor-2 modules only support a maximum of 252 port indexes when only Generation 1 switching modules, or a combination of Generation 1, Generation 2, Generation 3, or Generation 4 switching modules are installed in the chassis.
Note |
On a switch with the maximum limit of 252 as port index, any new module that exceeds the limit does not power up when installed. |
You can use the show port index-allocation command to display the allocation of port indexes on the switch.
switch# show port index-allocation
Module index distribution:
------------------------------------------------------+
Slot | Allowed | Allotted indices info |
| range | Total | Index values |
----|---------|-------|------------------------------|
1 | ----- | - | (None) |
2 | ----- | - | (None) |
3 | ----- | - | (None) |
4 | ----- | - | (None) |
5 | 0-1023| 32 | 0-31 |
6 | ----- | - | (None) |
9 | ----- | - | (None) |
10 | ----- | - | (None) |
11 | ----- | - | (None) |
12 | ----- | - | (None) |
13 | 0-1023| 48 | 32-79 |
SUP | 253-255 | 3 | 253-255 |
Generation 1 switching modules have specific numbering requirements. If these requirements are not met, the module does not power up. The port index numbering requirements include the following:
- If port indexes in the range of 256 to 1020 are assigned to operational ports, Generation 1 switching modules do not power up.
- A block of contiguous port indexes is available. If this block of port indexes is not available, Generation 1 modules do not power up. Table 1 shows the port index requirements for the Generation 1 modules.
Note |
If the switch has Supervisor-1 modules, the block of 32 contiguous port indexes must begin on the slot boundary. The slot boundary for slot 1 is 0, for slot 2 is 32, and so on. For Supervisor-2 modules, the contiguous block can start anywhere. |
Generation 1 Module | Number of Port Indexes Required | |
Supervisor-1 Module | Supervisor-2 Module | |
16-port 2-Gbps Fibre Channel module | 16 | 16 |
32-port 2-Gbps Fibre Channel module | 32 | 32 |
8-port Gigabit Ethernet IP Storage Services module | 32 | 32 |
4-port Gigabit Ethernet IP Storage Services module | 32 | 16 |
32-port 2-Gbps Fibre Channel Storage Services Module (SSM). | 32 | 32 |
14-port Fibre Channel/2-port Gigabit Ethernet Multiprotocol Services (MPS-14/2) module | 32 | 22 |
The allowed mix of Generation 1 and Generation 2 switching modules in a chassis is determined at run-time, either when booting up the switch or when installing the modules. In some cases, the sequence in which switching modules are inserted into the chassis determines if one or more modules is powered up.
When a module does not power up because of a resource limitation, you can display the reason by using the show module command.
When a module does not power up because of a resource limitation, you can see the reason by viewing the module information in the Information pane.
switch# show module
Mod Ports Module-Type Model Status
--- ----- ----------------------------------- ------------------ ----------
5 32 1/2/4/8/10 Gbps Advanced FC Module DS-X9232-256K9 ok
7 0 Supervisor/Fabric-2 DS-X9530-SF2-K9 active *
13 48 1/2/4/8/10 Gbps Advanced FC Module DS-X9248-256K9 ok
Mod Sw Hw World-Wide-Name(s) (WWN)
--- -------------- ------ --------------------------------------------------
5 5.2(2) 0.207 21:01:00:0d:ec:b7:28:c0 to 21:20:00:0d:ec:b7:28:c0
7 5.2(2) 1.9 --
13 5.2(2) 0.212 23:01:00:0d:ec:b7:28:c0 to 23:30:00:0d:ec:b7:28:c0
Mod MAC-Address(es) Serial-Num
--- -------------------------------------- ----------
5 68-ef-bd-a8-45-cc to 68-ef-bd-a8-45-d0 JAF1450CHQT
7 00-24-c4-60-00-f8 to 00-24-c4-60-00-fc JAE141502L2
13 68-ef-bd-a8-40-00 to 68-ef-bd-a8-40-04 JAF1450BMBP
Xbar Ports Module-Type Model Status
--- ----- ----------------------------------- ------------------ ----------
1 0 Fabric Module 3 DS-13SLT-FAB3 ok
2 0 Fabric Module 3 DS-13SLT-FAB3 ok
Xbar Sw Hw World-Wide-Name(s) (WWN)
--- -------------- ------ --------------------------------------------------
1 NA 0.4 --
2 NA 0.4 --
Xbar MAC-Address(es) Serial-Num
--- -------------------------------------- ----------
1 NA JAF1451AMHG
2 NA JAF1451AMHN
* this terminal session
The running configuration is updated when modules are installed. If you save the running configuration to the startup configuration (using the copy running-config startup-config command), during reboot the switch powers up the same set of modules as before the reboot regardless of the sequence in which the modules initialize. You can use the show port index-allocation startup command to display the index allocation the switch uses at startup.
switch# show port index-allocation startup
Startup module index distribution:
------------------------------------------------------+
Slot | Allowed | Alloted indices info |
| range | Total | Index values |
-----|---------|-------|------------------------------|
1 | ----- | 34 | 0-31,80-81 |
2 | ----- | 32 | 32-63 |
3 | ----- | 16 | 64-79 |(Slot 1 shares 80-81)
4 | ----- | 48 | 96-127,224-239 |
SUP | 253-255 | 3 | 253-255 |
Note |
The output of the show port index-allocation startup command does not display anything in the Allowed range column because the command extracts the indices from the persistent storage service (PSS) and displaying an allowed range for startup indices is meaningless. |
If a module fails to power up, you can use the show module slot recovery-steps command to display the reason.
For information on recovering a module powered-down because port indexes are not available, refer to the Cisco MDS 9000 Family Troubleshooting Guide .
Tip |
Whenever using mixed Generation 1 and Generation 2 modules, power up the Generation 1 modules first. During a reboot of the entire switch, the Generation 1 modules power up first (default behavior). |
PortChannel Limitations
PortChannels have the following restrictions:
- The maximum number of PortChannels allowed is 256 if all switching modules are Generation 2 or Generation 3, or both.
- The maximum number of PortChannels allowed is 128 whenever there is a Generation 1 switching module in use with a Generation 2 or Generation 3 switching module.
- Ports need to be configured in dedicated rate mode on the Generation 2 and Generation 3 switching module interfaces to be used in the PortChannel.
Note |
The number of PortChannels allowed does not depend on the type of supervisor module. However, Generation 3 modules require the Supervisor 2 module on the MDS 9506 and 9509 switches. |
The Generation1, Generation 2, and Generation 3 modules have the following restrictions for PortChannel configuration:
- Generation 1 switching module interfaces do not support auto speed with a maximum of 2 Gbps.
- Generation 1 and Generation 2 module interfaces do not support auto speed with maximum of 4 Gbps.
- Generation 2 and Generation 3 switching module interfaces cannot be forcefully added to a PortChannel if sufficient resources are not available.
Note |
Before adding a Generation 2 or Generation 3 interface to a PortChannel, use the show port-resources module command to check for resource availability. |
When configuring PortChannels on switches with Generation 1, Generation 2, and Generation 3 switching modules, follow one of these procedures:
- Configure the PortChannel, and then configure the Generation 2 and Generation 3 interfaces to auto with a maximum of 2 Gbps.
- Configure the Generation 1 switching modules followed by the Generation 2 switching modules, and then the Generation 3 switching modules, and then configure the PortChannel.
When configuring PortChannels on switches with only Generation 2 and Generation 3 switching modules, follow one of these procedures:
- Configure the PortChannel, and then configure the Generation 3 interfaces to auto with a maximum of 4 Gbps.
- Configure the Generation 2 switching modules, followed by the Generation 3 switching modules, and then configure the PortChannel.
Table 1 describes the results of adding a member to a PortChannel for various configurations.
PortChannel Members |
Configured Speed |
New Member Type |
Addition Type |
Result | |
---|---|---|---|---|---|
PortChannel | New Member | ||||
No members | Any | Any | Generation 1 or Generation 2 or Generation 3 or Generation 4 | Force | Pass |
Auto | Auto | Generation 1 or Generation 2 or Generation 3 or Generation 4 | Normal or force | Pass | |
Auto | Auto max 2000 | Generation 2 or Generation 3 or Generation 4 | Normal | Fail | |
Force | Pass or fail13 | ||||
Auto | Auto max 4000 | Generation 3 or Generation 4 | |||
Auto max 2000 | Auto | Generation 2 or Generation 3 or Generation 4 | Normal | Fail | |
Force | Pass | ||||
Auto max 2000 | Auto max 4000 | Generation 3 or or Generation 4 | |||
Auto max 4000 | Auto | Generation 2 or Generation 3 or or Generation 4 | |||
Auto max 4000 | Auto max 2000 | Generation 2 or Generation 3 or or Generation 4 | |||
Generation 1 interfaces | Auto | Auto | Generation 2 or Generation 3 | Normal | Fail |
Force | Pass | ||||
Auto max 2000 | Auto | Generation 1 | Normal or force | Pass | |
Auto max 2000 | Auto | Generation 2 or Generation 3 | Normal | Fail | |
Force | Pass or fail1 | ||||
Auto max 4000 | Auto | Generation 1 or Generation 2 | |||
Auto max 4000 | Auto | Generation 3 | |||
Generation 2 interfaces | Auto | Auto | Generation 1 | Normal or force | Fail |
Auto max 2000 | Auto | Generation 1 | Normal or force | Pass | |
Auto max 2000 | Auto | Generation 2 or Generation 3 | Normal | Fail | |
Force | Pass | ||||
Auto | Auto max 2000 | Generation 2 or Generation 3 | Normal | Fail | |
Force | Pass | ||||
Generation 3 interfaces | Auto | Auto | Generation 1 | Normal or force | Fail |
Auto max 2000 | Auto | Generation 1 | Normal or force | Pass | |
Auto max 2000 | Auto | Generation 2 | Normal | Fail | |
Force | Pass | ||||
Auto | Auto max 2000 | Generation 2 | Normal | Fail | |
Force | Pass | ||||
Auto max 2000 | Auto | Generation 3 | Normal | Fail | |
Force | Pass | ||||
Auto | Auto max 2000 | Generation 3 | Normal | Fail | |
Force | Pass | ||||
Generation 4 interfaces | Auto | Auto | Generation 1 | Normal or force | Fail |
Auto max 2000 | Auto | Generation 1 | Normal or force | Pass | |
Auto max 2000 | Auto | Generation 2 | Normal | Fail | |
Force | Pass | ||||
Auto | Auto max 2000 | Generation 2 | Normal | Fail | |
Force | Pass | ||||
Auto max 2000 | Auto | Generation 3 or Generation 4 | Normal | Fail | |
Force | Pass | ||||
Auto | Auto max 2000 | Generation 3 or Generation 4 | Normal | Fail | |
Force | Pass |
Use the show port-channel compatibility parameters command to obtain information about PortChannel addition errors.
Default Settings
Table 1 lists the default settings for Generation 2 interface parameters.
Parameter |
Default |
|||
---|---|---|---|---|
48-Port 4-Gbps Switching Module | 24-Port 4-Gbps Switching Module | 12-Port 4-Gbps Switching Module | 4-Port 10-Gbps Switching Module | |
Speed mode | auto14 | auto | auto | auto15 |
Rate mode | shared | shared | dedicated | dedicated |
Port mode | Fx | Fx | auto16 | auto17 |
BB_credit buffers | 16 | 16 | 250 | 250 |
Performance buffers | – | – | 14518 | 1455 |
Table 2 lists the default settings for Generation 3 interface parameters.
Parameter |
Default |
||
---|---|---|---|
48-Port 8-Gbps Switching Module |
24-Port 8-Gbps Switching Module |
4/44-Port 8-Gbps Host-Optimized Switching Module |
|
Speed mode19 | auto | auto | auto_max_4G20 |
Rate mode | shared | shared | shared |
Port mode | Fx | Fx | Fx |
BB_credit buffers | 32 | 32 | 32 |
Table 3 lists the default settings for Generation 4 interface parameters.
Parameter |
Default |
|
---|---|---|
48-Port 8-Gbps Advanced Fibre Channel Switching Module | 32-Port 8-Gbps Advanced Fibre Channel Switching Module | |
Speed mode | auto21 | auto |
Rate mode | shared | shared |
Port mode | Fx | Fx |
BB_credit buffers | 32 | 32 |
Configuring Fibre Channel Interfaces
This section includes the following topics:
Task Flow for Migrating Interfaces from Shared Mode to Dedicated Mode
The 48-Port, 24-Port, and 4/44-Port 8-Gbps Fibre Channel switching modules support the following features:
- 1-Gbps, 2-Gbps, 4-Gbps, and 8-Gbps speed traffic
- Shared and dedicated rate mode
- ISL and Fx port modes
- Extended BB_credits
The 48-port and 24-port 4-Gbps Fibre Channel switching modules support the following features:
- 1-Gbps, 2-Gbps, and 4-Gbps speed traffic
- Shared and dedicated rate mode
- ISL (E or TE) and Fx (F or FL) port modes
- Extended BB_credits
Note |
If you change the port bandwidth reservation parameters on a 48-port or 24-port 4-Gbps module, the change affects only the changed port. No other ports in the port group are affected. |
To configure the 4-Gbps and 8-Gbps Fibre Channel switching modules when starting with the default configuration or when migrating from shared rate mode to dedicated rate mode, follow these steps:
Procedure
Step 1 |
Take unused interfaces out of service to release resources for other interfaces, if necessary. See the Taking Interfaces Out of Service. |
||
Step 2 |
Configure the traffic speed to use (1 Gbps, 2 Gbps, 4 Gbps, 8 Gbps, or autosensing with a maximum of 2 Gbps or 4 Gbps). See the Dynamic Bandwidth Management. |
||
Step 3 |
Configure the rate mode (dedicated or shared). See the Configuring Rate Mode. |
||
Step 4 |
Configure the port mode. See the Configuring Interface Modes section.
|
||
Step 5 |
Configure the BB_credits and extended BB_credits, as necessary. See the Configuring Buffer-to-Buffer Credits section and the Configuring Extended BB_credits section. |
Task Flow for Migrating Interfaces from Dedicated Mode to Shared Mode
To configure the 4-Gbps and 8-Gbps Fibre Channel switching modules migrating from dedicated rate mode to shared rate mode, follow these steps:
Procedure
Step 1 |
Take unused interfaces out of service to release resources for other interfaces, if necessary. See the Taking Interfaces Out of Service. |
||
Step 2 |
Configure the BB_credits and extended BB_credits, as necessary. See the Configuring Buffer-to-Buffer Credits section, and the Extended BB_credits on Generation 1 Switching Modules section. |
||
Step 3 |
Configure the port mode. See the Configuring Interface Modes section.
|
||
Step 4 |
Configure the rate mode (dedicated or shared) to use. See the Configuring Rate Mode. |
||
Step 5 |
Configure the traffic speed (1 Gbps, 2 Gbps, 4 Gbps, or autosensing with a maximum of 2 Gbps or 4 Gbps) to use. See the Dynamic Bandwidth Management. |
Task Flow for Configuring 12-Port 4-Gbps Module Interfaces
The 12-port 4-Gbps switching modules support the following features:
- 1-Gbps, 2-Gbps, and 4-Gbps speed traffic
- Only dedicated rate mode
- ISL (E or TE) and Fx (F or FL) port modes
- Extended BB_credits
- Performance buffers
To configure 4-port 10-Gbps switching modules when starting with the default configuration, follow these steps:
Procedure
Step 1 |
Configure the traffic speed (1 Gbps, 2 Gbps, 4 Gbps, or autosensing with a maximum of 2 Gbps or 4 Gbps) to use. |
Step 2 |
Configure the port mode. See the Configuring Interface Modes section. |
Step 3 |
Configure the BB_credits, performance buffers, and extended BB_credits, as necessary. See the Configuring Buffer-to-Buffer Credits section, and the Configuring Extended BB_credits section. |
Task Flow for Configuring 4-Port 10-Gbps Module Interfaces
The 4-port 10-Gbps switching modules support the following features:
- Only 10-Gbps speed traffic
- Only dedicated rate mode
- ISL (E or TE) and F port modes
- Extended BB_credits
- Performance buffers
To configure 4-port 10-Gbps switching modules when starting with the default configuration, follow these steps:
Procedure
Step 1 |
Configure the port mode. See the Configuring Interface Modes section. |
Step 2 |
Configure the BB_credits, performance buffers, and extended BB_credits, as necessary. See the Configuring Buffer-to-Buffer Credits section, and the Configuring Extended BB_credits section. |
Reserving Bandwidth Quickly for the 8-Gbps Module Interfaces
Detailed Steps
To quickly reserve bandwidth for all the ports in the port groups on the Generation 3 Fibre Channel modules using the Device Manager, follow these steps:
Procedure
Step 1 |
In the Device Manager window, right-click the 8-Gbps Fibre Channel module. |
||||||||
Step 2 |
From the popup menu, select Bandwidth Reservation Config... |
||||||||
Step 3 |
In the Bandwidth Reservation Configuration dialog box that is displayed, choose a bandwidth reservation scheme. (Figure 2). Table 1 describes the default RateMode configuration schemes available in the Bandwidth Reservation Configuration dialog box for the 8-Gbps modules.
|
||||||||
Step 4 |
Click Apply . |
Configuring Port Speed
To configure dedicated bandwidth on an interface using DCNM-SAN, follow these steps:
Procedure
Step 1 |
From the Fabric pane, select a switch or select a group of switches (SAN, fabric, VSAN) from the Logical Domains pane. |
||||
Step 2 |
Expand Switches, expand FC Interfaces and select Physical from the Physical Attributes pane. You see the Physical > General tab in the Interfaces pane. |
||||
Step 3 |
Scroll until you see the row containing the switch and port you want to configure. |
||||
Step 4 |
Select auto, 1Gb, 4Gb, or autoMax2G from the Speed Admin column.
The auto parameter enables autosensing on the interface. The autoMax2G parameter enables autosensing on the interface with a maximum speed of 2 Gbps.
|
||||
Step 5 |
Click the Apply Changes icon. |
Configuring Rate Mode
To configure the rate mode (dedicated or shared) on an interface on a 4-Gbps or 8-Gbps Fibre Channel switching module using DCNM-SAN, follow these steps:
Procedure
Step 1 |
Select a switch from the Fabric pane, or select a group of switches (SAN, fabric, VSAN) from the Logical Domains pane. |
Step 2 |
Expand Switches > FC Interfaces and then select Physical from the Physical Attributes pane. You see the Physical > General tab in the Interfaces pane. |
Step 3 |
Scroll until you see the row containing the switch and port you want to configure. |
Step 4 |
Select dedicated or shared from the Rate Mode column. |
Step 5 |
Click the Apply Changes icon. |
Configuring Local Switching
Note |
We recommend that you shut down all of the ports on the module before you execute the local switching command.If local switching is enabled, then ports cannot be configured in dedicated mode. If there are dedicated ports and you enter the local switching command, a warning is displayed and the operation is prevented. |
Configuring Local Switching Using DCNM-SAN
To enable or disable local switching module using DCNM-SAN, follow these steps:
Procedure
Step 1 |
Choose Switches > Hardware . |
Step 2 |
Click the Module Config tab. You see the Module Config dialog box. |
Step 3 |
Select a module and from the LocalSwitchingMode drop-down list, select enabled or disabled . This step either enables or disables the local switching for the selected module. |
Step 4 |
Click Apply to save the changes. |
Configuring Local Switching Using Device Manager
To enable or disable local switching using Device Manager, follow these steps:
Procedure
Step 1 |
Right-click a module and select Configure . You see the Module dialog box. Click the Config tab. |
Step 2 |
Click the enabled or disabled radio button to enable or disable local switching in the selected module. |
Step 3 |
Click Apply to save the changes. |
Disabling Restrictions on Oversubscription Ratios Using DCNM-SAN
To disable restrictions on oversubscription ratios on multiple 48-port or 24-port 4-Gbps, or any 8-Gbps Fibre Channel switching modules using DCNM-SAN, follow these steps:
Procedure
Step 1 |
Choose Switches > Hardware . |
Step 2 |
Click the Module Config tab. You see the Module Config dialog box. |
Step 3 |
From the RateModeOversubscriptionLimit drop-down list, select disabled for each module for which you want to disable restrictions on oversubscription ratios. |
Step 4 |
Click Apply to save the changes. |
Disabling Restrictions on Oversubscription Ratios Using Device Manager
To disable restrictions on oversubscription ratios on a single 48-port or 24-port 4-Gbps, or any 8-Gbps Fibre Channel switching module using Device Manager, follow these steps:
Procedure
Step 1 |
Right-click a module and select Configure. You see the Module dialog box. |
Step 2 |
Click the disabled radio button to disable restrictions on oversubscription ratios. |
Step 3 |
Click Apply to save the changes. |
Enabling Restrictions on Oversubscription Ratios Using DCNM-SAN
To enable restrictions on over subscription ratios on multiple 48-port or 24-port 4-Gbps, or any 8-Gbps Fibre Channel switching modules using DCNM-SAN, follow these steps:
Procedure
Step 1 |
Choose Switches > Hardware . |
Step 2 |
Click the Module Config tab. You see the Module Config dialog box. |
Step 3 |
From the RateMode Oversubscription Limit drop-down list, select enabled for each module for which you want to enable restrictions on oversubscription ratios. |
Step 4 |
Click Apply to save the changes. |
Enabling Restrictions on Oversubscription Ratios Using Device Manager
To enable restrictions on over subscription ratios on a single 48-port or 24-port 4-Gbps, or any 8-Gbps Fibre Channel switching module using Device Manager, follow these steps:
Procedure
Step 1 |
Right-click a module and select Configure . You see the Module dialog box. |
Step 2 |
Click the enabled radio button to enable restrictions on oversubscription ratios. |
Step 3 |
Click Apply to save the changes. |
Enabling Bandwidth Fairness Using DCNM-SAN
To enable bandwidth fairness on multiple 48-port or 24-port 4-Gbps, or any 8-Gbps Fibre Channel switching modules using DCNM-SAN, follow these steps:
Procedure
Step 1 |
Choose Switches > Hardware . |
Step 2 |
Click the Module Config tab. You see the Module Config dialog box. |
Step 3 |
From the BandwidthFairness Admin drop-down list, select enable for each module for which you want to enable bandwidth fairness. |
Step 4 |
Click Apply to save the changes. |
Enabling Bandwidth Fairness Using Device Manager
To enable bandwidth fairness on a single 48-port or 24-port 4-Gbps Fibre Channel switching module using Device Manager, follow these steps:
Procedure
Step 1 |
Right-click a module and select Configure . You see the Module dialog box. |
Step 2 |
Click the enable radio button to enable bandwidth fairness. |
Step 3 |
Click Apply to save the changes. |
Disabling Bandwidth Fairness Using DCNM-SAN
To disable bandwidth fairness on multiple 48-port or 24-port 4-Gbps, or 8-Gbps Fibre Channel switching modules using DCNM-SAN, follow these steps:
Procedure
Step 1 |
Choose Switches > Hardware . |
Step 2 |
Click the Module Config tab. You see the Module Config dialog box. |
Step 3 |
From the BandwidthFairness Admin drop-down list, select disable for each module for which you want to disable bandwidth fairness. |
Step 4 |
Click Apply to save the changes. |
Disabling Bandwidth Fairness Using Device Manager
To disable bandwidth fairness on a single 48-port or 24-port 4-Gbps, or 8-Gbps Fibre Channel switching module using Device Manager, follow these steps:
Procedure
Step 1 |
Right-click a module and select Configure. You see the Module dialog box. |
Step 2 |
Click the disable radio button to disable bandwidth fairness. |
Step 3 |
Click Apply to save the changes. |
Taking Interfaces Out of Service
To take an interface out of service using DCNM-SAN, follow these steps:
Procedure
Step 1 |
Select a switch from the Fabric pane, or select a group of switches (SAN, fabric, VSAN) from the Logical Domains pane. |
Step 2 |
Expand Switches , and expand FC Interfaces > Physical in the Physical Attributes pane. |
Step 3 |
Click General tab. You see the General tab information in the Information pane. |
Step 4 |
Scroll down until you see the row containing the switch and port you want to configure. |
Step 5 |
Scroll right (if necessary) until you see the Status Service column. |
Step 6 |
Select in or out from the Status Service column. |
Step 7 |
Click the Apply Changes icon. |
Releasing Shared Resources in a Port Group
To release the shared resources for a port group using DCNM-SAN, follow these steps:
Procedure
Step 1 |
Select a switch from the Fabric pane, or select a group of switches (SAN, fabric, VSAN) from the Logical Domains pane. |
Step 2 |
Expand Switches , and expand FC Interfaces > Physical in the Physical Attributes pane. |
Step 3 |
Click General tab. You see the General tab information in the Information pane. |
Step 4 |
Scroll down until you see the row containing the switch and port you want to configure. |
Step 5 |
Scroll right (if necessary) until you see the Status Service column. |
Step 6 |
Select the status out from the Status Service column. |
Step 7 |
Click the Apply Changes icon. |
Step 8 |
Select the status in from the Status Service column. |
Step 9 |
Click the Apply Changes icon. |
Verifying Fibre Channel Interfaces Configuration
To display Fibre Channel interface configuration information, perform one of the following tasks:
Command |
Purpose |
---|---|
show module |
Displays the module. |
show module slot recovery-steps |
Displays the slot for the module. |
show port-resources module slot |
Displays the port resources for the slot. |
show interface fc slot/port |
Displays the slot or port information. |
show interface brief |
Displays the interface. |
show port index-allocation |
Displays the port in the index allocation. |
show port index-allocation startup |
Displays the startup port in the index allocation. |
show port-channel compatibility parameters |
Displays the PortChannel compatibility parameters. |
show module slot bandwidth-fairness |
Displays the module slot bandwidth fairness information. |
For detailed information about the fields in the output from these commands, refer to the Cisco MDS NX-OS Command Reference .
Displaying Diagnostics for Multiple Ports
To view diagnostic information for multiple ports using Device Manager, follow these steps:
Procedure
Step 1 |
Choose Interface > FC All and click the Diagnostics tab or hold down the Control key, and then click each port for which you want to view diagnostic information. |
Step 2 |
Right-click the selected ports, and then select Configure . You see the FC Interfaces dialog box. |
Step 3 |
Click Refresh to view the latest diagnostic information. To view diagnostic information for a single port using Device Manager, follow these steps: |