Connecting Cisco MDS 9000 Family in Heterogeneous Fabrics
PDF(186.8 KB) View with Adobe Reader on a variety of devices
Updated:Jun 14, 2006
Ease-of-migration and investment protection are critical requirements for a storage administrator planning to evolve multiple isolated SAN islands into an enterprisewide fabric. Specifically, a storage administrator should be able to build a heterogeneous fabric with SAN switches from multiple vendors. On the surface it sounds like a trivial task. There should be no problem if vendors adhere to the ANSI T11 FC-SW-2 specification for Fibre Channel switch interoperability.
Unfortunately, not all switch vendors strictly adhere to this standard, resulting in switch interoperability issues that prevent customers from building heterogeneous fabrics. Switch vendors support a special mode to address this issue, allowing them to interoperate with other switches. However, the process of enabling the interoperability mode on existing switches results in fabricwide disruption and loss of existing functionality. For customers deploying SANs in mission-critical environments, this may not an acceptable solution.
Starting with Cisco
® MDS 9000 SAN-OS Software Release 1.2(1) and later, Cisco MDS 9000 Family multilayer directors and fabric switches interoperate with an installed base of specific switches without disrupting the existing fabric services or losing functionality. This functionality allows storage administrators to consolidate SAN islands while preserving their investments in existing switches.
Standard Interoperability Modes
Both Brocade and McDATA switches support "interop" and "open" modes respectively, which when enabled, allow them to interoperate with each other and Cisco MDS 9000 switches. These special modes are in addition to the regular modes (enabled by default) in which the proprietary features are supported. These special modes result in the following when enabled:
• Fabric disruption-The interoperability mode has to be enabled on each Brocade and McDATA switch in the fabric. When enabled, this mode causes a switch to either reset (Brocade) or restart (McDATA). For customers deploying SANs in mission-critical environments this is not an acceptable solution.
• Loss of functionality-Some of the features that are available prior to enabling the interoperability mode are subsequently disabled. For example, when "interop" mode is enabled on Brocade switches, it disables domain/port-based zoning, Virtual Channel flow control, and trunking, etc. This creates operational challenges for storage administrators who have to give up functionality in order to build heterogeneous fabrics.
Cisco MDS 9000 SAN-OS Software Release 1.0(1) and later support an interoperability mode (mode 1) that allows Cisco MDS 9000 switches to interoperate with Brocade and McDATA switches configured in "interop" and "open" modes respectively. Unlike Brocade and McDATA modes, the Cisco MDS 9000 interoperability mode (mode 1) can be enabled on a per-virtual SAN (VSAN) basis without resetting or restarting the Cisco MDS 9000 switches, and it does not impact the features and functionality of other VSANs.
Functionality With Standard Interoperability Modes
Connecting a Cisco MDS 9000 switch to a fabric with other vendor's switches using Cisco MDS 9000 interop mode 1 requires enabling an interoperability mode on all the other vendor's switches. However, using this solution causes fabric disruption and loss of functionality in the other vendor's switches within the fabric.
To address this issue, Cisco MDS 9000 switches starting with Cisco MDS 9000 SAN-OS Software Release 1.2(1) and later offer additional modes termed as "legacy switch" interop modes. By enabling legacy switch modes on Cisco MDS 9000 switches, customers are able to nondisruptively connect a Cisco MDS 9000 switch to an older fabric without changing any configuration on the existing switches.
In order to preserve the integrity, stability, and quality of customers' mission-critical storage networks, the scope of the legacy switch modes (interop modes 2 through 4) is limited to a specific set of fabric switches running a given firmware release.
After connecting a Cisco MDS 9000 switch to a legacy switch, certain features on the legacy switch are not available:
• Brocade's proprietary flow control called Virtual Channel (VC) flow control is not available on links connecting to the Cisco MDS 9000. VC flow control still is available on the links between Brocade switches.
• Brocade trunking is not available on links connecting to a Cisco MDS 9000, but it is available on the links between Brocade switches. Customers can deploy multiple Inter-Switch Links (ISLs) between a Cisco MDS 9000 and Brocade switch. The traffic on these ISLs is load balanced by Exchange or S_ID / D_ID.
• Quickloop feature is only allowed between two Brocade switches; the Cisco MDS 9000 cannot be part of Quickloop.
• Fabric-Assisted Zoning is not supported on end devices (hosts and storage) connected to a Cisco MDS 9000.
• Brocade Fabric Manager is able to discover Cisco MDS 9000 switches but it is unable to launch Cisco Fabric Manager. However, the Cisco Fabric Manager is able to discover Brocade switches and launch Brocade Fabric Manager for managing Brocade switches.
Cisco MDS 9000 Interop Mode Descriptions
The Cisco MDS 9000 Family interop modes include the following:
• Default or native mode-This is the default mode or behavior for a VSAN that is communicating between a SAN composed entirely of Cisco MDS 9000 switches. It is available starting in Cisco MDS 9000 SAN-OS 1.0(1).
• Interop mode 1-This is the Fibre Channel standards-based interoperability mode. It interoperates with Brocade and McDATA switches that have been configured for their own interoperability modes. Brocade and McDATA switches must be running in interop mode to work with this Cisco MDS 9000 interop mode. Available starting in Cisco MDS 9000 SAN-OS 1.0(1).
• Interop mode 2-This mode, also known as legacy switch interop mode 2, allows easy integration with specific Brocade switches running in their own native mode of operation. Brocade switches must be configured with "core pid = 0" to work with this mode. Available starting in Cisco MDS 9000 SAN-OS 1.2(2).
• Interop mode 3-Similar to interop mode 2, interoperability mode 3 was introduced for Brocade switches that contained more than 16 ports. With this VSAN-based interop mode, Brocade switches do not have to be altered from their native mode (core pid = 1) and can be easily added to a new or existing VSAN. This mode is also known as legacy switch interop mode 3. Available starting in Cisco MDS 9000 SAN-OS 1.3(1).
• Interop mode 4-This mode, also known as legacy switch interop mode 4, provides easy integration between VSANs and McDATA switches running in McDATA Fabric 1.0 interop mode. Available starting in Cisco MDS 9000 SAN-OS 3.0(1).
Switch Interoperability Example
VSANs improve Fibre Channel SAN scalability, availability, and security by allowing multiple logical SANs to share a common physical infrastructure of switches and ISLs. These benefits are derived from the separation of Fibre Channel services in each VSAN and isolation of control and data traffic between VSANs.
Data traffic can still be transported between specific initiators and targets on different VSANs using Inter-VSAN Routing (IVR) without merging VSANs into a single logical fabric. Fibre Channel control traffic does not flow between VSANs, nor can initiators access any resources aside from the ones designated with IVR. Valuable resources such as tape libraries can be easily shared without compromise. IVR also can be used in conjunction with Fibre Channel Interface Protocol (FCIP) to create more efficient business-continuity and disaster-recovery solutions.
Figure 1 illustrates an example of Cisco MDS 9000 interoperability. VSAN functionality along with IVR can be enabled on Cisco MDS 9000 switches to interoperate with existing Brocade and McDATA fabrics. In this configuration, hosts and storage devices attached to any of the fabrics can be configured to access each other.
Figure 1. Cisco MDS 9000 Interoperability
• VSAN B on the Cisco MDS 9000 is assigned to Brocade fabric. The interop mode of VSAN B depends on customer's operational environment and requirements. It should be noted that VSAN B can be configured with interop modes 1, 2, or 3.
• VSAN M on the Cisco MDS 9000 is assigned to McDATA fabric. VSAN M can be configured with interop mode 1 or 4.
• VSAN C on the Cisco MDS 9000 is assigned to a Cisco fabric. VSAN C does not require any interop mode.
• IVR is used to route traffic between various fabrics.
Note: Please refer to the Cisco MDS 9000 Family Switch-to-Switch Interoperability Configuration Guide for a detailed technical description including configuration steps.
Cisco MDS 9000 Family switches support a variety of interoperability modes, allowing storage administrators to build a heterogeneous fabric with SAN switches from multiple vendors. These interop modes provide ease of migration and investment protection. Cisco Systems
® has tested and qualified these interop modes to help ensure that the integrity, stability, and quality of customers' mission-critical storage networks are not compromised. It is recommended that customers work with their storage subsystem vendor to assess compatibility and obtain a support matrix.