The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
This chapter describes the Inter-VSAN Routing (IVR) feature and provides basic instructions on sharing resources across VSANs using IVR management interfaces. After setting up a basic IVR configuration, see “Advanced Inter-VSAN Routing Configuration,” if you need to set up an advanced IVR configuration.
This chapter includes the following sections on IVR basic configuration:
Virtual SANs (VSANs) improve storage area network (SAN) scalability, availability, and security by allowing multiple Fibre Channel SANs to share a common physical infrastructure of switches and ISLs. These benefits are derived from the separation of Fibre Channel services in each VSAN and the isolation of traffic between VSANs. Data traffic isolation between the VSANs also inherently prevents sharing of resources attached to a VSAN, such as robotic tape libraries. Using IVR, you can access resources across VSANs without compromising other VSAN benefits.
This section includes the following topics:
IVR supports the following features:
Note Inter VSAN Routing (IVR) is supported in MDS 9148 switches from NX-OS 5.2.2 version.
Originator Exchange ID (OX ID) load balancing of IVR traffic from IVR-enabled switches is not supported on Generation 1 switching modules. OX ID-based load balancing of IVR traffic from a non-IVR MDS switch could work in some environments. Generation 2 switching modules support OX ID-based load balancing of IVR traffic from IVR-enabled switches.
Note To configure the sample scenario shown in Figure 1-1, follow the steps in IVR Auto Topology Mode Configuration Example.
Figure 1-1 Traffic Continuity Using IVR and FCIP
The following IVR-related terms are used in the IVR documentation:
Note An edge VSAN for one IVR path can be a transit VSAN for another IVR path.
Note When the source and destination edge VSANs are adjacent to each other, then a transit VSAN is not required between them.
Table 1-1 summarizes the configuration limits for IVR.
|
|
---|---|
As of Cisco SAN-OS Release 3.0(3), 20,000 IVR zone members per physical fabric Prior to Cisco SAN-OS Release 3.0(3), 10,000 IVR zone members per physical fabric |
|
As of Cisco SAN-OS Release 3.0(3), 8000 IVR zones per physical fabric Prior to Cisco SAN-OS Release 3.0(3), 2000 IVR zones per physical fabric |
|
Note We recommend IVR manual topology mode if you have more than 25 IVR switches. See Manually Configuring an IVR Topology. |
IVR virtualizes the remote end devices in the native VSAN using a virtual domain. When IVR is configured to link end devices in two disparate VSANs, the IVR border switches are responsible for modifying the Fibre Channel headers for all communication between the end devices. The sections of the Fibre Channel frame headers that are modified include:
When a frame travels from the initiator to the target, the Fibre Channel frame header is modified such that the initiator VSAN number is changed to the target VSAN number. If IVR Network Address Translation (NAT) is enabled, then the source and destination FCIDs are also translated at the edge border switch. If IVR NAT is not enabled, then you must configure unique domain IDs for all switches involved in the IVR path.
IVR Network Address Translation (NAT) can be enabled to allow non-unique domain IDs; however, without NAT, IVR requires unique domain IDs for all switches in the fabric. IVR NAT simplifies the deployment of IVR in an existing fabric where non-unique domain IDs might be present.
To use IVR NAT, it must be enabled on all IVR-enabled switches in the fabric. For information on distributing the IVR configuring using CFS, see Distributing the IVR Configuration Using CFS. By default, IVR NAT and IVR configuration distributions are disabled on all switches in the Cisco MDS 9000 Family.
See About IVR NAT and IVR Auto Topology Mode for information on IVR requirements and guidelines as well as configuration information.
IVR uses a configured IVR VSAN topology to determine how to route traffic between the initiator and the target across the fabric.
IVR auto topology mode automatically builds the IVR VSAN topology and maintains the topology database when fabric reconfigurations occur. IVR auto topology mode also distributes the IVR VSAN topology to IVR-enabled switches using CFS.
Using IVR auto topology mode, you no longer need to manually update the IVR VSAN topology when reconfigurations occur in your fabric. If an IVR manual topology database exists, IVR auto topology mode initially uses that topology information. The automatic update reduces disruption in the network by gradually migrating from the user-specified topology database to the automatically-learned topology database. User-configured topology entries that are not part of the network are aged out in about three minutes. New entries that are not part of the user-configured database are added as they are discovered in the network.
When IVR auto topology mode is enabled, it starts with the previously active IVR manual topology if it exists, and then the discovery process begins. New, alternate, or better paths my be discovered. If the traffic is switched to an alternate or better path, there may be temporary traffic disruptions that are normally associated with switching paths.
Note IVR topology in IVR auto topology mode requires Cisco MDS SAN-OS Release 2.1(1a) or later and CFS must be enabled for IVR on all switches in the fabric.
When using the IVR feature, all border switches in a fabric must be Cisco MDS switches. However, other switches in the fabric may be non-MDS switches. For example, end devices that are members of the active IVR zone set may be connected to non-MDS switches. Non-MDS switches may also be present in the transit VSAN(s) or in the edge VSANs if one of the interop modes is enabled.
For additional information on switch interoperability, refer to the Cisco Data Center Interoperability Support Matrix.
To configure IVR, follow these steps:
|
|
|
---|---|---|
See Enabling IVR. |
||
Note The following steps need to be performed on one switch in the fabric. |
||
This section describes how to configure IVR and contains the following sections:
The IVR Zone Wizard simplifies the process of configuring IVR zones in a fabric. The IVR Zone Wizard checks the following conditions and identifies any related issues:
The IVR feature must be enabled in all border switches in the fabric that participate in the IVR. By default, this feature is disabled in all Cisco MDS 9000 Family switches. You can manually enable IVR on all required switches in the fabric or configure fabric-wide distribution of the IVR configuration. See Distributing the IVR Configuration Using CFS.
Note The configuration and verification commands for the IVR feature are only available when IVR is enabled on a switch. When you disable this configuration, all related configurations are automatically discarded.
To enable IVR on any participating switch, follow these steps:
|
|
|
---|---|---|
The IVR feature uses the Cisco Fabric Services (CFS) infrastructure to enable efficient configuration management and to provide a single point of configuration for the entire fabric in the VSAN. For information on CFS, refer to the Cisco MDS 9000 Family NX-OS System Management Configuration Guide.
The following configurations are distributed:
Note IVR configuration distribution is disabled by default. For the feature to function correctly, you must enable it on all IVR-enabled switches in the network.
The IVR feature uses three databases to accept and implement configurations.
To enable IVR configuration distribution, follow these steps:
|
|
|
---|---|---|
The first action that modifies the database creates the pending database and locks the feature in the VSAN. Once you lock the fabric, the following situations apply:
If you commit the changes made to the active database, the configuration is committed to all the switches in the fabric. On a successful commit, the configuration change is applied throughout the fabric and the lock is released.
To commit IVR configuration changes, follow these steps:
|
|
|
---|---|---|
If you discard (abort) the changes made to the pending database, the configuration database remains unaffected and the lock is released.
To discard IVR configuration changes, follow these steps:
|
|
|
---|---|---|
Discards the IVR changes and clears the pending configuration database. |
If you have performed an IVR task and have forgotten to release the lock by either committing or discarding the changes, an administrator can release the lock from any switch in the fabric. If the administrator performs this task, your changes to the pending database are discarded and the fabric lock is released.
Tip The pending database is only available in the volatile directory and is subject to being discarded if the switch is restarted.
To use administrative privileges and release a locked DPVM session, use the clear ivr session command in EXEC mode.
Before configuring an IVR SAN fabric to use IVR NAT and IVR auto topology mode, consider the following:
Note The IVR over FCIP feature is bundled with the Cisco MDS 9216i Switch and does not require the SAN extension over IP package for the fixed IP ports on the supervisor module.
Tip If you change any FSPF link cost, ensure that the FSPF path distance (that is, the sum of the link costs on the path) of any IVR path is less than 30,000.
Note IVR-enabled VSANs can be configured when the interop mode is enabled (any interop mode) or disabled (no interop mode).
The requirements and guidelines for using IVR NAT are listed below:
|
|
|
---|---|---|
Consider the following guidelines for transit VSANs:
– If two edge VSANs in an IVR zone overlap, then a transit VSAN is not required (though, not prohibited) to provide connectivity.
– If two edge VSANs in an IVR zone do not overlap, you may need one or more transit VSANs to provide connectivity. Two edge VSANs in an IVR zone will not overlap if IVR is not enabled on a switch that is a member of both the source and destination edge VSANs.
Before configuring border switches, consider the following guidelines:
This section includes instructions on how to enable IVR NAT and how to enable IVR auto topology mode.
To enable IVR NAT, follow these steps:
|
|
|
---|---|---|
Note IVR configuration distribution must be enabled before configuring IVR auto topology mode (see Distributing the IVR Configuration Using CFS). Once IVR auto topology mode is enabled, you cannot disable IVR configuration distribution.
To enable IVR auto topology mode, follow these steps:
|
|
|
---|---|---|
To view an automatically discovered IVR topology, use the show ivr vsan-topology command.
Note The asterisk (*) indicates the local switch.
In a remote VSAN, the IVR application does not automatically add the virtual domain to the assigned domains list. Some switches (for example, the Cisco SN5428 switch) do not query the remote name server until the remote domain appears in the assigned domains list in the fabric. In such cases, add the IVR virtual domains in a specific VSAN to the assigned domains list in that VSAN. When adding IVR domains, all IVR virtual domains that are currently present in the fabric (and any virtual domain that is created in the future) will appear in the assigned domains list for that VSAN.
Tip Be sure to add IVR virtual domains if Cisco SN5428 or MDS 9020 switches exist in the VSAN.
When you enable the IVR virtual domains, links may fail to come up due to overlapping virtual domain identifiers. If this occurs, temporarily withdraw the overlapping virtual domain from that VSAN.
Note Withdrawing an overlapping virtual domain from an IVR VSAN disrupts IVR traffic to and from that domain.
Use the ivr withdraw domain command in EXEC mode to temporarily withdraw the overlapping virtual domain interfaces from the affected VSAN.
Tip Only add IVR domains in the edge VSANs and not in transit VSANs.
To manually configure an IVR virtual domain in a specified VSAN, follow these steps:
Note As of Cisco SAN-OS Release 3.1(2), Cisco Fabric Configuration Services (FCS) supports the discovery of virtual devices. The fcs virtual-device-add vsan-ranges command, issued in FCS configuration submode, allows you to discover virtual devices in a particular VSAN or in all VSANs. To discover the devices that are zoned for IVR using this command, the devices must have request domain_ID (RDI) enabled. For information on using FCS, refer to the Cisco MDS 9000 Family NX-OS System Management Configuration Guide.
To configure fabric-wide IVR virtual domains in a specified VSAN, follow these steps:
To view the status of the IVR virtual domain configuration, use the show ivr virtual-fcdomain-add-status command.
To clear the IVR fcdomain database, use the following command:
This section describes configuring IVR zones and IVR zone sets and includes the following topics:
As part of the IVR configuration, you need to configure one or more IVR zones to enable cross-VSAN communication. To achieve this result, you must specify each IVR zone as a set of (pWWN, VSAN) entries. Like zones, several IVR zone sets can be configured to belong to an IVR zone. You can define several IVR zone sets and activate only one of the defined IVR zone sets.
Note The same IVR zone set must be activated on all of the IVR-enabled switches.
Table 1-3 identifies the key differences between IVR zones and zones.
|
|
---|---|
IVR zone membership is specified using the VSAN and pWWN combination. |
Zone membership is specified using pWWN, fabric WWN, sWWN, or the AFID. |
Table 1-4 identifies the IVR zone limits per physical fabric.
|
|
|
|
---|---|---|---|
Note A zone member is counted twice if it exists in two zones. See Database Merge Guidelines.
Figure 1-2 depicts an IVR zone consisting of four members. To allow pwwn1 to communicate with pwwn2, they must be in the same zone in VSAN 1, as well as in VSAN 2. If they are not in the same zone, then the hard-zoning ACL entries will prohibit pwwn1 from communicating with pwwn2.
A zone corresponding to each active IVR zone is automatically created in each edge VSAN specified in the active IVR zone. All pWWNs in the IVR zone are members of these zones in each VSAN.
Figure 1-2 Creating Zones Upon IVR Zone Activation
The zones are created automatically by the IVR process when an IVR zone set is activated. They are not stored in a full zone set database and are lost when the switch reboots or when a new zone set is activated. The IVR feature monitors these events and adds the zones corresponding to the active IVR zone set configuration when a new zone set is activated. Like zone sets, IVR zone sets are also activated nondisruptively.
Note If pwwn1 and pwwn2 are in an IVR zone in the current as well as the new IVR zone set, then activation of the new IVR zone set does not cause any traffic disruption between them.
IVR zone and IVR zone set names are restricted to 64 alphanumeric characters.
To create IVR zones and IVR zone sets, follow these steps:
Once the zone sets have been created and populated, you must activate the zone set. When you activate an IVR zone set, IVR automatically adds an IVR zone to the regular active zone set of each edge VSAN. If a VSAN does not have an active zone set, IVR can only activate an IVR zone set using the force option, which causes IVR to create an active zone set called “nozoneset” and adds the IVR zone to that active zone set.
Note If IVR and iSLB are enabled in the same fabric, at least one switch in the fabric must have both features enabled. Any zoning-related configuration or activation operation (for normal zones, IVR zones, or iSLB zones) must be performed on this switch. Otherwise, traffic might be disrupted in the fabric.
You can also use the force command to activate IVR zone sets. Table 1-5 lists the various scenarios with and without the force command option.
|
|
|
|
|
|
|
---|---|---|---|---|---|---|
31 |
||||||
|
To activate or deactivate an existing IVR zone set, follow these steps:
|
|
|
---|---|---|
switch(config)# ivr zoneset activate name IVR_ZoneSet1 force |
||
Note To replace the active IVR zone set with a new IVR zone set without disrupting traffic, activate the new IVR zone set without deactivating the current active IVR zone set.
Verify the IVR zone and IVR zone set configurations using the show ivr zone and show ivr zoneset commands. See Example 1-1 to Example 1-9.
Example 1-1 Displays the IVR Zone Configuration
Example 1-2 Displays Information for a Specified IVR Zone
Example 1-3 Displays the Specified Zone in the Active IVR Zone
Example 1-4 Displays the IVR Zone Set Configuration
Example 1-5 Displays the Active IVR Zone Set Configuration
Example 1-6 Displays the Specified IVR Zone Set Configuration
Example 1-7 Displays Brief Information for All IVR Zone Sets
Example 1-8 Displays Brief Information for the Active IVR Zone Set
Example 1-9 Displays Status Information for the IVR Zone Set
Tip Repeat this configuration in all border switches participating in the IVR configuration.
Note You can use Cisco Fabric Manager to distribute IVR zone configurations to all IVR-capable switches in the interconnected VSAN network. Refer to the Cisco Fabric Manager Inter-VSAN Routing Configuration Guide.
Clearing a zone set only erases the configured zone database, not the active zone database.
To clear the IVR zone database, use the clear ivr zone database command.
This command clears all configured IVR zone information.
Note After issuing a clear ivr zone database command, you need to explicitly issue the copy running-config startup-config command to ensure that the running configuration is used when you next start the switch.
You can configure Telnet or SSH logging for the IVR feature. For example, if you configure the IVR logging level at level 4 (warning), then messages with a severity level of 4 or above are displayed. Use the instructions in this section to configure and verify the logging levels:
To configure the severity level for logging messages from the IVR feature, follow these steps:
|
|
|
---|---|---|
Configures Telnet or SSH logging for the IVR feature at level 4 (warning). As a result, logging messages with a severity level of 4 or above are displayed. |
Use the show logging level command to view the configured logging level for the IVR feature.
A database merge refers to the combination of the configuration database and static (unlearned) entries in the active database. For information on CFS merge support, refer to the Cisco MDS 9000 Family NX-OS System Management Configuration Guide or Cisco Fabric Manager System Management Configuration Guide.
Consider the following when merging two IVR fabrics:
Figure 1-3 Fabric Merge Consequences
– The configurations are merged even if both fabrics have different configurations.
– A combination of zones and zone sets are used to get the merged zones and zone sets. If a dissimilar zone exists in two fabrics, the dissimilar zones are cloned into the zone set with appropriate names so both zones are present.
– The merged topology contains a combination of the topology entries for both fabrics.
– The merge will fail if the merged database contains more topology entries than the allowed maximum.
– The total number of VSANs across the two fabrics cannot exceed 128.
Note VSANs with the same VSAN ID but different AFIDs are counted as two separate VSANs.
– The total number of IVR-enabled switches across the two fabrics cannot exceed 128.
– The total number of zone members across the two fabrics cannot exceed 10,000. As of Cisco SAN-OS Release 3.0(3), the total number of zone members across the two fabrics cannot exceed 20,000. A zone member is counted twice if it exists in two zones.
Note If one or more of the fabric switches are running Cisco SAN-OS Release 3.0(3) or later, and the number of zone members exceeds 10,000, you must either reduce the number of zone members in the fabric or upgrade all switches in both fabrics to Cisco SAN-OS Release 3.0(3) or later.
– The total number of zones across the two fabrics cannot exceed 2000. As of Cisco SAN-OS Release 3.0(3), the total number of zones across the two fabrics cannot exceed 8000.
Note If only some of the switches in the fabrics are running Cisco SAN-OS Release 3.0(3) or later, and if the number of zones exceeds 2000, you must either reduce the number of zones in the fabric or upgrade all switches in both fabrics to Cisco SAN-OS Release 3.0(3) or later.
– The total number or zone sets across the two fabrics cannot exceed 32.
Table 1-6 describes the results of a CFS merge of two IVR-enabled fabrics under different conditions.
|
|
|
---|---|---|
Combined configuration exceeds limits (such as maximum number of zones or VSANs) |
||
User-configured VSAN topology configuration without conflicts |
If a merge failure occurs, you can use the following CLI commands to display the error conditions:
To resolve merge failures, review the failure information indicated in the show command outputs, then find the scenario in this list that relates to the failure and follow the troubleshooting instructions:
Note After a successful CFS commit, the merge will be successful.
This section provides example configuration steps for enabling IVR auto topology mode.
Step 1 Enable IVR on every border switch in the fabric.
Step 2 Verify that IVR is enabled on every IVR-enabled switch.
Step 3 Enable CFS distribution on every IVR-enabled switch in the fabric.
Step 4 Enable IVR auto topology mode.
Step 5 Commit the change to the fabric.
Step 6 Verify the status of the commit request.
Step 7 Verify the active IVR auto topology.
Step 8 Configure IVR zone set and zones. Two zones are required:
Tip Instead of creating two IVR zones, you can also create one IVR zone with the tape and both servers.
Step 9 View the IVR zone configuration to confirm that the IVR zone set and IVR zones are properly configured.
Step 10 View the zone set prior to IVR zone set activation. Prior to activating the IVR zone set, view the active zone set. Repeat this step for VSANs 2 and 3.
Step 11 Activate the configured IVR zone set.
Step 12 Verify the IVR zone set activation.
Step 13 Verify the zone set updates. Upon successful IVR zone set activation, verify that appropriate zones are added to the active zone set. Repeat this step for VSANs 2 and 3.
Table 1-7 lists the default settings for IVR parameters.
|
|
---|---|