The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
This chapter provides an overview of the basic configurations that need to be completed before getting started with IOA-specific configurations:
SSH needs to be enabled on all the IOA switches for Cisco DCNM-SAN to provision IOA. By default, the SSH service is enabled with the RSA key.
CFS must be enabled on the IOA switches as well as those switches of which the hosts and targets are directly connected to. FC-Redirect internally uses CFS to configure the rules for any given flow in the fabric.
Cluster communication requires the use of the Management interface. IP ACL configurations must allow UDP and TCP traffic on ports 9333, 9334, 9335, and 9336.
For FC-Redirect to work correctly, the default zone policy on all the switches in the IOA environment must be configured to deny and the initiator-target pairs must be configured in user-defined zones.
FC-Redirect
FC-Redirect (FCR) is supported on the following switches and is enabled by default:
Note | Starting from Cisco MDS NX-OS Release 7.3(0)DY(1), FCR is supported on Cisco MDS 9148S and Cisco MDS 9396S switches. |
IOA does not support any FCoE connected devices including devices connected through any of the MDS FCoE linecards.
Starting from Cisco MDS NX-OS Release 5.2(x), you cannot install an FCoE module in a switch that is running DMM, SME, or IOA.
FC-Redirect is not supported on the following switches, which also means that IOA is not supported:
FC-Redirect requirements for IOA include the following:
Note | FCR support for IVR is disabled by default. If FCR support for IVR is enabled on one switch in a fabric, it gets propagated through the entire fabric and is enabled on all the switches. We recommend that you disable FCR support for IVR using the no fc-redirect ivr-support enable command in global configuration mode. |
To enable the v2 mode in FC-Redirect, use the fc-redirect version2 enable command in configuration mode. To disable the v2 mode in FC-Redirect, use the no form of the command.
This command is used to increase scalability of FC-Redirect. Disabling v2 mode after it is enabled in the fabric is not recommended. However, if you want to disable v2 mode, you cannot disable it until all FC-Redirect configurations are deleted. FC-Redirect configurations can be deleted only by deleting all corresponding application configurations.
The MDS switches not running Cisco NX-OS 3.3(1c) and later cannot be added to the fabric after the v2 mode is enabled. If the switches are added, all further FC-Redirect configuration changes will fail across the fabric. This could lead to traffic disruption for applications such as IOA, SME, and DMM.
Use the show fc-redirect configs command to see the list of applications that create FC-Redirect configurations.
If v2 mode is enabled in the fabric and you want to move a switch to a different fabric, use the clear fc-redirect decommission-switch command before moving the switch to a different fabric. If the mode is not enabled, all switches in the new fabric will be converted to v2 mode automatically.
Note | Ensure that there are no fabric changes or upgrades in progress. For more information see Software Requirements. Use the show fc-redirect peer-switches command (UP state) to see all the switches in the fabric. |
The FC-Redirect feature uses Cisco Fabric Services (CFS) regions to distribute the FC-Redirect configuration. By default, the configuration is propagated to all FC-Redirect-capable switches in the fabric. CFS regions can be used to restrict the distribution of the FC-Redirect configuration.
Note | Using FC Redirect with CFS regions is an optional configuration only if the number of switches in the SAN exceeds the scalability limit supported by IOA. As of MDS NX-OS Release 4.2(1), the number of switches supported in a fabric is 34. |
To learn more about CFS regions, refer to the Cisco MDS 9000 Family NX-OS System Management Configuration Guide.
To design CFS regions for FC-Redirect, follow these guidelines:
To configure the CFS regions for FC-Redirect, do the following tasks:
Internet protocol over Fibre Channel (IPFC) provides IP forwarding or in-band switch management over a Fibre Channel interface (instead of management using the Gigabit Ethernet mgmt 0 interface). You can use IPFC to specify that IP frames be transported over Fibre Channel using encapsulation techniques. IP frames are encapsulated into Fibre Channel frames so that cluster management information can transmit across the Fibre Channel network without using an overlay Ethernet network.
When an IOA cluster communicates via the IPFC interface, the cluster management messages can be sent and received on Fibre Channel ISLs by encapsulating cluster management messages in Fibre Channel frames instead of using the management interface.
To configure IOA cluster using the IPFC interface, follow these steps:
The process of configuring an IOA cluster to use the IPFC interface involves a number of configuration tasks that should be completed in the following order:
The first step in the process of configuring an IOA cluster to use the IPFC interface is to create a VSAN interface and configure IPv4 addresses.
To create an interface VSAN, perform this task:
After creating the VSAN and configuring the IPv4 address, use the show interface vsan command to verify the configuration:
sw-231-14# show interface vsan 1 vsan1 is up, line protocol is up WWPN is 10:00:00:0d:ec:18:a1:05, FCID is 0xec03c0 Internet address is 10.1.1.1/24 MTU 1500 bytes, BW 1000000 Kbit 0 packets input, 0 bytes, 0 errors, 0 multicast 6 packets output, 384 bytes, 0 errors, 0 dropped sw-231-14#
To enable IPv4 routing, perform this task:
After enabling IPv4 routing, use the show ip routing to verify the configuration.
sw-231-14(config)# show ip routing ip routing is enabled
To verify the connectivity, use the show ip route and ping commands.
sw-231-14# show ip route Codes: C - connected, S - static C 10.1.1.0/24 is directly connected, vsan1 sw-231-14# ping 10.1.1.2 PING 10.1.1.2 (10.1.1.2) 56(84) bytes of data. 64 bytes from 10.1.1.2: icmp_seq=1 ttl=64 time=0.875 ms 64 bytes from 10.1.1.2: icmp_seq=2 ttl=64 time=0.866 ms 64 bytes from 10.1.1.2: icmp_seq=3 ttl=64 time=0.884 ms 64 bytes from 10.1.1.2: icmp_seq=4 ttl=64 time=0.875 ms --- 10.1.1.2 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3023ms rtt min/avg/max/mdev = 0.866/0.875/0.884/0.006 ms
To create an IOA cluster and IOA interface in the local node, perform this task:
Note | To configure an IOA cluster, you can use the name of the switch if the network supports DNS service. The IOA cluster requires switch name to IP address resolution. |
To verify the cluster configuration, use the show ioa cluster name node summary command.
sw-231-14# sh ioa cluster cltr1 node sum ------------------------------------------------------------------------------- Switch Site Status Master Node ID ------------------------------------------------------------------------------- sw-231-14(L) site2 online yes 1
To verify the IP address of the node, use the show ioa cluster name node command.
sw-231-14# show ioa cluster cltr1 node Node sw-231-14 is local switch Node ID is 1 IP address is 10.1.1.1 Status is online Belongs to Site site2 Node is the master switch
To add a remote node, perform this task:
To verify the node configuration, use the show ioa cluster name node summary command:
sw-231-14# show ioa cluster cltr1 node summary ------------------------------------------------------------------------------- Switch Site Status Master Node ID ------------------------------------------------------------------------------- sw-231-14(L) site2 online yes 1 sw-231-19 site1 online no 2
To verify the IP address of the node, use the show ioa cluster name node command:
Node sw-231-14 is local switch Node ID is 1 IP address is 10.1.1.1 Status is online Belongs to Site site2 Node is the master switch Node sw-231-19 is remote switch Node ID is 2 IP address is 10.1.1.2 Status is online Belongs to Site site1 Node is not master switch sw-231-14#
To see all of the configured interfaces in the IOA cluster, use the show ioa cluster name interface summary command:
sw-231-14# show ioa cluster cltr1 interface summary ------------------------------------------------------------------------------- Switch Interface Status Flows ------------------------------------------------------------------------------- sw-231-14(L) ioa1/1 up 0 sw-231-14(L) ioa1/2 up 0 sw-231-19 ioa4/1 up 0 sw-231-19 ioa4/2 up 0 sw-231-14#
This section includes an example for creating an IOA cluster using IPFC interface. Figure 1 illustrates the IOA cluster configuration used in this example. The sample topology shows the FC ISL between sw-231-14 and sw-231-19 switches.
This Image 310152.jpg is not available in preview/cisco.com
The following example creates an interface VSAN and configures IP address on sw-231-14 and enables IP routing:
sw-231-14(config)# int vsan 1 sw-231-14(config-if)# ip address 10.1.1.1 255.255.255.0 sw-231-14(config-if)# no shut sw-231-14(config-if)# exit sw-231-14(config)# ip routing sw-231-14(config)#
The following example creates an interface VSAN and configure IP address on sw-231-19 and enables IP routing.
sw-231-19(config)# int vsan 1 sw-231-19(config-if)# ip address 10.1.1.12 255.255.255.0 sw-231-19(config-if)# no shut sw-231-19(config-if)# exit sw-231-19(config)# ip routing
The following example verifies the configuration of sw-231-14 using the show interface command.
sw-231-14# show interface vsan 1 vsan1 is up, line protocol is up WWPN is 10:00:00:0d:ec:18:a1:05, FCID is 0xec03c0 Internet address is 10.1.1.1/24 MTU 1500 bytes, BW 1000000 Kbit 758 packets input, 110841 bytes, 0 errors, 42 multicast 651 packets output, 122577 bytes, 0 errors, 0 dropped sw-231-14#
The following example verifies the configuration of sw-231-19 using the show interface command:
sw-231-19# show interface vsan 1 vsan1 is up, line protocol is up WWPN is 10:00:00:05:30:01:9f:09, FCID is 0xc60000 Internet address is 10.1.1.2/24 MTU 1500 bytes, BW 1000000 Kbit 675 packets input, 124613 bytes, 0 errors, 36 multicast 755 packets output, 111785 bytes, 0 errors, 0 dropped sw-231-19#
The following example verifies the connectivity using the ping command:
sw-231-14# ping 10.1.1.2 PING 10.1.1.2 (10.1.1.2) 56(84) bytes of data. 64 bytes from 10.1.1.2: icmp_seq=1 ttl=64 time=0.868 ms 64 bytes from 10.1.1.2: icmp_seq=2 ttl=64 time=0.898 ms 64 bytes from 10.1.1.2: icmp_seq=3 ttl=64 time=0.906 ms --- 10.1.1.2 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2017ms rtt min/avg/max/mdev = 0.868/0.890/0.906/0.038 ms sw-231-14#
The following example configures IOA site on switch sw-231-14:
sw-231-14(config)# ioa site-local site2 sw-231-14(config)#
The following example configures IOA site on switch sw-231-19:
sw-231-19(config)# ioa site-local site1 sw-231-19(config)#
The following example configures IOA cluster ctrl1 on switch sw-231-14:
sw-231-14(config)# ioa cluster cltr1 2011 Apr 8 05:00:46 sw-231-14 %CLUSTER-2-CLUSTER_LEADER_ANNOUNCE: Node 0x1 is the new Master of cluster 0x2e05000dec18a133 of 1 nodes 2011 Apr 8 05:00:46 sw-231-14 %CLUSTER-2-CLUSTER_QUORUM_GAIN: Cluster 0x2e05000dec18a133 now has quorum with 1 nodes
The following example forces the node to use IPFC interface addresss:
sw-231-14(config-ioa-cl)# node sw-231-14 ip-address 10.1.1.1 sw-231-14(config-ioa-cl-node)# ex
The following example adds a remote node to an IOA cluster:
sw-231-14(config-ioa-cl)# node sw-231-19 ip-address 10.1.1.2 2011 Apr 8 05:02:47 sw-231-14 %CLUSTER-2-CLUSTER_QUORUM_GAIN: Cluster 0x2e05000dec18a133 now has quorum with 1 nodes 2011 Apr 8 05:02:52 sw-231-14 %CLUSTER-2-CLUSTER_QUORUM_GAIN: Cluster 0x2e05000dec18a133 now has quorum with 2 nodes sw-231-14(config-ioa-cl-node)# ex
The following example adds an IOA interfaces on the switch sw-231-14:
sw-231-14(config-ioa-cl)# node sw-231-14 sw-231-14(config-ioa-cl-node)# int ioa 1/1 sw-231-14(config-ioa-cl-node)# ex
The following example adds an IOA interface on the switch sw-231-19:
sw-231-14(config-ioa-cl)# node sw-231-19 sw-231-14(config-ioa-cl-node)# int ioa 4/1 sw-231-14(config-ioa-cl-node)# exit
The following example verifies the cluster configuration using the show cluster name node summary command:
sw-231-14# show ioa cluster cltr1 node summary ------------------------------------------------------------------------------- Switch Site Status Master Node ID ------------------------------------------------------------------------------- sw-231-14(L) site2 online yes 1 sw-231-19 site1 online no 2
The following example verifies the IP address that is configured on the switch using the show ioa cluster cluster name node command:
sw-231-14# show ioa cluster cltr1 node Node sw-231-14 is local switch Node ID is 1 IP address is 10.1.1.1 Status is online Belongs to Site site2 Node is the master switch Node sw-231-19 is remote switch Node ID is 2 IP address is 10.1.1.2 Status is online Belongs to Site site1 Node is not master switch
The following example verifies the IOA interface that is configured on the switch using the show ioa cluster cluster name interface summary command:
sw-231-14# show ioa cluster cltr1 int summary ------------------------------------------------------------------------------- Switch Interface Status Flows ------------------------------------------------------------------------------- sw-231-14(L) ioa1/1 up 0 sw-231-14(L) ioa1/2 up 0 sw-231-19 ioa4/1 up 0 sw-231-19 ioa4/2 up 0 sw-231-14#
To convert an existing IOA cluster to use the IPFC Interface, follow these steps:
Configuration Example for Converting IOA Cluster to Use the IPFC Interface
The following example verifies the IOA cluster configuration that is configured on the switch using show ioa cluster cluster name node summary command:
sw-231-14(config)# show ioa cluster cltnew node summary ------------------------------------------------------------------------------- Switch Site Status Master Node ID ------------------------------------------------------------------------------- sw-231-14(L) site2 online yes 1 sw-231-19 site1 online no 2
The following example verifies the IP address that is configured on the switch using the show ioa cluster cluster name node command:
sw-231-14(config)# show ioa cluster cltnew node Node sw-231-14 is local switch Node ID is 1 IP address is 172.25.231.14 Status is online Belongs to Site site2 Node is the master switch Node sw-231-19 is remote switch Node ID is 2 IP address is 172.25.231.19 Status is online Belongs to Site site1 Node is not master switch
The following example verifies the status of the flows using the show ioa cluster cluster name flows command. The nodes in this example are using mgmt0 interface address.
sw-231-14(config)# show ioa cluster cltnew flows ------------------------------------------------------------------------------- Host WWN, VSAN WA TA Comp Status Switch,Interface Target WWN Pair ------------------------------------------------------------------------------- 21:01:00:1b:32:22:55:df, 1 Y Y N online sw-231-14, ioa1/1 21:01:00:0d:77:dd:f8:9d, 1 sw-231-19, ioa4/1
The following example shuts down the IOA cluster on a local node using shut down command:
sw-231-14(config)# ioa cluster cltnew sw-231-14(config-ioa-cl)# shut down This change can be disruptive. Please ensure you have read the IOA Cluster Recovery Procedure" in the configuration guide. -- Are you sure you want to continue? (y/n) [n] y 2011 Apr 8 05:36:41 sw-231-14 %CLUSTER-2-CLUSTER_LOCAL_NODE_EXIT: Local Node 0x1 has left the Cluster 0x2e06000dec18a133
The following example shuts down the IOA cluster on the remote node using the shut down command:
sw-231-19(config)# ioa cluster cltnew sw-231-19(config-ioa-cl)# shut This change can be disruptive. Please ensure you have read the IOA Cluster Recovery Procedure" in the configuration guide. -- Are you sure you want to continue? (y/n) [n] y 2011 Apr 8 05:37:03 sw-231-19 %CLUSTER-2-CLUSTER_LOCAL_NODE_EXIT: Local Node 0x2 has left the Cluster 0x2e06000dec18a133 sw-231-19(config-ioa-cl)# exit
The following example removes the IOA cluster from the remote node using the no ioa cluster cluster name command:
sw-231-19(config)# no ioa cluster cltnew
Note | In case of removal of IOA cluster from the master node, clear all the FC-redirect configurations from the fabric, refer to “Configuring CFS Regions For FC-Redirect”. |
The following example verifies the absence of the IOA cluster on the remote node using the show ioa cluster cluster name command:
sw-231-19(config)# show ioa cluster sw-231-19(config)#
The following example removes the remote node from the cluster in the local switch:
sw-231-14(config-ioa-cl)# no node sw-231-19 sw-231-14(config-ioa-cl)# show ioa cluster cltnew node summary ------------------------------------------------------------------------------- Switch Site Status Master Node ID ------------------------------------------------------------------------------- sw-231-14(L) -- unknown (cluster is offline) 1
The following example changes the local node to use the IPFC address:
sw-231-14(config-ioa-cl)# node id 1 sw-231-14 ip-address 10.1.1.1 sw-231-14(config-ioa-cl-node)# exit
The following example activates the single node cluster:
sw-231-14(config-ioa-cl)# no shut This change can be disruptive. Please ensure you have read the IOA Cluster Recovery Procedure" in the configuration guide. -- Are you sure you want to continue? (y/n) [n] y sw-231-14(config-ioa-cl)# 2011 Apr 8 05:39:17 sw-231-14 %CLUSTER-2-CLUSTER_LEADER_ANNOUNCE: Node 0x1 is the new Master of cluster 0x2e06000dec18a133 of 1 nodes 2011 Apr 8 05:39:17 sw-231-14 %CLUSTER-2-CLUSTER_QUORUM_GAIN: Cluster 0x2e06000dec18a133 now has quorum with 1 nodes
The following example adds a remote node with IPFC address:
sw-231-14(config-ioa-cl)# node sw-231-19 ip-address 10.1.1.2 2011 Apr 8 05:39:36 sw-231-14 %CLUSTER-2-CLUSTER_QUORUM_GAIN: Cluster 0x2e06000dec18a133 now has quorum with 1 nodes 2011 Apr 8 05:39:41 sw-231-14 %CLUSTER-2-CLUSTER_QUORUM_GAIN: Cluster 0x2e06000dec18a133 now has quorum with 2 nodes
The following example adds the IOA interfaces to the remote node:
sw-231-14(config-ioa-cl-node)# int ioa 4/1 sw-231-14(config-ioa-cl-node)# end sw-231-14#
The following example verifies the status of the IOA clusters using the show ioa cluster cluster name node summary command:
sw-231-14# show ioa cluster cltnew node summary ------------------------------------------------------------------------------- Switch Site Status Master Node ID ------------------------------------------------------------------------------- sw-231-14(L) site2 online yes 1 sw-231-19 site1 online no 2
The following example verifies the status of the IOA clusters using the show ioa cluster cluster name flows command:
sw-231-14# show ioa cluster cltnew flows ------------------------------------------------------------------------------- Host WWN, VSAN WA TA Comp Status Switch,Interface Target WWN Pair ------------------------------------------------------------------------------- 21:01:00:1b:32:22:55:df, 1 Y Y N online sw-231-14, ioa1/1 21:01:00:0d:77:dd:f8:9d, 1 sw-231-19, ioa4/1 sw-231-14#