-
Cisco MDS 9000 Family Fabric Manager Configuration Guide, Release 2.x
-
New and Changed Information
-
Index
-
Preface
- Part 1 - Fabric Manager Applications
- Part 2 - Cisco MDS SAN-OS Installation and Configuration Files
-
Part 3 - Switch Configuration
-
Cisco Fabric Services
-
VSAN Configuration
-
Dynamic VSAN Configuration
-
Zone Configuration
-
Inter-VSAN Routing Configuration
-
PortChannel Configuration
-
Interface Configuration
-
FCIP Configuration
-
Configuring the SAN Extension Tuner
-
iSCSI configuration
-
FICON Configuration
-
Configuring Intelligent Storage Services
-
Additional Configuration
-
- Part 4 - Security Configuration
- Part 5 - Network and Performance Monitoring
- Part 6 - Troubleshooting
-
GUI/CLI Usage Chart
-
Interface Nonoperational Reason Codes
-
Managing Cisco FabricWare
-
FeedbackTable Of Contents
About Gigabit Ethernet Interfaces
Configuring a Basic Gigabit Ethernet Interface
Modifying FCIP Profiles and FCIP Links
Verifying Interfaces and Extended Link Protocol
Modifying FCIP Write Acceleration or FCIP Compression
Enabling FCIP Tape Acceleration
Configuring Advanced FCIP Interfaces
Using B Port Interoperability Mode
Ethernet PortChannels and Fibre Channel PortChannels
FCIP Configuration
Cisco MDS 9000 Family IP storage services (IPS) modules extend the reach of Fibre Channel SANs by using open standard, IP-based technology. The switch connects separated SAN islands using Fibre Channel over IP (FCIP).
Note
FCIP features are specific to the IPS modules running Cisco MDS SAN-OS Release 1.1(x) or later and the Gigabit Ethernet ports on the MPS-14/2 module running Cisco MDS SAN-OS Release 2.0(x) or later.
This chapter includes the following sections:
•
•
About Gigabit Ethernet Interfaces
Both FCIP and iSCSI rely on TCP/IP for network connectivity. On each IPS module, connectivity is provided in the form of Gigabit Ethernet interfaces that are appropriately configured. This section covers the steps required to configure IP for subsequent use by FCIP and iSCSI.
A new port mode, called IPS, is defined for Gigabit Ethernet ports on each IPS module. IP storage ports are implicitly set to IPS mode, so it can only be used to perform iSCSI and FCIP storage functions. IP storage ports do not bridge Ethernet frames or route other IP packets.
Tip
Configuring a Basic Gigabit Ethernet Interface
To configure the general characteristics of a Gigabit Ethernet interface, follow these steps:
Step 1
From Device Manager, right-click the Gigabit Ethernet port that you want to configure and choose Configure.... You see the Gigabit Ethernet configuration dialog box.
Step 2
Step 3
Step 4
Step 5
Step 6
From Device Manager, click Apply to save these changes, or click Close to discard changes and close the Gigabit Ethernet configuration dialog box.
FCIP Configuration
The Fibre Channel over IP Protocol (FCIP) is a tunneling protocol that connects geographically distributed Fibre Channel storage area networks (SAN islands) transparently over IP local area networks (LANs), metropolitan area networks (MANs), and wide area networks (WANs). See Figure 19-1.
Figure 19-1 Fibre Channel SANs Connected by FCIP
FCIP uses TCP as a network layer transport.
FCIP and VE Ports
Figure 19-2 describes the internal model of FCIP with respect to Fibre Channel Inter-Switch Links (ISLs) and Cisco's enhanced ISLs (EISLs).
FCIP virtual E (VE) ports behave exactly like standard Fibre Channel E ports, except that the transport in this case is FCIP instead of Fibre Channel. The only requirement is that the other end of the FCIP link be another VE port.
A virtual ISL is established over a FCIP link and transports Fibre Channel traffic. Each associated virtual ISL looks like a Fibre Channel ISL with either an E port or a TE port at each end (see Figure 19-2).
Figure 19-2 FCIP Links and Virtual ISLs
See the "E Port" section on page 18-2.
FCIP Links
FCIP links consist of one or more TCP connections between two FCIP link endpoints. Each link carries encapsulated Fibre Channel frames.
When the FCIP link comes up, the VE ports at both ends of the FCIP link create a virtual Fibre Channel (E)ISL and initiate the E port protocol to bring up the (E)ISL.
By default, the FCIP feature on any Cisco MDS 9000 Family switch creates two TCP connections for each FCIP link.
•
•
To enable FCIP on the IP services modules, an FCIP profile and FCIP interface (interface FCIP) must be configured.
The FCIP link is established between two peers, the VE port initialization behavior is identical to a normal E port. This behavior is independent of the link being FCIP or pure Fibre Channel and is based on the E port discovery process (ELP, ESC).
Once the FCIP link is established, the VE port behavior is identical to E port behavior for all inter-switch communication (including domain management, zones, and VSANs). At the Fibre Channel layer, VE port, and E port operations are identical.
FCIP Write Acceleration
The FCIP write acceleration feature in Cisco MDS SAN-OS Release 1.3(3) enables you to significantly improve application performance when storage traffic is routed over wide area networks using FCIP. When FCIP write acceleration is enabled, WAN throughput is maximized by minimizing the impact of WAN latency for the command to transfer ready acknowledgments (see Figure 19-3).
Note
Figure 19-3 FCIP Link Write Acceleration
In Figure 19-3, the write command without write acceleration requires two round-trip transfers (RTT), while the write command with write acceleration only requires one RTT. The maximum sized Transfer Ready is sent from the host side of the FCIP link back to the host before the write command reaches the target. This enables the host to start sending the write data without waiting for the long latency over the FCIP link of the write command and Transfer Ready. It also eliminates the delay caused by multiple Transfer Readys needed for the exchange going over the FCIP link.
Tip
Caution
FCIP Compression
The FCIP compression feature introduced in Cisco MDS SAN-OS Release 1.3(x) allows IP packets to be compressed on the FCIP link if this feature is enabled on that link. By default the FCIP compression is disabled.
This feature uses the Lempel-Zif-Stac (LZS) compression algorithm to compress packets.
The high-throughput mode allows faster compression but the compression ratio may be lower. The high-comp-ratio mode allows a higher compression ratio, but the throughput may be lower.
FCIP Compression is an optional check box within the FCIP Wizard.
Using the FCIP Wizard
Note
To create and manage FCIP links with Fabric Manager, use the FCIP Wizard. First verify that the IP services module is inserted in the required Cisco MDS 9000 Family switches and that the Gigabit Ethernet interfaces on these switches are connected and the connectivity verified. The steps in creating FCIP links using the FCIP Wizard are:
•
•
•
•
To create FCIP links using the FCIP Wizard, follow these steps:
Step 1
Figure 19-4 FCIP Wizard
Step 2
Step 3
Step 4
Figure 19-5 Enabling IPsec on an FCIP link
Step 5
Step 6
Figure 19-6 Specifying Tunnel Properties
Step 7
Step 8
Step 9
Step 10
Step 11
Figure 19-7 Create FCIP ISL
Step 12
Modifying FCIP Links
Once you have created FCIP links using the FCIP wizard, you may need to modify parameters for these links. This includes modifying the FCIP profiles as well as the FCIP link parameters. Each Gigabit Ethernet interface can have three active FCIP links at one time.
About FCIP Profiles
The FCIP profile contains information about local IP address and TCP parameters. The profile defines the following information:
•
•
The FCIP profile's local IP address determines the Gigabit Ethernet port where the FCIP links terminate (see Figure 19-8).
Figure 19-8 FCIP Profile and FCIP Links
FCIP Interfaces
The FCIP interface is the local endpoint of the FCIP link and a VE port interface. All the FCIP and E port parameters are configured in context to the FCIP interface.
The FCIP parameters consist of the following:
•
•
•
•
Modifying FCIP Profiles and FCIP Links
You can use Fabric Manager or Device Manager to modify FCIP links between switches or create new FCIP links. First, you must create or modify FCIP profiles, and then bind the interfaces to the profile. To bind an FCIP profile to an interface, use the IP address of the interface in the FCIP profile's IP address configuration. Profile numbers range from 1 to 255. The interface associated with a profile can be either of the following:
•
•
To modify FCIP profiles and FCIP links on a Gigabit Ethernet interface, follow these steps.
Step 1
Step 2
Step 3
Step 4
Step 5
Step 6
Step 7
Step 8
Step 9
Step 10
Verifying Interfaces and Extended Link Protocol
To verify the FCIP interfaces and Extended Link Protocol (ELP) on Device Manager, follow these steps:
Step 1
Step 2
Step 3
Step 4
Checking Trunk Status
To check the trunk status for the FCIP interface on Device Manager, follow these steps:
Step 1
Step 2
Step 3
Step 4
Modifying FCIP Write Acceleration or FCIP Compression
To modify FCIP Write Acceleration or FCIP compression using Fabric Manager, follow these steps:
Step 1
On Device manager, choose IP > FCIP. You see the FCIP dialog box.Step 2
Step 3
Step 4
Step 5
FCIP Tape Acceleration
Tapes are storage devices that store and retrieve user data sequentially. Applications that access tape drives normally have only one SCSI write operation outstanding to it. This single command process limits the benefit of the write acceleration feature when using an FCIP tunnel over a long-distance WAN link. It impacts backup and archive performance because each SCSI write operation does not complete until the host receives a good status response from the tape drive.
The FCIP tape acceleration feature introduced in Cisco MDS SAN-OS Release 2.0(1b) solves this problem. It improves tape backup and archive operations by allowing faster data streaming from the host to the tape over the WAN link.
The backup server in Figure 19-9 issues write operations to a drive in the tape library. Acting as a proxy for the remote tape drives, the local Cisco MDS switch quickly returns a Transfer Ready to signal the host to start sending data. After receiving all the data, the local Cisco MDS switch responds to signal the successful completion of the SCSI write operation. This response allows the host to start the next SCSI write operation. This proxy method results in more data being sent over the FCIP tunnel in the same time period compared to the time taken to send data without proxying. The proxy method improves the use of WAN links.
At the other end of the FCIP tunnel, another Cisco MDS switch buffers the command and data it has received. It then acts as a backup server to the tape drive by listening to a Transfer Ready from the tape drive before forwarding the data.
The Cisco MDS SAN-OS provides reliable data delivery to the remote tape drives using tcp/ip over the wan. it maintains write data integrity by allowing the Write Filemarks operation to complete end-to-end without proxying. The write filemarks operation signals the synchronization of the buffer data with the tape library data. While tape media errors are returned to backup servers for error handling, tape busy errors are retried automatically by the Cisco MDS SAN-OS software.
Figure 19-9 FCIP Link Tape Acceleration
Note
Tip
Caution
When you enable the tape acceleration feature, you are automatically enabling the write acceleration feature. When you enable tape acceleration for an FCIP tunnel, the tunnel is reinitialized.
The flow control buffer size specifies the maximum amount of write data that an MDS switch buffers for an FCIP tunnel before it stops the tape acceleration proxying process. The default buffer size is 256 KB and the maximum buffer size is 32 MB.
Enabling FCIP Tape Acceleration
To enable FCIP tape acceleration , follow these steps:
Step 1
From Device Manager, choose IP > FCIP. You see the FCIP dialog box.
Step 2
Step 3
Step 4
Step 5
Step 6
Step 7
Configuring Advanced FCIP Interfaces
You can establish connection to a peer by configuring one or more of the following options for the FCIP interface.
•
To establish a peer connection, you must first create the interface. See the "FCIP Interfaces" section.
Configuring Peers
To establish a FCIP link with the peer, use one of two options:
•
•
Peer IP Address
The basic FCIP configuration uses the peer's IP address to configure the peer information. You can also specify the peer's port number to configure the peer information. If you do not specify a port, the default 3225 port number is used to establish connection.
You can configure the required mode for initiating an IP connection. By default, active mode is enabled to actively attempt an IP connection. If you enable the passive mode, the switch does not initiate a TCP connection and merely waits for the peer to connect to it. Ensure that both ends of the FCIP link are not configured as passive mode. If both ends are configured as passive, the connection is not initiated.
You can specify the number of TCP connections from a FCIP link. By default, the switch tries two TCP connections for each FCIP link. You can configure one or two TCP connections. For example, the Cisco PA-FC-1G Fibre Channel port adapter, which has only one TCP connection, interoperates with any switch in the Cisco MDS 9000 Family. One TCP connection is within the specified limit. If the peer initiates one TCP connection, and your MDS switch is configured for two TCP connections, then the software handles it gracefully and moves on with just one connection.
You can instruct the switch to discard packets that are outside the specified time for this FCIP link. By default, this option is disabled in all switches in the Cisco MDS 9000 Family. This option specifies the time range within which packets can be accepted. If the packet arrived within the range specified by this option, the packet is accepted. Otherwise, it is dropped. By default if a packet arrives within a 1000 millisecond interval (+ or -1000 ms), that packet is accepted. Use the time-stamp option to enable or disable FCIP timestamps on a packet.
Note
To assign the peer information based on the IP address, port number, or profile ID, follow these steps:
Step 1
From Device manager, choose IP > FCIP. You see the FCIP dialog box.
Step 2
Step 3
Step 4
Step 5
Step 6
Step 7
Step 8
Step 9
Special Frames
You can alternatively establish an FCIP link with a peer using an optional protocol called special frames. When special frames are enabled, the peer IP address (and optionally the port or the profile ID) only needs to be configured on one end of the link. Once the connection is established, a special frame is exchanged to discover and authenticate the link.
By default, the special frame feature is disabled.
Note
Tip
To configure special frames, follow these steps:
Step 1
From Device manager, choose IP > FCIP. You see the FCIP dialog box.
Step 2
Step 3
Step 4
Step 5
Step 6
Step 7
Step 8
Step 9
Using B Port Interoperability Mode
While E ports typically interconnect Fibre Channel switches, some SAN extender devices, such as Cisco's PA-FC-1G Fibre Channel port adapter and the SN 5428-2 storage router, implement a bridge port model to connect geographically dispersed fabrics. This model uses B port as described in the T11 Standard FC-BB-2. Figure 19-10 depicts a typical SAN extension over an IP network.
Figure 19-10 FCIP B Port and Fibre Channel E Port
B ports bridge Fibre Channel traffic from one E port to a remote E port without participating in fabric-related activities such as principal switch election, domain ID assignment, and Fibre Channel routing (FSPF). For example, Class F traffic entering a SAN extender does not interact with the B port. The traffic is transparently propagated (bridged) over a WAN interface before exiting the remote B port. This bridge results in both E ports exchanging Class F information that ultimately leads to normal ISL behavior such as fabric merging and routing.
FCIP links between B port SAN extenders do not exchange the same information as FCIP links between E ports, and are therefore incompatible. This is reflected by the terminology used in FC-BB-2: while VE ports establish a virtual ISL over a FCIP link, B ports use a B access ISL.
The IPS module supports FCIP links that originate from a B port SAN extender device by implementing the B access ISL protocol on a Gigabit Ethernet interface. Internally, the corresponding virtual B port connects to a virtual E port that completes the end-to-end E port connectivity requirement (see Figure 19-11).
Figure 19-11 FCIP Link Terminating in a B Port Mode
The B port feature in the IPS module allows remote B port SAN extenders to communicate directly with a Cisco MDS 9000 Family switch, therefore eliminating the need for local bridge devices.
Configuring B Ports
When a FCIP peer is a SAN extender device that only supports Fibre Channel B ports, you need to enable the B port mode for the FCIP link. When a B port is enabled, the E port functionality is also enabled and they coexist. If the B port is disabled, the E port functionality remains enabled.
To configure B ports for FCIP links using Fabric Manager, follow these steps:
Step 1
From Device manager, choose IP > FCIP. You see the FCIP dialog box.
Step 2
Step 3
Step 4
Step 5
Step 6
Step 7
Step 8
Step 9
Configuring E Ports
•
•
•
–
–
•
•
•
FCIP High Availability
The following high availability solutions are available for FCIP configurations:
•
•
•
Fibre Channel PortChannels
Figure 19-12 provides an example of a PortChannel-based load balancing configuration. To perform this configuration, you need two IP addresses on each SAN island. This solution addresses link failures.
Figure 19-12 PortChannel Based Load Balancing
The following characteristics set Fibre Channel PortChannel solutions apart from other solutions:
•
•
•
FSPF
Figure 19-13 displays a FSPF-based load balancing configuration example. This configuration requires two IP addresses on each SAN island, and addresses IP and FCIP link failures.
Figure 19-13 FSPF-Based Load Balancing
The following characteristics set FSPF solutions apart from other solutions:
•
•
•
VRRP
Figure 19-14 displays a VRRP-based high availability FCIP configuration example. This configuration requires at least two physical Gigabit Ethernet ports connected to the Ethernet switch on the island where you need to implement high availability using VRRP.
Figure 19-14 VRRP-Based High Availability
The following characteristics set VRRP solutions apart from other solutions:
•
•
•
Ethernet PortChannels
Figure 19-15 displays an Ethernet PortChannel-based high availability FCIP example. This solution addresses the problem caused by individual Gigabit Ethernet link failures.
Figure 19-15 Ethernet PortChannel-Based High Availability
The following characteristics set Ethernet PortChannel solutions apart from other solutions:
•
•
•
Ethernet PortChannels and Fibre Channel PortChannels
Ethernet PortChannels offer Ethernet-level redundancy and Fibre Channel PortChannels offer (E)ISL-level redundancy. FCIP is unaware of any Ethernet PortChannels or Fibre Channel PortChannels. Fibre Channel PortChannels are unaware of any Ethernet PortChannels, and there is no mapping between the two (see Figure 19-16).
Figure 19-16 PortChannels at the Fibre Channel and Ethernet Levels
