Guest

^Additional Storage Networking

MDS to MDS Basic Configuration with FCIP

Document ID: 47100

Updated: Nov 29, 2005

   Print

Introduction

This document provides a sample configuration for basic Fibre Channel Over TCP/IP (FCIP) Multilayer Director Switch (MDS) to MDS.

This sample configuration is relevant for 1.2 and 1.3 releases of SAN-OS. Some parameters change in the 2.0 release of SAN-OS. Reference the 2.0 SAN-OS configuration guide and release notes.

FCIP describes mechanisms that allow the interconnection of islands of Fibre Channel (FC) storage area networks (SANs) over IP-based networks to form a unified SAN in a single FC fabric. FCIP relies on IP-based network services to provide the connectivity between the SAN islands over local area networks, metropolitan area networks, or wide area networks.

Fibre Channel SANs Connected by FCIP

basic_fcip_mds-a.gif

FCIP uses Transmission Control Protocol (TCP) on port 3225 as a Network layer transport.

Prerequisites

Requirements

The IP backbone must be operational and delivering the required bandwidth to support the applications running across the FCIP link(s)- this could be a Layer 2 (L2) or Layer 3 (L3) topology. If L3, the intermediate routers or multilayer switches must be setup and configured to forward IP traffic between source and destination IP addresses of the FCIP tunnels appropriately. If Quality of Service (QoS) or traffic shaping is enforced at any network device in the path between the FCIP peers, the network manager administrating the IP infrastructure should be consulted to get the necessary details before configuring any TCP related parameters and features on the MDS FCIP profile(s) .

Components Used

The information in this document is based on the following software and hardware versions:

  • MDS 9509 with IP Storage (IPS) service module (DS-X9308-SMIP) running version 1.2.(2a)

  • MDS 9216 with IPS service module (DS-X9308-SMIP) running version 1.2.(2a)

  • Win2003 Server (HPQ Pro-Liant-P4) with Emulex LP9K HBA

  • IBM Storage Array (ESS-2105-F20)

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Conventions

For more information on document conventions, refer to the Cisco Technical Tips Conventions.

Background Information

FCIP consists of the following specifications:

ANSI T11

  1. FC-SW-2 describes the operation and interaction of FC switches including E_Port and fabric operation.

  2. FC-BB-2 is a mapping that pertains to the extension of FC switched networks across a TCP network backbone, and defines reference models that support E_Port and B_Port.

IETF IPS Working Group

  1. FC over TCP covers the TCP/IP requirements for transporting FC frames over an IP network.

  2. FC frame encapsulation defines the common fibre encapsulation format.

An interconnection between two SAN switches or fabrics across FCIP is called an FCIP link, and can contain one or more TCP connections. Each end of a FCIP link is associated with a Virtual E port (VE_port) or a B_port, depending on the implementation. FC-BB and FC-BB-2 are describing the differences between both approaches. The IP Services module (DS-X9308-SMIP) supports both modes, but defaults to the VE_Port, which is also the recommended mode to run if all relevant peers are DS-X9308-SMIP modules. The VE_Port functionality on MDS platforms also supports TE port functionality, which makes it capable of trunking traffic from multiple VSANs across one FCIP instance.

Configure

On the MDS, you need to familiarize yourself with the IPS configuration guides for both platforms. The most current version of the manual is Configuring IP Storage.

Network Diagram

This document uses the network setup shown in the diagram below.

basic_fcip_mds-b.gif

This diagram shows a typical lab-setup where no additional networking equipment is connected between both Gigabit Ethernet (GE) interfaces of both MDS switches. This is the simplest form of an MDS FCIP install, and is typically used in customer labs to verify basic functionality. In VSAN 600, the Emulex LightPulse 9000 HBA connects the Windows 2003 server to the MDS9509 called Bison, and a IBM storage Array connected to MDS9216 called Canterbury, where LUNs for the Windows 2003 server are configured.

The Agilent SAN test device is used as an emulator to populate VSAN 601 with two devices, as well as to generate substantial FC-2 non-FCP background traffic. This peripheral equipment is added to make the configuration more realistic and to have substantial entries in the distributed name server of both participating switches. The focus of this document is not end-to-end connectivity, and no screen shots of the server or storage array are included. The peripheral equipment is not knowledgeable about the FCIP, and it behaves as if the EISL link between both MDSs was running across a normal FC link.

Configurations

This document uses the configurations shown below.

MDS 9509 (Bison) with IPS-8 module
bison# sh ver
 Cisco Storage Area Networking Operating System (SAN-OS) Software
 TAC support: http://www.cisco.com/tac
 Copyright (c) 2002-2003 by Cisco Systems, Inc. All rights reserved.
 The copyright for certain works contained herein are owned by
 Andiamo Systems, Inc. and/or other third parties and are used and
 distributed under license.


 Software
 BIOS: version 1.0.8
 loader: version 1.2(2)
 kickstart: version 1.2(2a)
 system: version 1.2(2a)

 BIOS compile time: 08/07/03
 kickstart image file is: bootflash:/k122a
 kickstart compile time: 9/23/2003 11:00:00
 system image file is: bootflash:/s122a
 system compile time: 10/8/2003 18:00:00


 Hardware
 RAM 1024584 kB

 bootflash: 500736 blocks (block size 512b)
 slot0: 0 blocks (block size 512b)

 bison uptime is 1 days 15 hours 45 minute(s) 44 second(s)

 Last reset 
 Reason: Unknown
 System version: 1.2(2a)
 Service: 

bison# sh run

 Building Configuration ...
 fcip profile 1
 ip address 100.100.100.1

!--- FCIP profile 1 is bound to the local relevant IPS interface. In this 
!--- example, it is the IP address of interface Gig4/1.

 vsan database
 vsan 600 
 vsan 601 

 fcdomain priority 1 vsan 1
 fcdomain domain 1 preferred vsan 1
 fcdomain domain 1 preferred vsan 600
 fcdomain domain 1 preferred vsan 601

 interface fcip1
 no shutdown
 switchport trunk allowed vsan 600-601
 use-profile 1
 peer-info ipaddr 100.100.100.2

!--- Interface FCIP 1 is configured to act as an EISL port carrying traffic 
!--- for both VSAN 600 and VSAN 601 across the tunnel. The FCIP interface,
!--- in most respects, is configured identical then any normal FC interface 
!--- acting as ISL or EISL. Bind this interface to FCIP profile 1, and define
!--- the peer-ip address 100.100.100.2, which is the address of the MDS9216's
!--- Gig 2/1 interface in the example.

vsan database
 vsan 600 interface fc3/1
 vsan 601 interface fc3/2
 vsan 601 interface fc3/8
 vsan 600 interface fc3/16

 zone name z-fcip2 vsan 600
 member pwwn 50:05:07:63:00:d0:94:4c
 member pwwn 10:00:00:00:c9:32:a6:e3

 zone name Zone_a1 vsan 601
 member pwwn 10:00:00:00:00:01:00:00
 member pwwn 10:00:00:00:00:05:00:00

 zoneset distribute full vsan 600

 zoneset name zs-fcip2 vsan 600
 member z-fcip2

 zoneset name Agilent_1 vsan 601
 member Zone_a1

 zoneset activate name zs-fcip2 vsan 600
 zoneset activate name Agilent_1 vsan 601

 interface GigabitEthernet4/1
 ip address 100.100.100.1 255.255.255.252
 no shutdown

!--- Note that Gig4/1 in the default state is configured with an MTU size of
!--- 1500 bytes, if the network topology allows for larger end-to-end frame
!--- sizes known as jumbo frames.
!--- The default value may be changed to a higher value. A good value is 
!--- 3000 bytes, because this would avoid the fragmentation of full 2048 FC
!--- frames into multiple TCP segments. Not all networking equipment can handle
!--- jumbo frames, so the default value of 1500 bytes is a conservative 
!--- approach to avoid connectivity issues while bringing up the FCIP tunnel.

MDS 9216 (Canterbury) with IPS-8 module
canterbury# sh ver
 Cisco Storage Area Networking Operating System (SAN-OS) Software
 TAC support: http://www.cisco.com/tac
 Copyright (c) 2002-2003 by Cisco Systems, Inc. All rights reserved.
 The copyright for certain works contained herein are owned by
 Andiamo Systems, Inc. and/or other third parties and are used and
 distributed under license.


 Software
 BIOS: version 1.0.8
 loader: version 1.2(2)
 kickstart: version 1.2(2a)
 system: version 1.2(2a)

 BIOS compile time: 08/07/03
 kickstart image file is: bootflash:/k122a
 kickstart compile time: 9/23/2003 11:00:00
 system image file is: bootflash:/s122a
 system compile time: 10/8/2003 18:00:00


 Hardware
 RAM 960072 kB

 bootflash: 500736 blocks (block size 512b)
 slot0: 0 blocks (block size 512b)

 canterbury uptime is 6 days 22 hours 35 minute(s) 37 second(s)

Last reset at 995484 usecs after Wed Nov 5 15:05:04 2003
 Reason: Reset by installer
 System version: 1.2(1a)
 Service: 

canterbury# sh run

 Building Configuration ...
 fcip profile 1
 ip address 100.100.100.2

!--- At this side of the tunnel, choose the same profile number that you 
!--- used on the peer to make management easier. This is not mandatory, 
!--- and you can choose another value between 1 and 255.

 vsan database
 vsan 600 
 vsan 601 

 fcdomain domain 2 preferred vsan 600
 fcdomain domain 2 preferred vsan 601

 interface fcip1
 no shutdown
 switchport trunk allowed vsan 600-601
 use-profile 1
 peer-info ipaddr 100.100.100.1

!--- FCIP interface 1 is chosen for arbitrary reasons. You can choose another 
!--- FCIP number and still tunnel to the peer FCIP 1 instance.
!--- It is important that you bind the correct profile-id to your FCIP interface 
!--- of choice. Allow the same VSANs that you allowed on the peer FCIP interface,  
!--- which is good practice in general for normal EISL trunks. The peer ip-address 
!--- is the IP address of the MDS9505s interface Gig4/1
!--- in the Network Diagram above.

 vsan database
 vsan 600 interface fc1/1
 vsan 601 interface fc1/8
 vsan 600 interface fc1/16

 zone name z-fcip2 vsan 600
 member pwwn 50:05:07:63:00:d0:94:4c
 member pwwn 10:00:00:00:c9:32:a6:e3

 zone default-zone permit vsan 777
 zoneset distribute full vsan 600

 zoneset name zs-fcip2 vsan 600
 member z-fcip2

 zoneset activate name zs-fcip2 vsan 600
 zoneset activate name Agilent_1 vsan 601

 interface GigabitEthernet2/1
 ip address 100.100.100.2 255.255.255.252
 no shutdown

Verify

This section provides information you can use to confirm your configuration is working properly.

  • show interface gig x/y —Displays status of the relevant Gigabit interface bound to the FCIP profile.

  • show ips stats tcp int gig x/y —Displays TCP statistics and active connections for the relevant Gigabit interface.

  • show ips arp int gig x/y —Displays all Address Resolution Protocol (ARP) entries for the relevant Gigabit interface; the next hop or peer should be present in this list.

  • show ips ip route int gig x/y —Displays the specific routes going across the relevant Gigabit interface.

  • show interface fcip x —Displays the FCIP interface status and all details related to this FCIP tunnel.

  • show profile fcip x —Displays the IP address to which the profile is bound and all configured TCP parameters.

  • show int fcip x counters—Used to check if there are any frames going through the FCIP tunnel.

  • show fcdomain vsan x —Lists all domain-related details; used to verify that the fabric is formed across the FCIP tunnel(s).

  • show fcns da vsan x —Displays all pwwn, FC4-Types, and FCIDs of the relevant VSAN; used to verify that all expected entries are distributed across the FCIP tunnel(s).

Troubleshoot

Be sure to issue the show commands above multiple times to build a counter history. Counters that are not related to a point in time and collected only once are mostly useless.

Use the configurations shown below for more troubleshooting.

MDS 9509 (Bison)
GigabitEthernet4/1 is up
     Hardware is GigabitEthernet, address is 0005.3000.a85a
     Internet address is 100.100.100.1/30
     MTU 1500  bytes   !...default value
     Port mode is IPS
     Speed is 1 Gbps
     Beacon is turned off
     Auto-Negotiation is turned on
     5 minutes input rate 320 bits/sec, 40 bytes/sec, 0 frames/sec
     5 minutes output rate 312 bits/sec, 39 bytes/sec, 0 frames/sec
     933169199 packets input, 998306879592 bytes
       12 multicast frames, 0 compressed
       0 input errors, 0 frame, 0 overrun 0 fifo
     337209366 packets output, 214303313560 bytes, 0 underruns
       0 output errors, 0 collisions, 0 fifo
       0 carrier errors

 bison# sh ips stats tcp int gig 4/1
 TCP Statistics for port GigabitEthernet4/1
     Connection Stats
       272 active openings, 107 accepts
       206 failed attempts, 0 reset received, 163 established
     Segment stats
       932985717 received, 337201993 sent, 7 retransmitted
       0 bad segments received, 103 reset sent

 TCP Active Connections 
 Local Address Remote Address State Send-Q Recv-Q
 100.100.100.1:3225 100.100.100.2:65128 ESTABLISH 0 0
 100.100.100.1:3225 100.100.100.2:65130 ESTABLISH 0 0
 100.100.100.1:3225 0.0.0.0:0 LISTEN 0 0

!--- By default, MDS establishes two TCP connections per FCIP tunnel instance.


bison# sh ips stats tcp int gig 4/1 de
 TCP Statistics for port GigabitEthernet4/1
     TCP send stats
       337202017 segments, 222637392068 bytes
       130562402 data, 205533417 ack only packets
       503 control (SYN/FIN/RST), 0 probes, 1105737 window updates
       7 segments retransmitted, 2208 bytes
       4 retransmitted while on ethernet send queue, 40061909 packets split
       250922624 delayed acks sent
     TCP receive stats
       932985742 segments, 921498012 data packets in sequence,
                           936715052100 bytes in sequence
       770241 predicted ack, 856752348 predicted data
       0 bad checksum, 0 multi/broadcast, 0 bad offset
       0 no memory drops, 0 short segments
       0 duplicate bytes, 16 duplicate packets
       0 partial duplicate bytes, 0 partial duplicate packets
       53128 out-of-order bytes, 165 out-of-order packets
       0 packet after window, 0 bytes after window
       5 packets after close
       76225562 acks, 192030009160 ack bytes, 0 ack toomuch, 5851 duplicate acks
       0 ack packets left of snd_una, 0 non-4 byte aligned packets
       9124012 window updates, 0 window probe
       1381 pcb hash miss, 984 no port, 103 bad SYN, 0 paws drops
     TCP Connection Stats
       272 attempts, 107 accepts, 163 established
       511 closed, 3 drops, 206 conn drops
       3 drop in retransmit timeout, 20 drop in keepalive timeout
       0 drop in persist drops, 0 connections drained
     TCP Miscellaneous Stats
       61792500 segments timed, 76225541 rtt updated
       124 retransmit timeout, 0 persist timeout
       5760 keepalive timeout, 5740 keepalive probes
     TCP SACK Stats
       0 recovery episodes, 0 data packets, 0 data bytes
       0 data packets retransmitted, 0 data bytes retransmitted
       0 connections closed, 0 retransmit timeouts
     TCP SYN Cache Stats
       107 entries, 107 connections completed, 0 entries timed out
       0 dropped due to overflow, 0 dropped due to RST
       0 dropped due to ICMP unreach, 0 dropped due to bucket overflow
       0 abort due to no memory, 0 duplicate SYN, 0 no-route SYN drop
       0 hash collisions, 0 retransmitted

 TCP Active Connections 
 Local Address Remote Address State Send-Q Recv-Q
 100.100.100.1:3225 100.100.100.2:65128 ESTABLISH 0 0
 100.100.100.1:3225 100.100.100.2:65130 ESTABLISH 0 0
 100.100.100.1:3225 0.0.0.0:0 LISTEN 0 0
 bison# 

bison# sh ips stats tcp int gig 4/1 de
 TCP Statistics for port GigabitEthernet4/1
     TCP send stats
       337202017 segments, 222637392068 bytes
       130562402 data, 205533417 ack only packets
       503 control (SYN/FIN/RST), 0 probes, 1105737 window updates
       7 segments retransmitted, 2208 bytes
       4 retransmitted while on ethernet send queue, 40061909 packets split
       250922624 delayed acks sent
     TCP receive stats
       932985742 segments, 921498012 data packets in sequence,
                           936715052100 bytes in sequence
       770241 predicted ack, 856752348 predicted data
       0 bad checksum, 0 multi/broadcast, 0 bad offset
       0 no memory drops, 0 short segments
       0 duplicate bytes, 16 duplicate packets
       0 partial duplicate bytes, 0 partial duplicate packets
       53128 out-of-order bytes, 165 out-of-order packets
       0 packet after window, 0 bytes after window
       5 packets after close
       76225562 acks, 192030009160 ack bytes, 0 ack toomuch, 5851 duplicate acks
       0 ack packets left of snd_una, 0 non-4 byte aligned packets
       9124012 window updates, 0 window probe
       1381 pcb hash miss, 984 no port, 103 bad SYN, 0 paws drops
     TCP Connection Stats
       272 attempts, 107 accepts, 163 established
       511 closed, 3 drops, 206 conn drops
       3 drop in retransmit timeout, 20 drop in keepalive timeout
       0 drop in persist drops, 0 connections drained
     TCP Miscellaneous Stats
       61792500 segments timed, 76225541 rtt updated
       124 retransmit timeout, 0 persist timeout
       5760 keepalive timeout, 5740 keepalive probes
     TCP SACK Stats
       0 recovery episodes, 0 data packets, 0 data bytes
       0 data packets retransmitted, 0 data bytes retransmitted
       0 connections closed, 0 retransmit timeouts
     TCP SYN Cache Stats
       107 entries, 107 connections completed, 0 entries timed out
       0 dropped due to overflow, 0 dropped due to RST
       0 dropped due to ICMP unreach, 0 dropped due to bucket overflow
       0 abort due to no memory, 0 duplicate SYN, 0 no-route SYN drop
       0 hash collisions, 0 retransmitted

 TCP Active Connections 
 Local Address Remote Address State Send-Q Recv-Q
 100.100.100.1:3225 100.100.100.2:65128 ESTABLISH 0 0
 100.100.100.1:3225 100.100.100.2:65130 ESTABLISH 0 0
 100.100.100.1:3225 0.0.0.0:0 LISTEN 0 0
 bison# 

!--- Most of the TCP details displayed above can be used to determine the 
!--- health of your FCIP tunnel, provided that there is a one-to-one relationship 
!--- between the FCIP tunnel and the physical interface. Note that for this
!--- particular FCIP instance, both TCP connections were initiated from this peer,
!--- which you can derive from the local address x.x.x.x:3225 statement.


 bison# sh ips arp interface gig 4/1
 Protocol Address Age (min) Hardware Addr Type Interface
 Internet 100.100.100.2 9 0005.3000.ade6 ARPA GigabitEthernet4/1
 bison# 

 bison# sh ips ip route int gig 4/1
 Codes: C - connected, S - static

 No default gateway

 C 100.100.100.0/30 is directly connected, GigabitEthernet4/1
 bison#

!--- The FCIP tunnel is connected in a back-to-back fashion. Issue the
!--- sh ips ip route command to get the directly connected IP subnet.
!--- In a more realistic situation, where you would need to configure a
!--- next-hop to reach the FCIP peer ip-address, this command would show
!--- the configured routes through the relevant interfaces.


bison# sh fcip profile 1
 FCIP Profile 1
 Internet Address is 100.100.100.1 (interface GigabitEthernet4/1)
 Listen Port is 3225
 TCP parameters
 SACK is enabled
 PMTU discovery is enabled, reset timeout is 3600 sec
 Keep alive is 60 sec
 Minimum retransmission timeout is 200 ms
 Maximum number of re-transmissions is 4
 Send buffer size is 0 KB
 Maximum allowed bandwidth is 1000000 kbps
 Minimum available bandwidth is 15000 kbps
 Estimated round trip time is 1000 usec
 Congestion window monitoring is enabled, burst size is 10 KB

!--- The profile parameters are an easy way to directly verify your 
!--- configured TCP parameters per FCIP instance.


 bison# sh int fcip 1
 fcip1 is trunking
 Hardware is GigabitEthernet
 Port WWN is 20:c2:00:05:30:00:7a:de
 Peer port WWN is 20:42:00:0c:30:6c:24:40
 Admin port mode is auto, trunk mode is on
 Port mode is TE
 vsan is 1
 Trunk vsans (allowed active) (600-601)
 Trunk vsans (operational) (600-601)
 Trunk vsans (up) (600-601)
 Trunk vsans (isolated) ()
 Trunk vsans (initializing) ()
 Using Profile id 1 (interface GigabitEthernet4/1)
 Peer Information
 Peer Internet address is 100.100.100.2 and port is 3225
 Special Frame is disabled
 Maximum number of TCP connections is 2
 Time Stamp is disabled
 QOS control code point is 0
 QOS data code point is 0
 B-port mode disabled
 TCP Connection Information
 2 Active TCP connections
 Control connection: Local 100.100.100.1:3225, Remote 100.100.100.2:65128
 Data connection: Local 100.100.100.1:3225, Remote 100.100.100.2:65130
 272 Attempts for active connections, 58 close of connections
 TCP Parameters
 Path MTU 1500 bytes
 Current retransmission timeout is 200 ms
 Round trip time: Smoothed 2 ms, Variance: 1
 Advertised window: Current: 118 KB, Maximum: 118 KB, Scale: 1
 Peer receive window: Current: 118 KB, Maximum: 118 KB, Scale: 1
 Congestion window: Current: 10 KB, Slow start threshold: 112 KB
 5 minutes input rate 120 bits/sec, 15 bytes/sec, 0 frames/sec
 5 minutes output rate 120 bits/sec, 15 bytes/sec, 0 frames/sec
 72182460 frames input, 135382910244 bytes
 34626 Class F frames input, 3190588 bytes
 72147834 Class 2/3 frames input, 135379719656 bytes
 0 Error frames timestamp error 0
 47823751 frames output, 97610768920 bytes
 34632 Class F frames output, 3194464 bytes
 47789119 Class 2/3 frames output, 97607574456 bytes
 0 Error frames 373 reass frames


!--- You can see the specific details per FCIP interface, as they are taken 
!--- into account by a running FCIP instance. You can also derive the TCP
!--- parameters of the peer with this output.


 bison# sh fcdomain vsan 600
 The local switch is the Principal Switch.

 Local switch run time information:
 State: Stable
 Local switch WWN: 22:58:00:05:30:00:7a:df
 Running fabric name: 22:58:00:05:30:00:7a:df
 Running priority: 2
 Current domain ID: 0x01(1) 

 Local switch configuration information:
 State: Enabled
 FCID persistence: Disabled
 Auto-reconfiguration: Disabled
 Contiguous-allocation: Disabled
 Configured fabric name: 20:01:00:05:30:00:28:df
 Configured priority: 128
 Configured domain ID: 0x01(1) (preferred)

 Principal switch run time information:
 Running priority: 2

 Interface Role RCF-reject
---------------- ------------- ------------
 fcip1 Downstream Disabled
 ---------------- ------------- ------------

 bison# sh fcdomain vsan 601
 The local switch is the Principal Switch.

 Local switch run time information:
 State: Stable
 Local switch WWN: 22:59:00:05:30:00:7a:df
 Running fabric name: 22:59:00:05:30:00:7a:df
 Running priority: 2
 Current domain ID: 0x01(1) 

 Local switch configuration information:
 State: Enabled
 FCID persistence: Disabled
 Auto-reconfiguration: Disabled
 Contiguous-allocation: Disabled
 Configured fabric name: 20:01:00:05:30:00:28:df
 Configured priority: 128
 Configured domain ID: 0x01(1) (preferred)

 Principal switch run time information:
 ---------------- ------------- ------------
 fcip1 Downstream Disabled
 ---------------- ------------- ------------

 bison# sh fcdomain vsan 601
 The local switch is the Principal Switch.

 Local switch run time information:
 State: Stable
 Local switch WWN: 22:59:00:05:30:00:7a:df
 Running fabric name: 22:59:00:05:30:00:7a:df
 Running priority: 2
 Current domain ID: 0x01(1) 

 Local switch configuration information:
 State: Enabled
 FCID persistence: Disabled
 Auto-reconfiguration: Disabled
 Contiguous-allocation: Disabled
 Configured fabric name: 20:01:00:05:30:00:28:df
 Configured priority: 128
 Configured domain ID: 0x01(1) (preferred)

 Principal switch run time information:
 Running priority: 2

 Interface Role RCF-reject
 ---------------- ------------- ------------
 fcip1 Downstream Disabled
 ---------------- ------------- ------------
 bison# 
 
!--- Similar to normal (E)ISL troubleshooting, verify that 
!--- your fabric is formed as expected.


 bison# sh fcns da vsan 600-601

 VSAN 600:
 --------------------------------------------------------------------------
 FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE
 --------------------------------------------------------------------------
 0x010001 N 10:00:00:00:c9:32:a6:e3 (Emulex) scsi-fcp:init 
 0x020001 N 50:05:07:63:00:d0:94:4c (IBM) scsi-fcp:target fc.. 

 Total number of entries = 2

 VSAN 601:
 --------------------------------------------------------------------------
 FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE
 --------------------------------------------------------------------------
 0x010001 N 10:00:00:00:c9:32:a6:e2 (Emulex) scsi-fcp:init 
 0x010100 N 10:00:00:00:00:05:00:00 
 0x020100 N 10:00:00:00:00:01:00:00 

 Total number of entries = 3

MDS 9216 (Canterbury)
canterbury# sh int gig 2/1
 GigabitEthernet2/1 is up
     Hardware is GigabitEthernet, address is 0005.3000.ade6
     Internet address is 100.100.100.2/30
     MTU 1500  bytes
     Port mode is IPS
     Speed is 1 Gbps
     Beacon is turned off
     Auto-Negotiation is turned on
     5 minutes input rate 312 bits/sec, 39 bytes/sec, 0 frames/sec
     5 minutes output rate 312 bits/sec, 39 bytes/sec, 0 frames/sec
     337277325 packets input, 214308964948 bytes
       12 multicast frames, 0 compressed
       0 input errors, 0 frame, 0 overrun 0 fifo
     932989688 packets output, 998294817662 bytes, 0 underruns
       0 output errors, 0 collisions, 0 fifo
       0 carrier errors

 canterbury# sh ips arp int gig 2/1
 Protocol         Address   Age (min)   Hardware Addr  Type   Interface
 Internet   100.100.100.1        7     0005.3000.a85a  ARPA  GigabitEthernet2/1
 canterbury# 

 canterbury# sh ips ip route int gig 2/1
 Codes: C - connected, S - static

 No default gateway

 C 100.100.100.0/30 is directly connected, GigabitEthernet2/1
 canterbury# 

 canterbury# sh ips stats tcp int gig 2/1 de
 TCP Statistics for port GigabitEthernet2/1
 TCP send stats
 932982227 segments, 1022389174048 bytes
 921498559 data, 11061499 ack only packets
 401 control (SYN/FIN/RST), 0 probes, 421342 window updates
 454 segments retransmitted, 972180 bytes
 291 retransmitted while on ethernet send queue, 223642028 packets split
 76162595 delayed acks sent
 TCP receive stats
 337204879 segments, 130561386 data packets in sequence,

                     192030387428 bytes in sequence
 156457374 predicted ack, 65996627 predicted data
 0 bad checksum, 0 multi/broadcast, 0 bad offset
 0 no memory drops, 0 short segments
 48 duplicate bytes, 3542 duplicate packets
 48 partial duplicate bytes, 1 partial duplicate packets
 4336 out-of-order bytes, 131 out-of-order packets
 0 packet after window, 0 bytes after window
 0 packets after close
 268794983 acks, 936715866930 ack bytes, 0 ack toomuch, 4152 duplicate acks
 0 ack packets left of snd_una, 0 non-4 byte aligned packets
 50179371 window updates, 0 window probe
 1251 pcb hash miss, 1061 no port, 0 bad SYN, 0 paws drops
 TCP Connection Stats
 204 attempts, 73 accepts, 155 established
 357 closed, 64 drops, 70 conn drops
 4 drop in retransmit timeout, 10 drop in keepalive timeout
 0 drop in persist drops, 0 connections drained
 TCP Miscellaneous Stats
 233047332 segments timed, 268794618 rtt updated
105 retransmit timeout, 0 persist timeout
 105 keepalive timeout, 94 keepalive probes
 TCP SACK Stats
 3 recovery episodes, 25938540 data packets, 71110030772 data bytes
 180 data packets retransmitted, 272884 data bytes retransmitted
 1 connections closed, 388 retransmit timeouts
 TCP SYN Cache Stats
 93 entries, 73 connections completed, 0 entries timed out
 0 dropped due to overflow, 18 dropped due to RST
 0 dropped due to ICMP unreach, 0 dropped due to bucket overflow
 0 abort due to no memory, 6 duplicate SYN, 0 no-route SYN drop
 0 hash collisions, 8 retransmitted

 TCP Active Connections 
 Local Address Remote Address State Send-Q Recv-Q
 100.100.100.2:65128 100.100.100.1:3225 ESTABLISH 0 0
 100.100.100.2:65130 100.100.100.1:3225 ESTABLISH 0 0
 100.100.100.2:3225 0.0.0.0:0 LISTEN 0 0
 0.0.0.0:3260 0.0.0.0:0 LISTEN 0 0
 canterbury#

!--- This MDS initiated both TCP connections for FCIP 1. Although no passive
!--- statement was configured on the peer MDS, MDS9216 Canterbury has the
!--- highest IP address configured on the tunnel. This makes the other side
!--- disconnect its TCP connection.


 canterbury# sh fcip profile 1
 FCIP Profile 1
 Internet Address is 100.100.100.2 (interface GigabitEthernet2/1)
 Listen Port is 3225
 TCP parameters
 SACK is enabled
 PMTU discovery is enabled, reset timeout is 3600 sec
 Keep alive is 60 sec
 Minimum retransmission timeout is 200 ms
 Maximum number of re-transmissions is 4
 Send buffer size is 0 KB
 Maximum allowed bandwidth is 1000000 kbps
 Minimum available bandwidth is 15000 kbps
 Estimated round trip time is 1000 usec
 Congestion window monitoring is enabled, burst size is 10 KB

 canterbury# sh interface fcip 1
 fcip1 is trunking
 Hardware is GigabitEthernet
 Port WWN is 20:42:00:0c:30:6c:24:40
 Peer port WWN is 20:c2:00:05:30:00:7a:de
 Admin port mode is auto, trunk mode is auto
 Port mode is TE
 vsan is 1
 Trunk vsans (allowed active) (600-601)
 Trunk vsans (operational) (600-601)
 Trunk vsans (up) (600-601)
 Trunk vsans (isolated) ()
 Trunk vsans (initializing) ()
 Using Profile id 1 (interface GigabitEthernet2/1)
 Peer Information
 Peer Internet address is 100.100.100.1 and port is 3225
 Special Frame is disabled
 Maximum number of TCP connections is 2
 Time Stamp is disabled
 QOS control code point is 0
 QOS data code point is 0
 B-port mode disabled
 TCP Connection Information
 2 Active TCP connections
 Control connection: Local 100.100.100.2:65128, Remote 100.100.100.1:3225
 Data connection: Local 100.100.100.2:65130, Remote 100.100.100.1:3225
 204 Attempts for active connections, 72 close of connections
 TCP Parameters
 Path MTU 1500 bytes
 Current retransmission timeout is 200 ms
 Round trip time: Smoothed 2 ms, Variance: 1
 Advertized window: Current: 118 KB, Maximum: 118 KB, Scale: 1
 Peer receive window: Current: 118 KB, Maximum: 118 KB, Scale: 1
 Congestion window: Current: 10 KB, Slow start threshold: 112 KB
 5 minutes input rate 120 bits/sec, 15 bytes/sec, 0 frames/sec
 5 minutes output rate 120 bits/sec, 15 bytes/sec, 0 frames/sec
 91063905 frames input, 192030052404 bytes
 41991 Class F frames input, 3931568 bytes
 91021914 Class 2/3 frames input, 192026120836 bytes
 0 Error frames timestamp error 0
 753551524 frames output, 936716093696 bytes
 42028 Class F frames output, 3909128 bytes
 753509496 Class 2/3 frames output, 936712184568 bytes
 0 Error frames 40061908 reass frames

 canterbury#

Related Information

Updated: Nov 29, 2005
Document ID: 47100