Guest

^Additional Storage Networking

MDS to MDS 802.1Q Configuration with FCIP

Document ID: 47101

Updated: Oct 28, 2005

   Print

Introduction

This document provides a sample configuration for Fibre Channel Over TCP/IP (FCIP) with 802.1Q Multilayer Director Switch (MDS) to MDS.

FCIP describes mechanisms that allow the interconnection of islands of Fibre Channel (FC) storage area networks (SANs) over IP-based networks to form a unified SAN in a single FC fabric. FCIP relies on IP-based network services to provide the connectivity between the SAN islands over local area networks, metropolitan area networks, or wide area networks.

Fibre Channel SANs Connected by FCIP

ips8_dot1q_1.gif

FCIP uses Transmission Control Protocol (TCP) on port 3225 as a network layer transport.

Prerequisites

Requirements

The IP backbone must be operational and delivering the required bandwidth to support the applications running across the FCIP links—this could be a Layer 2 (L2) or Layer 3 (L3) topology. If it is a L3 topology, the intermediate routers or multilayer switches must be set up and configured to appropriately forward IP traffic between source and destination IP addresses of the FCIP tunnels. If Quality of Service (QoS)or traffic shaping is enforced at any network device in the path between the FCIP peers, the network manager administrating the IP infrastructure should be consulted to get the necessary details before configuring any TCP-related parameters and features on the Multilayer Director Switch (MDS) FCIP profile(s). The Ethernet switches which are adjacent to the MDSes must support and be configured for 802.1Q trunking if subinterfaces are configured on the MDS IP Storage (IPS) services module.

Components Used

The information in this document is based on these software and hardware versions:

  • MDS 9509 with IPS service module (DS-X9308-SMIP) running version 1.2.(2a)

  • MDS 9216 with IPS service module (DS-X9308-SMIP) running version 1.2.(2a)

  • Catalyst 6509 running Catalyst OS (CatOS) 7.4(3)

  • Win2003 Server (HPQ Pro-Liant-P4) with Emulex LP9K HBA

  • IBM Storage Array (ESS-2105-F20)

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Conventions

For more information on document conventions, refer to the Cisco Technical Tips Conventions.

Background Information

FCIP consists of these specifications:

ANSI T11

  1. FC-SW-2 describes the operation and interaction of FC switches including E_Port and fabric operation.

  2. FC-BB-2 is a mapping that pertains to the extension of FC switched networks across a TCP network backbone, and defines reference models that support E_Port and B_Port.

IETF IPS Working Group

  1. FC over TCP covers the TCP/IP requirements for transporting FC frames over an IP network.

  2. FC frame encapsulation defines the common fibre encapsulation format.

IEEE 802 Standards

IEEE 802 LANs of all types may be connected together with MAC Bridges, as specified in ISO/IEC 15802-3. This standard defines the operation of VLAN Bridges that permit the definition, operation, and administration of VLAN topologies within a Bridged LAN infrastructure.

An interconnection between two SAN switches or fabrics across FCIP is called an FCIP link and can contain one or more TCP connections. Each end of a FCIP link is associated with a Virtual E port (VE_port) or a B_port, depending on the implementation. FC-BB and FC-BB-2 are describing the differences between both approaches. The IP services module (DS-X9308-SMIP) is supporting both modes but defaults to VE_Port, which is also the recommended mode to run if all relevant peers are DS-X9308-SMIP modules. The VE_Port functionality on MDS platforms is also supporting TE port functionality, which makes it capable of trunking traffic from multiple virtual SANs (VSANs) across one FCIP instance. The Gigabit Ethernet (GE) interfaces residing aboard the Cisco X9308-SMIP modules support 802.1Q in order to leverage the bandwidth of 1 Gbps between two or more FCIP tunnels in situations where low bandwidth requirements per FCIP tunnel exist. One must understand that sharing bandwidth by using dot1q is not providing deterministic Bandwidth per FCIP tunnel when FCIP Profile TCP parameters are left in default state.

Configure

On the MDSes, you need to familiarize yourself with the IPS configuration guides for both platforms. The most current version of the manuals can be found at Configuring IP Storage on Cisco.com. On the Ethernet switch side, one needs to be familiar with the dot1q trunking configuration specifics. In this particular example, a Catalyst running Hybrid CatOS is deployed; different configuration may apply to other Cisco switches or to switches from other vendors.

For Catalyst 6000 series running hybrid mode, refer to Configuring Ethernet VLAN Trunks. For Native IOS, refer to Configuring VLANs. For Catalyst XL type switches running Native IOS, refer to Configuring VLANs.

Note: To find additional information on the commands used in this document, use the Command Lookup Tool (registered customers only) .

Network Diagram

This document uses the network setup shown in the diagram below.

Topology 2

ips8_dot1q_2.gif

Topology 2 depicts one FCIP tunnel running across a 802.1Q trunk on either side of the IP cloud. The IP cloud is collapsed into one multilayer switch (Catalyst 6500) which routes traffic from VLAN 1000 to VLAN 500 and from VLAN 500 to VLAN 1000. VLAN 1000 conceptually maps to IP subnet 100.100.100.0/30, and VLAN 500 maps to IP subnet 100.100.100.4/30. The way the MDS maps and retrieves dot1q frames will become clear in the configuration section below. For simplicity, only one FCIP tunnel across one physical interface on both MDSes is defined; in reality, one would only use dot1q trunking to share the bandwidth of one Gigabit interface between multiple FCIP tunnels.

Configurations

MDS 9509 (Bison) with IPS-8 module
bison# sh ver

Cisco Storage Area Networking Operating System (SAN-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2003 by Cisco Systems, Inc. All rights reserved.
The copyright for certain works contained herein are owned by
Andiamo Systems, Inc. and/or other third parties and are used and
distributed under license.

Software
BIOS: version 1.0.8
loader: version 1.2(2)
kickstart: version 1.2(2a)
system: version 1.2(2a)

BIOS compile time: 08/07/03
kickstart image file is: bootflash:/k122a
kickstart compile time: 9/23/2003 11:00:00
system image file is: bootflash:/s122a
system compile time: 10/8/2003 18:00:00

Hardware
RAM 1024584 kB

bootflash: 500736 blocks (block size 512b)
slot0: 0 blocks (block size 512b)

bison uptime is 1 days 15 hours 45 minute(s) 44 second(s)

Last reset 
Reason: Unknown
System version: 1.2(2a)
Service: 

bison# sh run

Building Configuration ...
fcip profile 1
ip address 100.100.100.1

!--- FCIP profile 1 is bound to the local relevant IPS interface. 
!--- In this example, it is the IP address of interface Gig4/1.

vsan database
vsan 200 name test 
vsan 600 
vsan 601 

fcdomain priority 1 vsan 1
fcdomain domain 1 preferred vsan 1
fcdomain domain 1 preferred vsan 600
fcdomain domain 1 preferred vsan 601

interface fcip1
no shutdown
switchport trunk allowed vsan 600-601
use-profile 1
peer-info ipaddr 100.100.100.6


<!--- peer IP address is the address on remote MDS Canterbury
<!--- configured on interface Gig4/1.500.

vsan database
vsan 600 interface fc3/1
vsan 601 interface fc3/8

boot system bootflash:/s122a sup-1
boot kickstart bootflash:/k122a sup-1
boot system bootflash:/s122a sup-2
boot kickstart bootflash:/k122a sup-2
boot asm-sfn bootflash:/ilc1.bin module 4

ip domain-name cisco.com
ip name-server 144.254.10.123
ip default-gateway 10.48.69.129
ip route 100.100.100.4 255.255.255.252 100.100.100.2 distance 2

!--- The next hop IP address is 100.100.100.2 
!--- and is owned by the intermediate 
!--- Ethernet multilayer switch Aspendos.

zone name z-fcip2 vsan 600
member pwwn 50:05:07:63:00:d0:94:4c
member pwwn 10:00:00:00:c9:32:a6:e3

zone name Zone_a1 vsan 601
member pwwn 10:00:00:00:00:01:00:00
member pwwn 10:00:00:00:00:05:00:00

zone default-zone permit vsan 1
zone default-zone permit vsan 603

zoneset distribute full vsan 600

zoneset name zs-fcip2 vsan 600
member z-fcip2

zoneset name Agilent_1 vsan 601
member Zone_a1

zoneset activate name zs-fcip2 vsan 600

zoneset activate name Agilent_1 vsan 601

interface fc3/1
no shutdown

!--- Output supressed.

interface fc3/8
no shutdown

interface mgmt0
ip address 10.48.69.151 255.255.255.192

interface GigabitEthernet4/1
no shutdown

interface GigabitEthernet4/1.1000
ip address 100.100.100.1 255.255.255.252

switchport mtu 3000
no shutdown

!--- Here the subinterface 1000 is configured, 
!--- which ties into dot1q VLAN 1000 on the Ethernet switch.
!--- The MTU size is changed from the default 1500 bytes to 3000, 
!--- because the intermediate switch supports jumbo frames 
!--- on both L2 and L3.

MDS 9216 (Canterbury) with IPS-8 module
canterbury# sh run

Building Configuration ...
fcip profile 1
ip address 100.100.100.6

vsan database
vsan 600 
vsan 601 

fcdomain domain 2 preferred vsan 600
fcdomain domain 2 preferred vsan 601

interface fcip1
use-profile 1
peer-info ipaddr 100.100.100.1

vsan database
vsan 600 interface fc1/1
vsan 601 interface fc1/8

boot system bootflash:/s122a 
boot kickstart bootflash:/k122a 
fcalias name test vsan 1

ip domain-name cisco.com
ip name-server 144.254.10.123
ip default-gateway 10.48.69.129
ip route 10.61.0.0 255.255.0.0
ip route 10.61.0.0 255.255.0.0 10.48.69.200
ip route 100.100.100.0 255.255.255.252 100.100.100.5 distance 2

!--- Static IPS route required to reach the FCIP peer address.
 
line vty
exec-timeout 0

switchname canterbury
system default switchport trunk mode auto

zone name z-fcip2 vsan 600
member pwwn 50:05:07:63:00:d0:94:4c
member pwwn 10:00:00:00:c9:32:a6:e3

zone default-zone permit vsan 777
zoneset distribute full vsan 600

zoneset name zs-fcip2 vsan 600
member z-fcip2

zoneset activate name zs-fcip2 vsan 600
zoneset activate name Agilent_1 vsan 601

interface GigabitEthernet2/1
no shutdown

interface GigabitEthernet2/1.500
ip address 100.100.100.6 255.255.255.252

switchport mtu 3000
no shutdown

interface fc1/1
no shutdown

interface fc1/8
no shutdown

interface mgmt0
ip address 10.48.69.156 255.255.255.128

Catalyst 6000 (Aspendos) with IPS-8 module
Aspendos> (enable) sh vlan 500

VLAN Name                             Status    IfIndex Mod/Ports, Vlans
---- -------------------------------- --------- ------- -------------------
500  VLAN0500                         active    191     1/1
                                                        3/3
                                                        15/1

VLAN Type SAID MTU  Parent RingNo BrdgNo Stp    Brdg   Mode   Trans1 Trans2
---- ---- ---- ---- ------ ------ ------ ------ ------ ------ ------ ------
500  enet 100  500  1500   -      -      -      -      -      0      0


VLAN MISTP-Inst DynCreated RSPAN
---- ---------- ---------- --------
500  -          static     disabled

Aspendos> (enable) sh vlan 1000

VLAN Name                             Status    IfIndex Mod/Ports, Vlans
---- -------------------------------- --------- ------- -------------------
1000 fcip-extra-hop                   active    131     3/7
                                                        15/1

VLAN Type SAID MTU  Parent RingNo BrdgNo Stp    Brdg   Mode   Trans1 Trans2
---- ---- ---- ---- ------ ------ ------ ------ ------ ------ ------ ------
1000 enet 101  000  1500   -      -      -      -      -      0      0

VLAN MISTP-Inst DynCreated RSPAN
---- ---------- ---------- --------
1000 -          static     disabled

Aspendos> (enable) sh trunk 3/3

* - indicates vtp domain mismatch
Port Mode Encapsulation Status        Native vlan
---- ---- ------------- ------------- -----------
3/3  on   dot1q         trunking      1

Port Vlans allowed on trunk
---- ----------------------------------------------------------------------
3/3  1,500

Port Vlans allowed and active in management domain 
---- ----------------------------------------------------------------------
3/3  1,500

Port Vlans in spanning tree forwarding state and not pruned
---- ----------------------------------------------------------------------
3/3  1,500

Aspendos> (enable) sh trunk 3/7

* - indicates vtp domain mismatch
Port Mode Encapsulation Status       Native vlan
---- ---- ------------- ------------ -----------
3/7  on   dot1q         trunking     1

Port Vlans allowed on trunk
---- ----------------------------------------------------------------------
3/7  1,1000

Port Vlans allowed and active in management domain 
---- ----------------------------------------------------------------------
3/7  1,1000

Port Vlans in spanning tree forwarding state and not pruned
---- ----------------------------------------------------------------------
3/7  1,1000

!--- Note that VLAN 500 is trunked on port 3/3, which is physically connected 
!--- to MDS Canterbury while port 3/7 trunks for VLAN 1000 and is connected 
!--- to MDS Bison. The port 15/1 is an internal port and directs to the 
!--- multilayer entitity in the switch.


Aspendos> (enable) sh port jumbo 

Jumbo frames MTU size is 9216 bytes.
Jumbo frames enabled on port(s) 1/2,3/1-16.

!--- L2 Jumbo support must be enabled for all relevant ports.
 

Aspendos> (enable) sh spantree 3/3 

Port Vlan Port-State Cost Prio Portfast Channel_id
---- ---- ---------- ---- ---- -------- ----------
3/3  1    forwarding 4    32   disabled 0 
3/3  500  forwarding 4    32   disabled 0 

Aspendos> (enable) sh spantree 3/7

Port Vlan Port-State Cost Prio Portfast Channel_id
---- ---- ---------- ---- ---- -------- ----------
3/7  1    forwarding 4    32   disabled 0 
3/7  1000 forwarding 4    32   disabled 0 

Aspendos> (enable) sh spantree 15/1

Port Vlan Port-State Cost Prio Portfast Channel_id
---- ---- ---------- ---- ---- -------- ----------
15/1 192  forwarding 4    32   enabled  0 
15/1 500  forwarding 4    32   enabled  0 
15/1 691  forwarding 4    32   enabled  0 
15/1 1000 forwarding 4    32   enabled  0

!--- All relevant ports on the Catalyst are forwarding 
!--- for their respective VLANs.


Aspendos> (enable) ses 15

Trying Router-15...
Connected to Router-15.
Escape character is '^]'.

User Access Verification

Password: 
Aspendos_MSFC2> en
Password: 
Aspendos_MSFC2#

!
interface Vlan500
mtu 3000
ip address 100.100.100.5 255.255.255.252
end

Aspendos_MSFC2# sh run int vlan 1000
Building configuration...

Current configuration : 113 bytes
!
interface Vlan1000
description "test-vlan-fcip-1000"
mtu 3000
ip address 100.100.100.2 255.255.255.252
end

!--- The VLANs 500 and 1000 are configured properly on the MSFC 
!--- of the Catalyst 6000.
!--- Both subnets are directly connected to the same Ethernet 
!--- switch, so no additional routes are needed here. Also note 
!--- that the MTU size is changed accordingly on the MDS Gigabit 
!--- interfaces to 3000 bytes. This change is required to support 
!--- jumbo multilayer switching.

Verify

This section provides information you can use to confirm that your configuration is working properly.

Certain show commands are supported by the Output Interpreter Tool (registered customers only) , which allows you to view an analysis of show command output.

  • show interface gig x/y —Displays status of the relevant Gigabit interface bound to the FCIP profile.

  • show ips stats tcp int gig x/y —Displays TCP statistics and active connections for the relevant Gigabit interface.

  • show ips arp int gig x/y —Displays all Address Resolution Protocol (ARP) entries for the relevant Gigabit interface; the next hop or peer should be present in this list.

  • show ips ip route int gig x/y —Displays the specific routes going across the relevant Gigabit interface.

  • show interface fcip x —Displays the FCIP interface status and all details related to this FCIP tunnel.

  • show profile fcip x —Displays the IP address to which the profile is bound and all configured TCP parameters.

  • show int fcip x counters—Used to check if there are any frames going through the FCIP tunnel.

  • show fcdomain vsan x —Lists all domain-related details; used to verify that the fabric is formed across the FCIP tunnel(s).

  • show fcns da vsan x —Displays all pwwn, FC4-Types, and FCIDs of the relevant VSAN; used to verify that all expected entries are distributed across the FCIP tunnel(s).

Note: The dot1q interface acts as a normal physical interface for FCIP. There is no specific requirement to verify proper operation other than the show interface gig x/y.z command where z represents the subinterface. Depending upon the device connected to the local MDS Gigabit interfaces, you must consult the related documentation to verify that interfaces and ports are configured correctly.

Troubleshoot

Be sure to issue the show commands multiple times to build a counter history. Counters that are not related to a point in time and just collected only once are mostly useless.

Use the configurations shown below for more troubleshooting.

MDS 9509 (Bison)
bison# sh int gig 4/1.1000

GigabitEthernet4/1.1000 is up
    Hardware is GigabitEthernet, address is 0005.3000.a85a
    Internet address is 100.100.100.1/30
    MTU 3000 bytes

!--- Configured to 3000 bytes.

    5 minutes input rate 488 bits/sec, 61 bytes/sec, 0 frames/sec
    5 minutes output rate 488 bits/sec, 61 bytes/sec, 0 frames/sec
    1785 packets input, 996030 bytes
      0 multicast frames, 0 compressed
      0 input errors, 0 frame, 0 overrun 0 fifo
    1812 packets output, 354152 bytes, 0 underruns
      0 output errors, 0 collisions, 0 fifo
      0 carrier errors

!--- MTU is configured to 3000 bytes to avoid unnecessary 
!--- TCP segmentation and limit overhead.


bison# sh ips stats tcp int gig 4/1 de

TCP Statistics for port GigabitEthernet4/1
    TCP send stats
      337202017 segments, 222637392068 bytes
      130562402 data, 205533417 ack only packets
      503 control (SYN/FIN/RST), 0 probes, 1105737 window updates
      7 segments retransmitted, 2208 bytes
      4 retransmitted while on ethernet send queue, 40061909 packets split
      250922624 delayed acks sent
    TCP receive stats
      932985742 segments, 921498012 data packets in sequence, 
        936715052100 bytes in sequence
      770241 predicted ack, 856752348 predicted data
      0 bad checksum, 0 multi/broadcast, 0 bad offset
      0 no memory drops, 0 short segments
      0 duplicate bytes, 16 duplicate packets
      0 partial duplicate bytes, 0 partial duplicate packets
      53128 out-of-order bytes, 165 out-of-order packets
      0 packet after window, 0 bytes after window
      5 packets after close
      76225562 acks, 192030009160 ack bytes, 0 ack toomuch,
        5851 duplicate acks
      0 ack packets left of snd_una, 0 non-4 byte aligned packets
      9124012 window updates, 0 window probe
      1381 pcb hash miss, 984 no port, 103 bad SYN, 0 paws drops
    TCP Connection Stats
      272 attempts, 107 accepts, 163 established
      511 closed, 3 drops, 206 conn drops
      3 drop in retransmit timeout, 20 drop in keepalive timeout
      0 drop in persist drops, 0 connections drained
    TCP Miscellaneous Stats
      61792500 segments timed, 76225541 rtt updated
      124 retransmit timeout, 0 persist timeout
      5760 keepalive timeout, 5740 keepalive probes
    TCP SACK Stats
      0 recovery episodes, 0 data packets, 0 data bytes
      0 data packets retransmitted, 0 data bytes retransmitted
      0 connections closed, 0 retransmit timeouts
    TCP SYN Cache Stats
      107 entries, 107 connections completed, 0 entries timed out
      0 dropped due to overflow, 0 dropped due to RST
      0 dropped due to ICMP unreach, 0 dropped due to bucket overflow
      0 abort due to no memory, 0 duplicate SYN, 0 no-route SYN drop
      0 hash collisions, 0 retransmitted

100.100.100.1:64860 100.100.100.6:3225 ESTABLISH 0 0
100.100.100.1:64862 100.100.100.6:3225 ESTABLISH 0 0
100.100.100.1:3225 0.0.0.0:0 LISTEN 0 0

!--- No specific counters are maintained per subinterface.
!--- All stats for all subinterfaces on Gig 4/1 are seen here.


bison# sh ips arp interface gig 4/1

Protocol Address Age (min) Hardware Addr Type Interface
Internet 100.100.100.2 13 0008.e21e.c7bc ARPA GigabitEthernet4/1.1000 

!--- No specific ARP table is maintained per subinterface.
!--- All entries for all subinterface on Gig4/1 are seen here.


bison# sh ips ip route int gig 4/1

Codes: C - connected, S - static

No default gateway

S 100.100.100.4/30 via 100.100.100.2, GigabitEthernet4/1.1000
C 100.100.100.0/30 is directly connected, GigabitEthernet4/1.1000

!--- IPS routes are derived from the main-interface, 
!--- not specifically per subinterface. The next hop is verified here.


bison# sh cdp ne int gig 4/1

Capability Codes: R - Router, T - Trans-Bridge, B - Source-Route-Bridge
S - Switch, H - Host, I - IGMP, r - Repeater

Device ID   Local Intrfce Hldtme Capability Platform Port ID
----------- ----- ------- ------ ---------- -------- -------
TBM06033144 (Aspe Gig4/1  136    T S        WS-C6506 3/7 

!--- Use this command if the neighbor supports it to verify 
!--- physical connectivity.


bison# sh fcip profile 1

FCIP Profile 1
Internet Address is 100.100.100.1 (interface GigabitEthernet4/1.1000)
Listen Port is 3225
TCP parameters
SACK is enabled
PMTU discovery is enabled, reset timeout is 3600 sec
Keep alive is 60 sec
Minimum retransmission timeout is 200 ms
Maximum number of re-transmissions is 4
Send buffer size is 0 KB
Maximum allowed bandwidth is 1000000 kbps
Minimum available bandwidth is 15000 kbps
Estimated round trip time is 1000 usec
Congestion window monitoring is enabled, burst size is 10 KB 

!--- The profile parameters are an easy way to directly verify 
!--- the configured TCP/IP parameters per FCIP instance.
 

bison# sh int fcip 1

fcip1 is trunking
Hardware is GigabitEthernet
Port WWN is 20:c2:00:05:30:00:7a:de
Peer port WWN is 20:42:00:0c:30:6c:24:40
Admin port mode is auto, trunk mode is on
Port mode is TE
vsan is 1
Trunk vsans (allowed active) (1,600-601)
Trunk vsans (operational) (1,600-601)
Trunk vsans (up) (1,600-601)
Trunk vsans (isolated) ()
Trunk vsans (initializing) ()
Using Profile id 1 (interface GigabitEthernet4/1.1000)
Peer Information
Peer Internet address is 100.100.100.6 and port is 3225
Special Frame is disabled
Maximum number of TCP connections is 2
Time Stamp is disabled
QOS control code point is 0
QOS data code point is 0
B-port mode disabled
TCP Connection Information
2 Active TCP connections
Control connection: Local 100.100.100.1:64860, Remote 100.100.100.6:3225
Data connection: Local 100.100.100.1:64862, Remote 100.100.100.6:3225
2 Attempts for active connections, 0 close of connections
TCP Parameters
Path MTU 3000 bytes
Current retransmission timeout is 200 ms
Round trip time: Smoothed 2 ms, Variance: 1
Advertized window: Current: 118 KB, Maximum: 118 KB, Scale: 1
Peer receive window: Current: 118 KB, Maximum: 118 KB, Scale: 1
Congestion window: Current: 10 KB, Slow start threshold: 112 KB
5 minutes input rate 200 bits/sec, 25 bytes/sec, 0 frames/sec
5 minutes output rate 200 bits/sec, 25 bytes/sec, 0 frames/sec
1306 frames input, 891212 bytes
472 Class F frames input, 46972 bytes
834 Class 2/3 frames input, 844240 bytes
0 Error frames timestamp error 0
867 frames output, 252424 bytes
470 Class F frames output, 48860 bytes
397 Class 2/3 frames output, 203564 bytes
0 Error frames 0 reass frames

!--- Here, the specific details per FCIP interface are shown 
!--- by a running FCIP instance. You can also derive the 
!--- TCP parameters of the peer with this output.


bison# sh fcdomain vsan 600

The local switch is the Principal Switch.

Local switch run time information:
State: Stable
Local switch WWN: 22:58:00:05:30:00:7a:df
Running fabric name: 22:58:00:05:30:00:7a:df
Running priority: 2
Current domain ID: 0x01(1) 

Local switch configuration information:
State: Enabled
FCID persistence: Disabled
Auto-reconfiguration: Disabled
Contiguous-allocation: Disabled
Configured fabric name: 20:01:00:05:30:00:28:df
Configured priority: 128
Configured domain ID: 0x01(1) (preferred)

Principal switch run time information:
Running priority: 2

Interface        Role          RCF-reject
---------------- ------------- ------------
fcip1            Downstream    Disabled
---------------- ------------- ------------

bison# sh fcdomain vsan 601

The local switch is the Principal Switch.

Local switch run time information:
State: Stable
Local switch WWN: 22:59:00:05:30:00:7a:df
Running fabric name: 22:59:00:05:30:00:7a:df
Running priority: 2
Current domain ID: 0x01(1) 

Local switch configuration information:
State: Enabled
FCID persistence: Disabled
Auto-reconfiguration: Disabled
Contiguous-allocation: Disabled
Configured fabric name: 20:01:00:05:30:00:28:df
Configured priority: 128
Configured domain ID: 0x01(1) (preferred)

Principal switch run time information:
Running priority: 2

Interface        Role          RCF-reject
---------------- ------------- ------------
fcip1            Downstream    Disabled
---------------- ------------- ------------

!--- Similar to normal (E)ISL troubleshooting, verify 
!--- that the fabric is formed as expected.


bison# sh fcns da vsan 600-601

VSAN 600:
-------- ---- ----------------------- -------- ---------------------------
FCID     TYPE PWWN                    (VENDOR) FC4-TYPE:FEATURE
-------- ---- ----------------------- -------- ---------------------------
0x010001 N    10:00:00:00:c9:32:a6:e3 (Emulex) scsi-fcp:init 
0x020001 N    50:05:07:63:00:d0:94:4c (IBM)    scsi-fcp:target fc..

Total number of entries = 2

VSAN 601:
-------- ---- ----------------------- -------- ---------------------------
FCID     TYPE PWWN                    (VENDOR) FC4-TYPE:FEATURE
-------- ---- ----------------------- -------- ---------------------------
0x010001 N    10:00:00:00:c9:32:a6:e2 (Emulex) scsi-fcp:init 
0x010100 N    10:00:00:00:00:05:00:00 
0x020100 N    10:00:00:00:00:01:00:00 

Total number of entries = 3

MDS 9216 (Canterbury)
canterbury# sh int gig 2/1.500

GigabitEthernet2/1.500 is up
    Hardware is GigabitEthernet, address is 0005.3000.ade6
    Internet address is 100.100.100.6/30
    MTU 3000 bytes
    5 minutes input rate 2248 bits/sec, 281 bytes/sec, 0 frames/sec
    5 minutes output rate 696 bits/sec, 87 bytes/sec, 0 frames/sec
    2263 packets input, 482038 bytes
      0 multicast frames, 0 compressed
      0 input errors, 0 frame, 0 overrun 0 fifo
    2479 packets output, 1077822 bytes, 0 underruns
      0 output errors, 0 collisions, 0 fifo
      0 carrier errors

canterbury# sh cdp ne int gig 2/1

Capability Codes: R - Router, T - Trans-Bridge, B - Source-Route-Bridge
                  S - Switch, H - Host, I - IGMP, r - Repeater

Device ID   Local Intrfce Hldtme Capability Platform Port ID
----------- ----- ------- ------ ---------- -------- -------
TBM06033144 (Aspe Gig2/1  165    T S        WS-C6506 3/3

!--- If the neighbor supports CDP, use this command
!--- to verify physical connectivity.


canterbury# sh fcip profile 1
FCIP Profile 1
Internet Address is 100.100.100.6 (interface GigabitEthernet2/1.500)
Listen Port is 3225
TCP parameters
SACK is enabled
PMTU discovery is enabled, reset timeout is 3600 sec
Keep alive is 60 sec
Minimum retransmission timeout is 200 ms
Maximum number of re-transmissions is 4
Send buffer size is 0 KB
Maximum allowed bandwidth is 1000000 kbps
Minimum available bandwidth is 15000 kbps
Estimated round trip time is 1000 usec
Congestion window monitoring is enabled, burst size is 10 KB

canterbury# sh int fcip 1
fcip1 is trunking
Hardware is GigabitEthernet
Port WWN is 20:42:00:0c:30:6c:24:40
Peer port WWN is 20:c2:00:05:30:00:7a:de
Admin port mode is auto, trunk mode is auto
Port mode is TE
vsan is 1
Trunk vsans (allowed active) (1,600-601)
Trunk vsans (operational) (1,600-601)
Trunk vsans (up) (1,600-601)
Trunk vsans (isolated) ()
Trunk vsans (initializing) ()
Using Profile id 1 (interface GigabitEthernet2/1.500)
Peer Information
Peer Internet address is 100.100.100.1 and port is 3225
Special Frame is disabled
Maximum number of TCP connections is 2
Time Stamp is disabled
QOS control code point is 0
QOS data code point is 0
B-port mode disabled
TCP Connection Information
2 Active TCP connections
Control connection: Local 100.100.100.6:3225, Remote 100.100.100.1:64860
Data connection: Local 100.100.100.6:3225, Remote 100.100.100.1:64862
0 Attempts for active connections, 0 close of connections
TCP Parameters
Path MTU 3000 bytes
Current retransmission timeout is 200 ms
Round trip time: Smoothed 2 ms, Variance: 1
Advertized window: Current: 118 KB, Maximum: 118 KB, Scale: 1
Peer receive window: Current: 118 KB, Maximum: 118 KB, Scale: 1
Congestion window: Current: 10 KB, Slow start threshold: 112 KB
5 minutes input rate 184 bits/sec, 23 bytes/sec, 0 frames/sec
5 minutes output rate 184 bits/sec, 23 bytes/sec, 0 frames/sec
1163 frames input, 336700 bytes
722 Class F frames input, 72176 bytes
441 Class 2/3 frames input, 264524 bytes
0 Error frames timestamp error 0
1588 frames output, 917216 bytes
724 Class F frames output, 70288 bytes
864 Class 2/3 frames output, 846928 bytes
0 Error frames 0 reass frames

Note on Native VLAN Mismatch

Under some conditions, customers may see the native vlan mismatch error message in the system log of the adjacent (Catalyst) Ethernet switch. This indicates an incorrect configuration on the trunkport of those switches. If a subinterface of .1000 is configured on one of the main interfaces on the MDS IP Services module, the adjacent switchport on the Catalyst switch needs to be trunking for that same VLAN ID of 1000. If, for some reason, the Catalyst is running VLAN 1000 as Port VLAN or Native VLAN, then error messages are generated and connectivity is broken, because the dot1q encapsulated frames from the MDS side are not understood by the switch and will be discarded. An example of native vlan mismatch errors is shown below.

2003 Nov 13 02:36:39 %CDP-4-NVLANMISMATCH:Native vlan mismatch detected on port 3/7
2003 Nov 13 02:37:26 %CDP-4-NVLANMISMATCH:Native vlan mismatch detected on port 3/3

Related Information

Updated: Oct 28, 2005
Document ID: 47101