Guest

Cisco MDS 9000 Series Multilayer Switches

FC Analyzer and SPAN for MDS Switches Configuration Example

Cisco - FC Analyzer and SPAN for MDS Switches Configuration Example

Document ID: 46220

Updated: Jul 23, 2008

   Print

Introduction

Similar to the Cisco router product line’s debug capability, the Cisco MDS storage switches have a Fibre Channel (FC) analyzer to examine packets. The FC analyzer examines packets to and from the entities that the switch provides. The FC analyzer is able to debug frames that the switch is responsible for receiving or sending to a storage device. Frames between end stations can not be examined by the FC analyzer.

To examine session flow, the Switched Port Analyzer (SPAN) functionality of the MDS switches should be used. Much like the SPAN function on a Cisco Ethernet switch, SPAN on the MDS product line replicates data to SPAN destination ports, so that it can be collected by a third-party device.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

The information in this document is based on these software and hardware versions:

  • Cisco MDS 9216 Switch

  • Cisco MDS 9509 Switch

  • Both run Storage Area Networking Operating System (SAN-OS) 1.2.1a.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Conventions

For more information on document conventions, refer to the Cisco Technical Tips Conventions.

Background Theory

You must know when to use the FC analyzer tool and when to use the SPAN feature.

The FC analyzer is a tool that collects frames that are destined to or originate from the MDS supervisor. Node-to-switch or switch-to-switch traffic can be seen with this tool.

The SPAN is a feature that allows frames that are transient to the switch to be copied to a second port for analysis. Node-to-node traffic can be seen with this method.

Refer to this diagram for an illustration:

analyze_span_1.gif

The green arrows show traffic that can be traced with the FC analyzer tool, while the pink arrow shows traffic that can be captured with the SPAN method. Traffic from the host to the storage can not be observed by the FC analyzer. Only traffic from the host to the switch or from the switch on the right can be seen, when you run the FC analyzer on the switch on the left.

SPAN can be used to trace traffic in (ingress) and out (egress) of any port on the switch. Remote SPAN (RSPAN), as shown in the previous diagram, can be used to collect frames in and out of the host port on the left switch, with the analyzer attached to the right side switch.

Configure

In this section, you are presented with the information to configure the features described in this document.

Note: To find additional information on the commands used in this document, use the Command Lookup Tool (registered customers only) .

Configuring Local FC Analyzer

analyze_span_2.gif

Note: The intent is to collect FC frames that originate from, or are destined to, the 9612 supervisor. Frames from the host to JBOD are not collected with the FC analyzer tool.

FC analyzer local is run from the command-line interface (CLI) via console attachment or Telnet. You may run a brief display to show only a small portion of each frame, or you may run a detailed trace to show the entire frame.

The trace is started while in configuration mode, and it is stopped when you press Ctrl-C. By default, only 100 frames are captured. To capture more than 100 frames, add the limit-captured-frames command option to the command that you use to start the trace.

You may also use a display filter to limit the output of the trace to only specific frames.


!--- VSAN 13 (0xd) is used here as example.

MDS9216# show fcdomain domain-list vsan 13
 
Number of domains: 2
Domain ID              WWN
---------    -----------------------
0x66(102)    20:0d:00:05:30:00:47:9f [Local] [Principal]
0x6b(107)    20:0d:00:05:30:00:51:1f

MDS9216# show fcns data vsan 13

VSAN 13:
--------------------------------------------------------------------------
FCID        TYPE  PWWN                    (VENDOR)        FC4-TYPE:FEATURE
--------------------------------------------------------------------------
0x6600dc    NL    21:00:00:20:37:15:a2:49 (Seagate)       scsi-fcp:target
0x6600e0    NL    21:00:00:04:cf:6e:4a:8c (Seagate)       scsi-fcp:target
0x6600e1    NL    21:00:00:04:cf:6e:37:8b (Seagate)       scsi-fcp:target
0x660101    NL    10:00:00:01:73:00:81:82 (JNI)
0x660201    N     10:00:00:05:30:00:47:9f (Cisco)         ipfc
0x6b0001    N     10:00:00:05:30:00:51:23 (Cisco)         ipfc

Total number of entries = 6

!--- Configure FC analyzer for brief output.

MDS9216# config t

Enter configuration commands, one per line.  End with CNTL/Z.

MDS9216(config)# fcanalyzer local brief display-filter mdshdr.vsan==0xd

Capturing on eth2
  0.000000     ff.ff.fd -> ff.ff.fd     SW_ILS HLO
  0.000095     ff.ff.fd -> ff.ff.fd     FC Link Ctl, ACK1
 18.721559     ff.ff.fd -> ff.ff.fd     SW_ILS HLO
 18.721879     ff.ff.fd -> ff.ff.fd     FC Link Ctl, ACK1
 19.970287     ff.ff.fd -> ff.ff.fd     SW_ILS HLO
 19.970368     ff.ff.fd -> ff.ff.fd     FC Link Ctl, ACK1
 38.941558     ff.ff.fd -> ff.ff.fd     SW_ILS HLO
 38.941849     ff.ff.fd -> ff.ff.fd     FC Link Ctl, ACK1
 39.940546     ff.ff.fd -> ff.ff.fd     SW_ILS HLO
 39.940628     ff.ff.fd -> ff.ff.fd     FC Link Ctl, ACK1

In the next example, you have the same data. This time, however, the brief option is omitted from the command, to provide a detailed view of each packet.

MDS9216(config)# fcanalyzer local display-filter mdshdr.vsan==0xd

Capturing on eth2
Frame 1 (100 bytes on wire, 100 bytes captured)
    Arrival Time: Jul  4, 2003 12:31:18.310251000
    Time delta from previous packet: 0.000000000 seconds
    Time relative to first packet: 0.000000000 seconds
    Frame Number: 1
    Packet Length: 100 bytes
    Capture Length: 100 bytes
Ethernet II, Src: 00:00:00:00:00:0a, Dst: 00:00:00:00:ee:00
    Destination: 00:00:00:00:ee:00 (00:00:00:00:ee:00)
    Source: 00:00:00:00:00:0a (00:00:00:00:00:0a)
    Type: Unknown (0xfcfc)
Vegas (FC, SOFf/EOFn)
    Vegas Header
        .000 .... = Version: 0
        .... 0000 = Andiamo Type: Normal FC frame (0)
        #MPLS Labels: 0
        Packet Len: 70
        TTL: 255
        0111 .... = User Priority: 7
        .... 0000 0010 11.. = Dst Index: 0x000b
        .... ..00 1111 1111 = Src Index: 0x00ff
        Ctrl Bits: Index Directed frame (0x01)
        Timestamp: 42678
        .... .000 = Status: 0 (0)
        0000 0... = Reason Code: 0 (0x00)
        .... 0000 0000 1101 = VSAN: 13
        Checksum: 0
    Vegas Trailer
        EOF: EOFn (3)
        CRC: 4022250974
Fibre Channel
    R_CTL: 0x02
    Dest Addr: ff.ff.fd
    CS_CTL: 0x00
    Src Addr: ff.ff.fd
    Type: SW_ILS (0x22)
    F_CTL: 0x380000 (Exchange Originator, Seq Initiator, Exchg First,
                    Exchg Last, Seq Last, CS_CTL, Last Data Frame - No Info,
                    ABTS - Abort/MS, )
    SEQ_ID: 0xe7
    DF_CTL: 0x00
    SEQ_CNT: 0
    OX_ID: 0x1eb4
    RX_ID: 0xffff
    Parameter: 0x00000000
SW_ILS
    Cmd Code: HLO (0x14)
    FSPF Header
        Version: 0x02
        AR Number: 0x00
        Authentication Type: 0x00
        Originating Domain ID: 102
        Authentication: 0000000000000000
    Options: 00000000
    Hello Interval (secs): 20
    Dead Interval (secs): 80
    Recipient Domain ID: 107
    Originating Port Idx: 0x01000b

Frame 2 (60 bytes on wire, 60 bytes captured)
    Arrival Time: Jul  4, 2003 12:31:18.310563000
    Time delta from previous packet: 0.000312000 seconds
    Time relative to first packet: 0.000312000 seconds
    Frame Number: 2
    Packet Length: 60 bytes
    Capture Length: 60 bytes
Ethernet II, Src: 00:00:00:00:00:00, Dst: 00:00:00:00:00:00
    Destination: 00:00:00:00:00:00 (00:00:00:00:00:00)
    Source: 00:00:00:00:00:00 (00:00:00:00:00:00)
    Type: Unknown (0x0000)
Vegas (FC, SOFf/EOFt)
    Vegas Header
        .000 .... = Version: 0
        .... 0000 = Andiamo Type: Normal FC frame (0)
        #MPLS Labels: 0
        Packet Len: 30
        TTL: 255
        0111 .... = User Priority: 7
        .... 0011 1111 11.. = Dst Index: 0x00ff
        .... ..00 0000 1011 = Src Index: 0x000b
        Ctrl Bits: 0 (0x00)
        Timestamp: 42679
        .... .000 = Status: 0 (0)
        0000 0... = Reason Code: 0 (0x00)
        .... 0000 0000 1101 = VSAN: 13
        Checksum: 241
    Vegas Trailer
        EOF: EOFt (1)
        CRC: 1019832848
Fibre Channel
    R_CTL: 0xc0(ACK1)
    Dest Addr: ff.ff.fd
    CS_CTL: 0x00
    Src Addr: ff.ff.fd
    Type: Unknown (0x00)
    F_CTL: 0xf80000 (Exchange Responder, Seq Recipient, Exchg First,
                    Exchg Last, Seq Last, CS_CTL, Last Data Frame - No Info,
                    ABTS - Cont, )
    SEQ_ID: 0xe7
    DF_CTL: 0x00
    SEQ_CNT: 0
    OX_ID: 0x1eb4
    RX_ID: 0x1e66
    Parameter: 0x00000001

Frame 3 (100 bytes on wire, 100 bytes captured)
    Arrival Time: Jul  4, 2003 12:31:19.309559000
    Time delta from previous packet: 0.998996000 seconds
    Time relative to first packet: 0.999308000 seconds
    Frame Number: 3
    Packet Length: 100 bytes
    Capture Length: 100 bytes
Ethernet II, Src: 00:00:00:00:00:00, Dst: 00:00:00:00:00:00
    Destination: 00:00:00:00:00:00 (00:00:00:00:00:00)
    Source: 00:00:00:00:00:00 (00:00:00:00:00:00)
    Type: Unknown (0x0000)
Vegas (FC, SOFf/EOFn)
    Vegas Header
        .000 .... = Version: 0
        .... 0000 = Andiamo Type: Normal FC frame (0)
        #MPLS Labels: 0
        Packet Len: 70
        TTL: 255
        0111 .... = User Priority: 7
        .... 0011 1111 11.. = Dst Index: 0x00ff
        .... ..00 0000 1011 = Src Index: 0x000b
        Ctrl Bits: 0 (0x00)
        Timestamp: 42779
        .... .000 = Status: 0 (0)
        0000 0... = Reason Code: 0 (0x00)
        .... 0000 0000 1101 = VSAN: 13
        Checksum: 101
    Vegas Trailer
        EOF: EOFn (3)
        CRC: 4200187557
Fibre Channel
    R_CTL: 0x02
    Dest Addr: ff.ff.fd
    CS_CTL: 0x00
    Src Addr: ff.ff.fd
    Type: SW_ILS (0x22)
    F_CTL: 0x380000 (Exchange Originator, Seq Initiator, Exchg First,
                    Exchg Last, Seq Last, CS_CTL, Last Data Frame - No Info,
                    ABTS - Abort/MS, )
    SEQ_ID: 0xe7
    DF_CTL: 0x00
    SEQ_CNT: 0
    OX_ID: 0x1e67
    RX_ID: 0xffff
    Parameter: 0x00000000
SW_ILS
    Cmd Code: HLO (0x14)
    FSPF Header
        Version: 0x02
        AR Number: 0x00
        Authentication Type: 0x00
        Originating Domain ID: 107
        Authentication: 0000000000000000
    Options: 00000000
    Hello Interval (secs): 20
    Dead Interval (secs): 80
    Recipient Domain ID: 102
    Originating Port Idx: 0x01011c

Frame 4 (60 bytes on wire, 60 bytes captured)
    Arrival Time: Jul  4, 2003 12:31:19.309646000
    Time delta from previous packet: 0.000087000 seconds
    Time relative to first packet: 0.999395000 seconds
    Frame Number: 4
    Packet Length: 60 bytes
    Capture Length: 60 bytes
Ethernet II, Src: 00:00:00:00:00:0a, Dst: 00:00:00:00:ee:00
    Destination: 00:00:00:00:ee:00 (00:00:00:00:ee:00)
    Source: 00:00:00:00:00:0a (00:00:00:00:00:0a)
    Type: Unknown (0xfcfc)
Vegas (FC, SOFf/EOFt)
    Vegas Header
        .000 .... = Version: 0
        .... 0000 = Andiamo Type: Normal FC frame (0)
        #MPLS Labels: 0
        Packet Len: 30
        TTL: 255
        0111 .... = User Priority: 7
        .... 0000 0010 11.. = Dst Index: 0x000b
        .... ..00 1111 1111 = Src Index: 0x00ff
        Ctrl Bits: Index Directed frame (0x01)
        Timestamp: 42778
        .... .000 = Status: 0 (0)
        0000 0... = Reason Code: 0 (0x00)
        .... 0000 0000 1101 = VSAN: 13
        Checksum: 0
    Vegas Trailer
        EOF: EOFt (1)
        CRC: 4022250974
Fibre Channel
    R_CTL: 0xc0(ACK1)
    Dest Addr: ff.ff.fd
    CS_CTL: 0x00
    Src Addr: ff.ff.fd
    Type: Unknown (0x00)
    F_CTL: 0xf80000 (Exchange Responder, Seq Recipient, Exchg First,
                    Exchg Last, Seq Last, CS_CTL, Last Data Frame - No Info,
                    ABTS - Cont, )
    SEQ_ID: 0xe7
    DF_CTL: 0x00
    SEQ_CNT: 0
    OX_ID: 0x1e67
    RX_ID: 0x1eb5
    Parameter: 0x00000001

Again, the brief trace is shown. This time, however, the PC on port 1/16 is unplugged and re-plugged to force a log in. You see frames to and from the other FC switch, and to and from the attached local node (the PC).

MDS9216(config)# fcanalyzer local brief display-filter mdshdr.vsan==0xd

Capturing on eth2
  0.000000     ff.ff.fd -> ff.ff.fd     SW_ILS HLO
  0.000310     ff.ff.fd -> ff.ff.fd     FC Link Ctl, ACK1
  0.999598     ff.ff.fd -> ff.ff.fd     SW_ILS HLO
  0.999684     ff.ff.fd -> ff.ff.fd     FC Link Ctl, ACK1
 19.990040     ff.ff.fd -> ff.ff.fd     SW_ILS HLO
 19.990295     ff.ff.fd -> ff.ff.fd     FC Link Ctl, ACK1
 20.990602     ff.ff.fd -> ff.ff.fd     SW_ILS HLO
 20.990682     ff.ff.fd -> ff.ff.fd     FC Link Ctl, ACK1
 26.028780     ff.fc.66 -> ff.fc.6b     SW_ILS SW_RSCN
 26.029087     ff.fc.6b -> ff.fc.66     FC Link Ctl, ACK1
 26.029541     ff.fc.6b -> ff.fc.66     SW_ILS SW_ACC (SW_RSCN)
 26.029596     ff.fc.66 -> ff.fc.6b     FC Link Ctl, ACK1
 31.151197     00.00.01 -> ff.ff.fe     FC ELS FLOGI
 31.162809     ff.ff.fe -> 66.01.01     FC ELS ACC (FLOGI)
 31.162841     ff.ff.fe -> 66.01.01     FC ELS ACC (FLOGI)
 31.163139     66.01.01 -> ff.ff.fd     FC ELS SCR
 31.163583     ff.ff.fd -> 66.01.01     FC ELS ACC (SCR)
 31.163603     ff.ff.fd -> 66.01.01     FC ELS ACC (SCR)
 31.163835     66.01.01 -> ff.ff.fc     FC ELS PLOGI
 31.163965     ff.ff.fc -> 66.01.01     FC ELS ACC (PLOGI)
 31.163985     ff.ff.fc -> 66.01.01     FC ELS ACC (PLOGI)
 31.164186     66.01.01 -> ff.ff.fc     dNS GA_NXT
 31.164305     ff.fc.66 -> ff.fc.6b     SW_ILS SW_RSCN
 31.164479     ff.fc.6b -> ff.fc.66     FC Link Ctl, ACK1
 31.164628     ff.fc.6b -> ff.fc.66     SW_ILS SW_ACC (SW_RSCN)
 31.164670     ff.fc.66 -> ff.fc.6b     FC Link Ctl, ACK1
 31.165030     ff.ff.fc -> 66.01.01     dNS ACC (GA_NXT)
 31.165050     ff.ff.fc -> 66.01.01     dNS ACC (GA_NXT)
 31.165125     ff.fc.6b -> ff.fc.66     dNS GE_ID
 31.165193     ff.fc.66 -> ff.fc.6b     FC Link Ctl, ACK1
 31.165419     66.01.01 -> ff.ff.fc     dNS GA_NXT
 31.165577     ff.fc.66 -> ff.fc.6b     dNS ACC (GE_ID)
 31.165781     ff.ff.fc -> 66.01.01     dNS ACC (GA_NXT)
 31.165804     ff.ff.fc -> 66.01.01     dNS ACC (GA_NXT)
 31.165943     ff.fc.6b -> ff.fc.66     FC Link Ctl, ACK1
 31.166063     66.01.01 -> ff.ff.fc     dNS GA_NXT
 31.166870     ff.ff.fc -> 66.01.01     dNS ACC (GA_NXT)
 31.166892     ff.ff.fc -> 66.01.01     dNS ACC (GA_NXT)
 31.167268     66.01.01 -> ff.ff.fc     dNS GA_NXT
 31.167529     ff.ff.fc -> 66.01.01     dNS ACC (GA_NXT)
 31.167549     ff.ff.fc -> 66.01.01     dNS ACC (GA_NXT)
 31.168704     66.01.01 -> ff.ff.fc     dNS GA_NXT
 31.169272     ff.ff.fc -> 66.01.01     dNS ACC (GA_NXT)
 31.169294     ff.ff.fc -> 66.01.01     dNS ACC (GA_NXT)
 31.169568     66.01.01 -> ff.ff.fc     dNS GA_NXT
 31.170453     ff.ff.fc -> 66.01.01     dNS ACC (GA_NXT)
 31.170473     ff.ff.fc -> 66.01.01     dNS ACC (GA_NXT)
 31.170756     66.01.01 -> ff.ff.fc     dNS GA_NXT
 31.170975     ff.ff.fc -> 66.01.01     dNS ACC (GA_NXT)
 31.170994     ff.ff.fc -> 66.01.01     dNS ACC (GA_NXT)
 31.171400     66.01.01 -> 66.02.01     FC ELS PLOGI
 31.171562     66.02.01 -> 66.01.01     FC ELS ACC (PLOGI)
 31.171581     66.02.01 -> 66.01.01     FC ELS ACC (PLOGI)
 31.171752     66.01.01 -> 66.02.01     FC ELS PRLI
 31.171812     66.02.01 -> 66.01.01     FC ELS LS_RJT (PRLI)
 31.171832     66.02.01 -> 66.01.01     FC ELS LS_RJT (PRLI)
 31.173863     66.01.01 -> ff.ff.fc     FC ELS LOGO
 31.175020     ff.ff.fc -> 66.01.01     FC ELS ACC (LOGO)
 31.175047     ff.ff.fc -> 66.01.01     FC ELS ACC (LOGO)
 31.175182     66.01.01 -> ff.ff.fc     FC ELS PLOGI
 31.175290     ff.ff.fc -> 66.01.01     FC ELS ACC (PLOGI)
 31.175310     ff.ff.fc -> 66.01.01     FC ELS ACC (PLOGI)
 31.175632     66.01.01 -> ff.ff.fa     FC ELS PLOGI
 31.175753     ff.ff.fa -> 66.01.01     FC ELS ACC (PLOGI)
 31.175777     ff.ff.fa -> 66.01.01     FC ELS ACC (PLOGI)
 32.460020     ff.fc.66 -> 66.01.01     FC ELS PLOGI
 32.460050     ff.fc.66 -> 66.01.01     FC ELS PLOGI
 32.460207     66.01.01 -> ff.fc.66     FC ELS ACC (PLOGI)
 32.460246     66.01.01 -> ff.fc.66     FC ELS ACC (PLOGI)
 32.460340     ff.fc.66 -> 66.01.01     FC ELS PRLI
 32.460362     ff.fc.66 -> 66.01.01     FC ELS PRLI
 32.460492     66.01.01 -> ff.fc.66     FC ELS LS_RJT (PRLI)
 32.460525     66.01.01 -> ff.fc.66     FC ELS LS_RJT (PRLI)
 32.461839     ff.fc.66 -> 66.01.01     FC ELS LOGO
 32.461866     ff.fc.66 -> 66.01.01     FC ELS LOGO
 32.462046     66.01.01 -> ff.fc.66     FC ELS ACC (LOGO)
 32.462080     66.01.01 -> ff.fc.66     FC ELS ACC (LOGO)

MDS9216(config)# ^C

MDS9216(config)# exit

Configuring for Remote FC Analyzer

analyze_span_3.gif

Note: The intent is to collect FC frames that originate from, or are destined to, the 9612 supervisor. Frames from the host to JBOD are not collected with the FC analyzer tool.

FC analyzer remote is run on a PC that is using Ethereal leavingcisco.com 0.9(9) or later and WinPcap leavingcisco.com. The IP address of the PC is specified in the command that is issued to start the FC analyzer trace on the MDS CLI. On the PC, Ethereal must also be started from the command line, and the IP address of the MDS management interface must be specified in the command.

  1. To stop the MDS FC analyzer trace, you must press Ctrl-C from the CLI.

    MDS9216# config t
    
    Enter configuration commands, one per line.  End with CNTL/Z.
    
    MDS9216(config)# fcanalyzer remote 64.102.58.114
    
    MDS9216(config)# ^C
    

    Do not specify the active option on the previous command, or you will need to add additional options to the command line on your PC when you start Ethereal. Adding the active keyword usually means that you have to also configure the TCP port number. It is recommended that you use the defaults.

  2. On the PC, verify the IP address, and start the Ethereal remote capture program.

    d:\> ipconfig
    
    Windows 2000 IP Configuration
    
    Ethernet adapter wireless:
    
            Connection-specific DNS Suffix  . : cisco.com
            IP Address. . . . . . . . . . . . : 64.102.58.114
            Subnet Mask . . . . . . . . . . . : 255.255.255.128
            Default Gateway . . . . . . . . . : 64.102.58.1
    
    Ethernet adapter builtinE:
    
            Connection-specific DNS Suffix  . : cisco.com
            Autoconfiguration IP Address. . . : 169.254.219.141
            Subnet Mask . . . . . . . . . . . : 255.255.0.0
            Default Gateway . . . . . . . . . :
    
    d:\> cd ethereal099
    
    D:\Ethereal099> ethereal099 -i rpcap://172.18.172.56/eth2
    
  3. Once the program starts, choose Capture and then click OK to initiate packet collection.

    analyze_span_4.gif

    The FC packets that are collected appear as Other in the summary display.

    analyze_span_5.gif

  4. Click Stop to halt packet collection and start the trace view portion of the program.

    analyze_span_6.gif

    You may use filters to limit the display to a specific traffic stream.

  5. If there is problem with the remote capture initiation, you may see an error screen similar to the one in the next image. The FC analyzer is not active on the MDS, or the active keyword was used without a specified port.

    analyze_span_7.gif

Configuring for Local SPAN

analyze_span_8.gif

Note: The intent is to collect—with the FC analyzer on port 1/15—FC frames to and from the host on port 1/16 of the 9216.

An FC analyzer on port 1/15 shows ordered sets, but not the ordered sets that occur on the link that is being SPANed. The FC analyzer device can be a Port Analyzer Adapter (PAA) and a PC that is running Ethereal, similar to a Finisar device.

MDS 9216 Configuration

MDS9216# show run

vsan 13

vsan 13 interface fc1/16
vsan 13 interface fc2/16

boot system bootflash:/m9200-ek9-mzg.1.2.0.77.bin
boot kickstart bootflash:/m9200-ek9-kickstart-mzg.1.2.0.77.bin

 interface fc1/15
switchport mode SD
switchport speed 2000
no shutdown

 interface fc1/16
no shutdown

 interface mgmt0
ip address 172.18.172.56 255.255.255.0

span session 1
destination interface fc1/15
source interface fc1/16 rx

source interface fc1/16 tx

MDS 9216 Displays

MDS9216# show interface fc 1/15

fc1/15 is up
    Hardware is Fibre Channel
    Port WWN is 20:0f:00:05:30:00:47:9e
    Admin port mode is SD
    Port mode is SD
    Port vsan is 1
    Speed is 2 Gbps
    Beacon is turned off
    5 minutes input rate 73704 bits/sec, 9213 bytes/sec, 13 frames/sec
    5 minutes output rate 2275584 bits/sec, 284448 bytes/sec, 430 frames/sec
      2839098 frames input, 1883173240 bytes
        0 discards, 0 errors
        0 CRC,  0 unknown class
        0 too long, 0 too short
      3049460 frames output, 2038253240 bytes
        0 discards, 0 errors
      0 input OLS, 0 LRR, 0 NOS, 0 loop inits
      0 output OLS, 0 LRR, 0 NOS, 0 loop inits

MDS9216# show interface fc 1/16 

fc1/16 is up
    Hardware is Fibre Channel
    Port WWN is 20:10:00:05:30:00:47:9e
    Admin port mode is auto, trunk mode is on
    Port mode is FL, FCID is 0x660100
    Port vsan is 13
    Speed is 2 Gbps
    Transmit B2B Credit is 0
    Receive B2B Credit is 16
    Receive data field Size is 2112
    Beacon is turned off
    5 minutes input rate 771568 bits/sec, 96446 bytes/sec, 171 frames/sec
    5 minutes output rate 1503144 bits/sec, 187893 bytes/sec, 258 frames/sec
      1238843 frames input, 691853044 bytes
        0 discards, 0 errors
        0 CRC,  0 unknown class
        0 too long, 0 too short
      1864744 frames output, 1357707740 bytes
        0 discards, 0 errors
      0 input OLS, 0 LRR, 0 NOS, 49 loop inits
      10 output OLS, 0 LRR, 10 NOS, 14 loop inits

MDS9216# show interface fc 2/16

fc2/16 is up
    Hardware is Fibre Channel
    Port WWN is 20:50:00:05:30:00:47:9e
    Admin port mode is FX
    Port mode is FL, FCID is 0x660000
    Port vsan is 13
    Speed is 1 Gbps
    Transmit B2B Credit is 0
    Receive B2B Credit is 12
    Receive data field Size is 2112
    Beacon is turned off
    5 minutes input rate 1647552 bits/sec, 205944 bytes/sec, 283 frames/sec
    5 minutes output rate 845624 bits/sec, 105703 bytes/sec, 188 frames/sec
      1867680 frames input, 1361393600 bytes
        0 discards, 0 errors
        0 CRC,  0 unknown class
        0 too long, 0 too short
      1241179 frames output, 694505284 bytes
        0 discards, 0 errors
      0 input OLS, 0 LRR, 0 NOS, 2 loop inits
      0 output OLS, 0 LRR, 0 NOS, 2 loop inits

MDS9216# show fcns data vsan 13

VSAN 13:
--------------------------------------------------------------------------
FCID        TYPE  PWWN                    (VENDOR)        FC4-TYPE:FEATURE
--------------------------------------------------------------------------
0x6600dc    NL    21:00:00:20:37:15:a2:49 (Seagate)       scsi-fcp:target
0x6600e0    NL    21:00:00:04:cf:6e:4a:8c (Seagate)       scsi-fcp:target
0x6600e1    NL    21:00:00:04:cf:6e:37:8b (Seagate)       scsi-fcp:target
0x660101    NL    10:00:00:01:73:00:81:82 (JNI)

Total number of entries = 4

MDS9216# show span session brief

--------------------------------------------------------
 Session  Admin          Oper           Destination
          State          State          Interface
--------------------------------------------------------
 1        no suspend     active         fc1/15

MDS9216# show span session 1

Session 1 (active)
   Destination is fc1/15
   No session filters configured
   Ingress (rx) sources are
     fc1/16,
   Egress (tx) sources are
     fc1/16,

MDS9216# show span internal info session 1

====================================
Admin Configuration for session [1]
====================================
Name:
Destination port: [100e000] [fc1/15] Flags [1]
State: [0] not suspended
Session Flags: [0] <>
Session Filter rx:  none
Session Filter tx:  none
Source interface - rx: fc1/16
Source interface - tx: fc1/16
Source vsan (rx):  none
Session [1] is UNLOCKED txn[0] cfg[0] rid[80000000]
====================================
Runtime Data for session [1]
====================================
Status <active: 0 inactive 1> : [0] active
State reason:[0] Flags [6]rx_span_bit [0] tx_span_bit[1] ( 4s invalid)
oper configured PHYSICAL ports
fc1/16
PHYSICAL ports undergoing configuration
none
PHYSICAL ports in error state
none
PHYSICAL ports (incl. dest) link status
fc1/15, fc1/16

Configuring for Remote SPAN

analyze_span_9.gif

Note: The intent is to collect—with the FC analyzer attached to the 9509—FC frames to and from the host on the 9216. The ST interface must have a Gigabit Interface Converter (GBIC) installed and the speed must match the Span Destination (SD) port on the 9509.

Before you attempt to configure RSPAN, ensure that these points are addressed:

  • All switches must be running MDS code 1.2 or later.

  • No cable should be attached to the Small Form factor pluggable (SFP) in the Span Terminal (ST) port.

  • Make sure that the FC tunnel is UP before you begin to collect frames.

  • FC analyzer can be a PAA and a PC that is running Ethereal, similar to a Finisar device.

If there are any intermediate switches between the SPAN source and the SPAN destination switch, follow this procedure:

  1. Create an active VSAN interface in the same subnet as the tunnel source and destination.

  2. Enable IP routing.

  3. Enable FC-tunneling.

  4. Use SAN-OS 1.2 or later.

MDS 9216 Configuration

MDS9216# show version

Cisco Storage Area Networking Operating System (SAN-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2003 by Cisco Systems, Inc. All rights reserved.
The copyright for certain works contained herein are owned by
Andiamo Systems, Inc. and/or other third parties and are used and
distributed under license.

Software
  BIOS:      version 1.0.7
  loader:    version 1.0(3a)
  kickstart: version 1.2(1) [build 1.2(0.77)] [gdb]
  system:    version 1.2(1) [build 1.2(0.77)] [gdb]

  BIOS compile time:       03/20/03
  kickstart image file is: bootflash:/m9200-ek9-kickstart-mzg.1.2.0.77.bin
  kickstart compile time:  6/29/2003 0:00:00
  system image file is:    bootflash:/m9200-ek9-mzg.1.2.0.77.bin
  system compile time:     6/29/2003 0:00:00

Hardware
  RAM 963108 kB

  bootflash: 503808 blocks (block size 512b)
  slot0:          0 blocks (block size 512b)

  MDS9216 uptime is 0 days 21 hours 28 minute(s) 20 second(s)

  Last reset at 50030 usecs after Thu Jul  3 13:09:31 2003
    Reason: Reset Requested by CLI command reload
    System version: 1.2(0.45c)

MDS9216# show run

Building Configuration ...

 interface fc-tunnel 13
destination 10.0.0.2
source 10.0.0.1
no shutdown

vsan database
vsan 13

 interface vsan13
ip address 10.0.0.1 255.255.255.0
no shutdown

vsan 13 interface fc1/16
vsan 13 interface fc2/16

boot system bootflash:/m9200-ek9-mzg.1.2.0.77.bin
boot kickstart bootflash:/m9200-ek9-kickstart-mzg.1.2.0.77.bin
fc-tunnel enable

ip routing
zone default-zone permit vsan 13

 interface fc1/12
no shutdown

 interface fc1/15
switchport mode ST
switchport speed 1000
rspan-tunnel interface fc-tunnel 13
no shutdown

 interface fc1/16
no shutdown

 interface fc2/16
no shutdown

 interface mgmt0
ip address 172.18.172.56 255.255.255.0

span session 1
destination interface fc-tunnel 13
source interface fc1/16 rx

source interface fc1/16 tx

!--- Output suppressed.

MDS 9216 Displays

MDS9216# show interface fc 1/16

fc1/16 is up
    Hardware is Fibre Channel
    Port WWN is 20:10:00:05:30:00:47:9e
    Admin port mode is auto, trunk mode is on
    Port mode is FL, FCID is 0x660100
    Port vsan is 13
    Speed is 2 Gbps
    Transmit B2B Credit is 0
    Receive B2B Credit is 16
    Receive data field Size is 2112
    Beacon is turned off
    5 minutes input rate 1480080 bits/sec, 185010 bytes/sec, 331 frames/sec
    5 minutes output rate 2907712 bits/sec, 363464 bytes/sec, 498 frames/sec
      574444 frames input, 320246452 bytes
        0 discards, 0 errors
        0 CRC,  0 unknown class
        0 too long, 0 too short
      865170 frames output, 629303788 bytes
        0 discards, 0 errors
      0 input OLS, 0 LRR, 0 NOS, 10 loop inits
      5 output OLS, 0 LRR, 5 NOS, 9 loop inits

MDS9216# show interface fc 2/16

fc2/16 is up
    Hardware is Fibre Channel
    Port WWN is 20:50:00:05:30:00:47:9e
    Admin port mode is FX
    Port mode is FL, FCID is 0x660000
    Port vsan is 13
    Speed is 1 Gbps
    Transmit B2B Credit is 0
    Receive B2B Credit is 12
    Receive data field Size is 2112
    Beacon is turned off
    5 minutes input rate 2905056 bits/sec, 363132 bytes/sec, 498 frames/sec
    5 minutes output rate 1480184 bits/sec, 185023 bytes/sec, 330 frames/sec
      867932 frames input, 632889576 bytes
        0 discards, 0 errors
        0 CRC,  0 unknown class
        0 too long, 0 too short
      576681 frames output, 322771132 bytes
        0 discards, 0 errors
      0 input OLS, 0 LRR, 0 NOS, 2 loop inits
      0 output OLS, 0 LRR, 0 NOS, 2 loop inits

MDS9216# show interface fc 1/15

fc1/15 is up
    Hardware is Fibre Channel
    Port WWN is 20:0f:00:05:30:00:47:9e
    Admin port mode is ST
    Port mode is ST
    Port vsan is 1
    Speed is 1 Gbps
    Rspan tunnel is fc-tunnel 13
    Beacon is turned off
    5 minutes input rate 4391896 bits/sec, 548987 bytes/sec, 827 frames/sec
    5 minutes output rate 4391896 bits/sec, 548987 bytes/sec, 820 frames/sec
      1431232 frames input, 941079708 bytes
        0 discards, 0 errors
        0 CRC,  0 unknown class
        0 too long, 0 too short
      1406853 frames output, 941079708 bytes
        0 discards, 0 errors
      0 input OLS, 0 LRR, 0 NOS, 0 loop inits
      0 output OLS, 0 LRR, 0 NOS, 0 loop inits

MDS9216# show interface fc 1/12

fc1/12 is trunking
    Hardware is Fibre Channel
    Port WWN is 20:0c:00:05:30:00:47:9e
    Peer port WWN is 20:5d:00:05:30:00:51:1e
    Admin port mode is auto, trunk mode is on
    Port mode is TE
    Port vsan is 1
    Speed is 2 Gbps
    Transmit B2B Credit is 12
    Receive B2B Credit is 255
    Receive data field Size is 2112
    Beacon is turned off
    Trunk vsans (admin allowed and active) (1-5,13,20,777)
    Trunk vsans (up)                       (1,13)
    Trunk vsans (isolated)                 (2-5,20,777)
    Trunk vsans (initializing)             ()
    5 minutes input rate 384 bits/sec, 48 bytes/sec, 0 frames/sec
    5 minutes output rate 4458296 bits/sec, 557287 bytes/sec, 827 frames/sec
      19865 frames input, 2220112 bytes
        0 discards, 0 errors
        0 CRC,  0 unknown class
        0 too long, 0 too short
      1468709 frames output, 971064244 bytes
        0 discards, 0 errors
      0 input OLS, 2 LRR, 0 NOS, 0 loop inits
      2 output OLS, 2 LRR, 0 NOS, 2 loop inits

MDS9216# show interface fc-tunnel 13

fc-tunnel 13 is up
Dest   IP Addr: 10.0.0.2   Tunnel ID: 13
Source IP Addr: 10.0.0.1   LSP ID: 1
Explicit Path Name:
Outgoing interface: fc1/12
Outgoing Label(s) to Insert: 10005:0:1:ff'h
Record Routes:
10.0.0.2

MDS9216# show interface vsan 13

vsan13 is up, line protocol is up
    WWPN is 10:00:00:05:30:00:47:9f, FCID is 0x660201
    Internet address is 10.0.0.1/24
    MTU 1500 bytes, BW 1000000 Kbit
    2207 packets input, 170332 bytes, 0 errors, 0 multicast
    14952 packets output, 2225444 bytes, 0 errors, 0 dropped

MDS9216# show span session 1

Session 1 (active)
   Destination is fc-tunnel 13
   No session filters configured
   Ingress (rx) sources are
     fc1/16,
   Egress (tx) sources are
     fc1/16,

MDS9216# show fc-tunnel internal states

number of sessions : 1
Sess: 10.0.0.2 Tunnel-ID 13 Ext-Tunnel-ID 10.0.0.1

MDS9216# show fc-tunnel internal data

vsan interfaces:
  vsan 13: 10.0.0.1/255.255.255.0  [2]
  vsan 2: 15.0.0.4/255.255.255.0  [2]
next hop switch information:
  10.0.0.2 {vsan (13), 0x6b0001/8}: [4] fc1/12
layer 2 interfaces:
  fc1/12: Trunking, Up

MDS 9509 Configuration

RTP-9509-1# show run

Building Configuration ...
vsan database
vsan 13

 interface vsan13
ip address 10.0.0.2 255.255.255.0
no shutdown

vsan 13 interface fc2/16

boot system bootflash:/m9500-sf1ek9-mzg.1.2.0.77.bin sup-1
boot kickstart bootflash:/m9500-sf1ek9-kickstart-mzg.1.2.0.77.bin sup-1
boot system bootflash:/m9500-sf1ek9-mzg.1.2.0.77.bin sup-2
boot kickstart bootflash:/m9500-sf1ek9-kickstart-mzg.1.2.0.77.bin sup-2

fc-tunnel enable
fc-tunnel tunnel-id-map 13 interface fc2/6

ip routing

switchname RTP-9509-1

 interface fc2/6
switchport mode SD
switchport speed 1000
no shutdown

 interface fc2/29
switchport mode E
no shutdown

 interface mgmt0
ip address 172.18.172.57 255.255.255.0

MDS 9509 Displays

RTP-9509-1# show interface fc 2/29

fc2/29 is trunking
    Hardware is Fibre Channel
    Port WWN is 20:5d:00:05:30:00:51:1e
    Peer port WWN is 20:0c:00:05:30:00:47:9e
    Admin port mode is E, trunk mode is on
    Port mode is TE
    Port vsan is 501
    Speed is 2 Gbps
    Transmit B2B Credit is 255
    Receive B2B Credit is 12
    Receive data field Size is 2112
    Beacon is turned off
    Trunk vsans (admin allowed and active) (1,13,86,100,501)
    Trunk vsans (up)                       (1,13)
    Trunk vsans (isolated)                 (86,100,501)
    Trunk vsans (initializing)             ()
    5 minutes input rate 4497752 bits/sec, 562219 bytes/sec, 835 frames/sec
    5 minutes output rate 344 bits/sec, 43 bytes/sec, 0 frames/sec
      1934604 frames input, 1285716656 bytes
        0 discards, 0 errors
        0 CRC,  0 unknown class
        0 too long, 0 too short
      16903 frames output, 932076 bytes
        0 discards, 0 errors
      1 input OLS, 1 LRR, 2 NOS, 0 loop inits
      3 output OLS, 1 LRR, 2 NOS, 0 loop inits

RTP-9509-1# show interface fc 2/6

fc2/6 is up
    Hardware is Fibre Channel
    Port WWN is 20:46:00:05:30:00:51:1e
    Admin port mode is SD
    Port mode is SD
    Port vsan is 1
    Speed is 1 Gbps
    Beacon is turned off
    5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
    5 minutes output rate 4421448 bits/sec, 552681 bytes/sec, 835 frames/sec
      0 frames input, 0 bytes
        0 discards, 0 errors
        0 CRC,  0 unknown class
        0 too long, 0 too short
      1912319 frames output, 1263982444 bytes
        0 discards, 0 errors
      0 input OLS, 0 LRR, 0 NOS, 0 loop inits
      0 output OLS, 0 LRR, 0 NOS, 0 loop inits

RTP-9509-1# show interface fc-tunnel 13
                           ^
% invalid interface range detected at '^' marker.

!--- This is because the tunnel is not defined on the 9509.

RTP-9509-1# show interface vsan 13

vsan13 is up, line protocol is up
    WWPN is 10:00:00:05:30:00:51:23, FCID is 0x6b0001
    Internet address is 10.0.0.2/24
    MTU 1500 bytes, BW 1000000 Kbit
    15071 packets input, 2243728 bytes, 0 errors, 1 multicast
    2342 packets output, 185864 bytes, 0 errors, 0 dropped

RTP-9509-1# show fc-tunnel tunnel-id-map

tunnel id  egress interface
     13             fc2/6
     14

RTP-9509-1# show fc-tunnel internal states

number of sessions : 1
Sess: 10.0.0.2 Tunnel-ID 13 Ext-Tunnel-ID 10.0.0.1

RTP-9509-1# show fc-tunnel internal data

vsan interfaces:
  vsan 13: 10.0.0.2/255.255.255.0  [2]
next hop switch information:
layer 2 interfaces:
  fc2/6: Non-Trunking, Up

Notes for Port Analyzer Adapter Devices

The Ethernet port is copper, and it has auto-detects speeds of 1 Gbps or 100 Mbps. Ethereal 0.9(9) or later and WinPcap must be installed on the PC.

The FC port requires an SFP and an LC-to-LC cable for attachment to the MDS.

These are switch settings on the PAA:

  • Switch positions are numbered 1, 2, 3, and 4 from left to right.

  • In the next list, a 1 indicates that the dip switch is ON or UP. A 0 indicates the dip switch is DOWN or OFF.

    0001 1G  NTM
    1001 1G  ETM
    0101 1G  STM
    0011 1G  DTM
    
    0000 2G  NTM
    1000 2G  ETM
    0100 2G  STM
    0010 2G  DTM
    
    1111 1G  MNM
    
    !--- Used for diagnostics only.
    
    
  • Switch 4 dictates the speed (on = 1G, off = 2G). Switches 1, 2, and 3 dictate truncate mode. Any changes require a power cycle.

These are the modes:

  • No Truncate Mode (NTM)—FC frames are passed without any modifications.

  • Ethernet Truncate Mode (ETM)—Reduces the payload size from 528 lines to 368 lines, to truncates FC frame to a maximum of 1496 bytes.

  • Shallow Truncate Mode (STM)—Reduces the payload size from 528 lines to 58 lines, to truncates FC frame to a maximum of 256 bytes.

  • Deep Truncate Mode (DTM)—Reduces the payload size from 528 lines to 10 lines, to truncates FC frame to a maximum of 64 bytes.

Verify

There is currently no verification procedure available for this configuration.

Troubleshoot

There is currently no specific troubleshooting information available for this configuration.

Related Information

Updated: Jul 23, 2008
Document ID: 46220