Table Of Contents
Troubleshooting Switch Fabric Level Issues
Troubleshooting Name Server Issues
Overview
Nx Port Registration Problems
Troubleshooting FSPF Issues
Overview
Loss of Two-way Communication
Wrong Hello Interval on an ISL Interface
Resolving the Wrong Hello Interval Problem
Wrong Dead Interval on an ISL Interface
Resolving a Wrong Dead Interval Problem
Region Mismatch on Switch
Resolving a Region Mismatch Problem
FSPF Issues in a Single-VSAN Environment
FSPF Issues in a Multi-VSAN Environment
Troubleshooting Zoning Issues
Mismatched Active Zonesets Within the Same VSAN
Importing or Exporting a Zoneset Between Switches
Deactivating a Zoneset and Restarting the Zone Merge Process
Misconfigured Zones Within an Active Zoneset in the Same VSAN
Troubleshooting Switch Fabric Level Issues
This chapter describes switch fabric-level troubleshooting procedures and includes the following sections:
•Troubleshooting Name Server Issues
•Troubleshooting FSPF Issues
•Troubleshooting Zoning Issues
Troubleshooting Name Server Issues
This section describes how to identify and resolve problems with the Cisco MDS 9000 Family name server. It includes the following sections:
•Overview
•Nx Port Registration Problems
Overview
The name server provides a way for N ports and NL ports to register and discover FibreChannel attributes. Registrations may happen explicitly, as a consequence of a request of the Nx port, or implicitly by the switch at FLOGI time. Once registered, the attributes are made available to other Nx ports or other switches. A separate name server database is maintained for each VSAN. The name server uses the Fibre Channel Common Transport protocol (FC-CT) to communicate with Nx ports and represents itself as an N port at the well-known FCID 0xFFFFFC.
One instance of the name server process runs on each Cisco MDS 9000 Family switch. In a multi-switch fabric configuration, instances running on different switches share information and create a distributed database of Nx port attributes. The name server defines a set of attribute objects with the following operations on those objects:
•Register Object
•Deregister Object
•Get Object
To troubleshoot name server problems, check the status of these three operations and verify the correct distribution of attributes among name server instances within the fabric. These attributes include the following:
•Port Type
•Port Identifier
•Port Name
•Node Name
•Port Symbolic Name
•Node Symbolic Name
•Class of Service
•FC-4 Type(s)
•Port IP Addresses
•Node IP Addresses
•Hard Address
•FC-4 Descriptor
•FC-4 Features
•Initial Process Associator
Nx Port Registration Problems
To troubleshoot port registration, follow these steps:
Step 1 From the CLI exec mode, enter the following command:
This ensures that the fibre channel (FC) interface connected to the device in question is up and free of any errors.
The system output might look like this:
switch# show int fc3/14
fc3/14 is up
Hardware is Fibre Channel
Port WWN is 20:8e:00:05:30:00:86:9e
Admin port mode is FX
Port mode is F, FCID is 0x780200 /* Operational State of the Port */
Port vsan is 99 /* This is the vsan */
Speed is 2 Gbps
Receive B2B Credit is 16
Receive data field size is 2112
Beacon is turned off
5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
1700 frames input, 106008 bytes, 0 discards
0 CRC, 0 unknown class
0 too long, 0 too short
2904 frames output, 364744 bytes, 0 discards
0 input OLS, 0 LRR, 0 NOS, 0 loop inits
1 output OLS, 1 LRR, 0 NOS, 0 loop inits
If the interface is not working correctly, check the cabling and the host or storage device interface for faults. If the interface is working correctly, proceed to the next step.
Step 2 Verify that the device in question appears in the FLOGI database. To do this, enter the following command:
show flogi database vsan vsanid
The system output might look like this:
switch# show flogi database vsan 99
---------------------------------------------------------------------------
INTERFACE VSAN FCID PORT NAME NODE NAME
---------------------------------------------------------------------------
fc3/14 99 0x780200 21:00:00:e0:8b:07:a4:36 20:00:00:e0:8b:07:a4:36
If the device in question appears in this output, skip to Step 8. If the device does not appear in the output, go to the next step.
Step 3 From interface mode, shut down the FC interface connected to the device in question.
Step 4 Enter the following command on the FC interface:
By shutting down the interface and bringing it back up, you can determine what happens when the connected device tries to log in to the interface.
Step 5 Enter the following command to view the events that occurred on the interface after you enabled it again:
switch# show flogi internal event-history interface fc3/14
The system output looks like this:
>>>>FSM: <[99]21:00:00:e0:8b:07:a4:36> has 9 logged transitions<<<<<
/* This is the [VSAN] followed by the pwwn of the N/NL port */
1) FSM:<[99]21:00:00:e0:8b:07:a4:36> Transition at 321686 usecs after Sun Feb 1
04:18:15 1980
Previous state: [FLOGI_ST_FLOGI_RECEIVED]
Triggered event: [FLOGI_EV_VALID_FLOGI]
Next state: [FLOGI_ST_GET_FCID]
/* The hba has sent an FLOGI to the switch */
2) FSM:<[99]21:00:00:e0:8b:07:a4:36> Transition at 322974 usecs after Sun Feb 1
04:18:15 1980
Previous state: [FLOGI_ST_GET_FCID]
Triggered event: [FLOGI_EV_VALID_FCID]
Next state: [FLOGI_ST_PERFORM_CONFIG]
/* Port Manager Obtains a valid FC_ID from the Domain Mgr */
3) FSM:<[99]21:00:00:e0:8b:07:a4:36> Transition at 323731 usecs after Sun Feb 1
04:18:15 1980
Previous state: [FLOGI_ST_PERFORM_CONFIG]
Triggered event: [FLOGI_EV_CONFIG_DONE_PENDING]
Next state: [FLOGI_ST_PERFORM_CONFIG]
/* ACLs are programmed and FIB {VSAN, FC_ID, portindex} is set */
4) FSM:<[99]21:00:00:e0:8b:07:a4:36> Transition at 323948 usecs after Sun Feb 1
04:18:15 1980
Previous state: [FLOGI_ST_PERFORM_CONFIG]
Triggered event: [FLOGI_EV_LCP_RESPONSE]
Next state: [FLOGI_ST_PERFORM_CONFIG]
/* LineCard responds that it is done */
5) FSM:<[99]21:00:00:e0:8b:07:a4:36> Transition at 325962 usecs after Sun Feb 1
04:18:15 1980
Previous state: [FLOGI_ST_PERFORM_CONFIG]
Triggered event: [FLOGI_EV_NAME_SERVER_REG_RESPONSE]
Next state: [FLOGI_ST_PERFORM_CONFIG]
/* Program the NameServer with wwn and FCID */
6) FSM:<[99]21:00:00:e0:8b:07:a4:36> Transition at 330381 usecs after Sun Feb 1
04:18:15 1980
Previous state: [FLOGI_ST_PERFORM_CONFIG]
Triggered event: [FLOGI_EV_ZS_CFG_RESPONSE]
Next state: [FLOGI_ST_PERFORM_CONFIG]
/* Response from ZoneServer */
7) FSM:<[99]21:00:00:e0:8b:07:a4:36> Transition at 331187 usecs after Sun Feb 1
04:18:15 1980
Previous state: [FLOGI_ST_PERFORM_CONFIG]
Triggered event: [FLOGI_EV_RIB_RESPOSE]
Next state: [FLOGI_ST_PERFORM_CONFIG]
8) FSM:<[99]21:00:00:e0:8b:07:a4:36> Transition at 331768 usecs after Sun Feb 1
04:18:15 1980
Previous state: [FLOGI_ST_PERFORM_CONFIG]
Triggered event: [FLOGI_EV_ACL_CFG_RESPONSE]
Next state: [FLOGI_ST_PERFORM_CONFIG]
9) FSM:<[99]21:00:00:e0:8b:07:a4:36> Transition at 331772 usecs after Sun Feb 1
04:18:15 1980
Previous state: [FLOGI_ST_PERFORM_CONFIG]
Triggered event: [FLOGI_EV_CONFIG_DONE_COMPLETE]
Next state: [FLOGI_ST_FLOGI_DONE]
Curr state: [FLOGI_ST_FLOGI_DONE]
/* Flogi was successful */
The comments that follow each section of output explain the meaning of the output.
If the device logs in successfully, proceed to the next step. Otherwise, you may have a problem with the device or its associated software.
Step 6 From interface mode, shut down the FC interface and issue a no shutdown after turning on the debug described in the following steps.
Step 7 To watch the FLOGI process take place, enter the following command:
switch# debug fcns events register vsan 99
This command enables debug mode for nameserver registration. The system output looks like this:
switch# conf t
Enter configuration commands, one per line. End with CNTL/Z.
switch(config)# int fc3/14
switch(config-if)# no shut /* enable the port */
switch(config-if)# Feb 17 04:42:54 fcns: vsan 99: Created entry for port-id 27800
Feb 17 04:42:54 fcns: vsan 99: Got Entry for port-id 27800
Feb 17 04:42:54 fcns: vsan 99: Registered port-name 36a4078be0000021 for port-id 780200
Feb 17 04:42:54 fcns: vsan 99: Registered node-name 36a4078be0000020 for port-id 780200
/* The wwpn and FCID for the port, note that the bytes in the world wide name are reversed
*/
Feb 17 04:42:54 fcns: vsan 99: Registered cos 8 for port-id 780200
/* Class of Service */
Feb 17 04:42:54 fcns: vsan 99: Registered port-type 1 for port-id 780200
Feb 17 04:42:54 fcns: vsan 99: Reading configuration for entry with port-name
36a4078be0000021, node-name 36a4078be0000020
Feb 17 04:42:54 fcns: vsan 99: No configuration present for this portname
Feb 17 04:42:54 fcns: vsan 99: No configuration present for this nodename
/* Port is now registered in nameserver, will send out RSCN to it */
Feb 17 04:42:54 fcns: vsan 99: Trying to send RSCN; affected port 780200
Feb 17 04:42:54 fcns: vsan 99: rscn timer started for port 780200
Feb 17 04:42:54 fcns: vsan 99: Saving new entry into pss
Feb 17 04:42:54 fcns: vsan 99: Sending sync message to the standby
Feb 17 04:42:54 fcns: vsan 99: sending accept response to 780200
/* RSCN was received by N/NL port */
Feb 17 04:42:54 fcns: vsan 99: sending accept response to fffc61
/* Other switch in fabric is notified */
Feb 17 04:42:55 fcns: vsan 99: rscn timer expired for port 780200
Feb 17 04:42:55 fcns: vsan 99: Saving modified entry into pss
Feb 17 04:42:55 fcns: vsan 99: Sending sync message to the standby
Feb 17 04:42:55 fcns: vsan 99: Registered fc4-types for port-id 780200
Feb 17 04:42:55 fcns: vsan 99: Registered fc4-features for fc4_type 8 for port-id 780200
/* FC4 Type, type 8 FCP has been registered */
Additional lines like these will be listed if additional nameserver objects are registered
Step 8 From the CLI exec mode, enable FC name server (FCNS) debugging by entering the following command:
debug fcns events register vsan x
If you are managing the switch over a telnet connection, enable terminal monitoring by entering the terminal monitor command from the CLI exec mode.
The system output looks like this:
switch# show fcns database detail v 99
------------------------
VSAN:99 FCID:0x780200
------------------------
port-wwn (vendor) :21:00:00:e0:8b:07:a4:36 (QLogic) /* Port world wide name */
node-wwn :20:00:00:e0:8b:07:a4:36
class :3 /* Fibrechannel class of service */
node-ip-addr :0.0.0.0 /* IP Address */
ipa :ff ff ff ff ff ff ff ff
fc4-types:fc4_features:scsi-fcp:init /* Registered FC4 Types: example SCSI and
initiator */
symbolic-port-name :
symbolic-node-name :
port-type :N /* Fibrechannel port type (F,FL) */
port-ip-addr :0.0.0.0
fabric-port-wwn :20:8e:00:05:30:00:86:9e /* wwn of the switch port */
hard-addr :0x000000
Other attribute objects of the Nx port are registered one per register operation after the FLOGI process is complete. The Nx port performs PLOGI to the well-known WWN of the Name Server, 0xFFFFFC. The FC_CT Common Transport protocol uses Request and Accept messages to conduct transactions. To verify that additional attributes are correctly registered and recorded in the database, you can use the SAN-OS debug facility.
Note The command show fcns database detail vsan X displays a detailed list of all devices registered in the fabric.
Troubleshooting FSPF Issues
This section describes how to identify and resolve FSFP problems and includes the following sections;
•Overview
•Loss of Two-way Communication
•Wrong Hello Interval on an ISL Interface
•Resolving the Wrong Hello Interval Problem
•Wrong Dead Interval on an ISL Interface
•Resolving a Wrong Dead Interval Problem
•Region Mismatch on Switch
•Resolving a Region Mismatch Problem
•FSPF Issues in a Single-VSAN Environment
•FSPF Issues in a Multi-VSAN Environment
Overview
To see all the correct FSPF information, as shown in the previous examples, the switches must be configured correctly. If FSPF is misconfigured, then the switches will not reach the "two-way" state. This can happen when:
•The switch fails to receive Hello in expected time (dead interval)
•The switch receives Hello from neighbor that does not contain the correct domain ID in the Recipient Domain ID field.
Note This would not be a configuration issue.
•The switch receives Hello from neighbor with 0xFFFFFFFF in the Recipient Domain ID field.
Note Hellos are sent with 0xFFFFFFFF in neighbor field until switch learns its neighbor's domain ID.
•The switch receives Hello with incorrect Hello and dead intervals.
Loss of Two-way Communication
The following events occur when two-way communication is lost:
1. The Port enters Init state and removes its neighbor's domain ID from the Recipient Domain ID field and inserts 0xFFFFFFFF.
2. FSPF Removes the ISL (inter switch link) from the topology database.
3. New LSRs (Link State Records) are flooded to adjacent switches to notify them that the FSPF database has changed.
Wrong Hello Interval on an ISL Interface
To identify a mismatch in the Hello interval on the two sides of an ISL interface, follow these steps:
Step 1 To turn on debugging, enter the following command:
The system output looks like this:
Jan 5 00:28:14 fspf: Wrong hello interval for packet on interface 100f000 in VSAN 1
Jan 5 00:28:14 fspf: Error in processing hello packet , error code = 4
Step 2 To show FSPF information, enter the following command:
switch1# show fspf internal route v 1
The system output looks like this:
switch1# show fspf internal route v 1
---------------------------
VSAN Number Dest Domain Route Cost Next hops
----------------------------------------------------------------------
1 0xEF(239) 1000 fc1/1 -----1
1 0x01(1) 3000 fc1/1 -----2
1. Indicates that there is no second path to Domain 238, through Domain 1 switch2.
2. Indicates that there is no direct path to Domain 1 switch2; traffic must travel through 3 ISLs.
Step 3 To view the currently configured FSPF parameters on the ISL, enter the following command:
switch1# show fspf v 1 interfac fc1/16
The system output looks like this:
switch1# show fspf v 1 interfac fc1/16
FSPF interface fc1/16 in VSAN 1
FSPF routing administrative state is active
Timer intervals configured, Hello 5 s, Dead 80 s, Retransmit 5 s -----1
FSPF State is INIT -----2
Number of packets received : LSU 0 LSA 0 Hello 2 Error packets 1
Number of packets transmitted : LSU 0 LSA 0 Hello 4 Retransmitted LSU 0
Number of times inactivity timer expired for the interface = 0
1. Shows that the hello timer is not set to the default, so you should check the neighbor configuration to make sure it matches.
2. Shows that FSPF is not in FULL state, indicating a problem.
Step 4 To determine the value of the Hello timer on the adjacent switch, enter the following command:
switch2# show fspf v 1 interfac fc1/16
The system output looks like this:
switch2# show fspf v 1 interfac fc1/16
FSPF interface fc1/16 in VSAN 1
FSPF routing administrative state is active
Timer intervals configured, Hello 20 s, Dead 80 s, Retransmit 5 s -----1
FSPF State is INIT -----2
Number of packets received : LSU 0 LSA 0 Hello 2 Error packets 1
Number of packets transmitted : LSU 0 LSA 0 Hello 4 Retransmitted LSU 0
Number of times inactivity timer expired for the interface = 0
1. Shows that the neighbor FSPF Hello interval is set to the default (20 seconds).
2. Indicates that FSPF is not in FULL state, indicating a problem.
Step 5 An alternative to check the hello interval setting is the show run command.
fspf hello-interval 5 vsan 1
The system output looks like this.
This output indicates that the neighbor FSPF hello is set to the default. The default setting does not display in the output from the show run command.
Resolving the Wrong Hello Interval Problem
The Hello interval must match on both sides of an ISL, so you should change either side to match the other. Use the default Hello interval unless you are sure you need change it. To change the default Hello interval, enter the following commands:
ML-88(config)# interface fc 1/16
ML-88(config-if)# fspf hello-interval XX vsan 1
Wrong Dead Interval on an ISL Interface
To identify a mismatch in dead intervals on the two sides of an ISL interface, follow these steps:
Step 1 To enable the debug output for identifying this condition, enter the following command:
The system output looks like this:
Jan 5 00:28:14 fspf: Wrong dead interval for packet on interface 100f000 in VSAN 1
Jan 5 00:28:14 fspf: Error in processing hello packet , error code = 4
Step 2 To display the currently configured FSPF parameters on the interface, enter the following command:
switch1# show fspf v 1 interfac fc1/16
You should run this command on the local interface and on the switch at the other end of the ISL on which the problem is occurring.
The system output looks like this:
switch1# show fspf v 1 interfac fc1/16
FSPF interface fc1/16 in VSAN 1
FSPF routing administrative state is active
Timer intervals configured, Hello 20 s, Dead 95 s, Retransmit 5 s -----1
FSPF State is INIT -----2
Number of packets received : LSU 0 LSA 0 Hello 2 Error packets 1
Number of packets transmitted : LSU 0 LSA 0 Hello 4 Retransmitted LSU 0
Number of times inactivity timer expired for the interface = 0
1. Indicates that the dead timer is not set to the default, so you should check the neighbor configuration.
2. Indicates that FSPF is not in full state, which indicates a problem.
Resolving a Wrong Dead Interval Problem
The dead interval must match on both sides of the ISL, so to resolve this problem, change either side to match the other. Note that under most conditions, the default dead interval value should be used unless there is a demonstrated need to change it.
ML-88(config)# interface fc 1/16
ML-88(config-if)# fspf dead-interval XX vsan 1
Region Mismatch on Switch
To identify a region mismatch problem on a switch, follow these steps:
Step 1 To display the currently configured region in a VSAN, enter the following command:
switch# show fspf vsan 99
The system output looks like this:
FSPF routing for VSAN 99
FSPF routing administration status is enabled
FSPF routing operational status is UP
It is an intra-domain router
Autonomous region is 0 /* This is the region */
SPF hold time is 0 msec
MinLsArrival = 1000 msec , MinLsInterval = 5000 msec
Local Domain is 0x78(120)
Number of LSRs = 2, Total Checksum = 0x000133de
Step 2 To turn on debugging, enter the following command:
The system output looks like this:
Jan 5 00:39:31 fspf: FC2 packet received for non existent region 0 in VSAN 1 -----1
Jan 5 00:39:33 fspf: FC2 packet received for non existent region 0 in VSAN 1
Jan 5 00:39:45 fspf: Interface fc1/1 in VSAN 1 : Event INACTIVITY , State change INIT ->
INIT
Jan 5 00:39:45 fspf: Interface fc1/2 in VSAN 1 : Event INACTIVITY , State change INIT ->
INIT -----2
1. Indicates that the neighbor switch advertising region is 0.
2. Indicates that FSPF is in INIT state for each ISL.
Step 3 To check region settings, issue a show run command.
Resolving a Region Mismatch Problem
The region must match on all switches in the VSAN. To correct a region mismatch, enter the following commands:
switch1(config)# fspf config vsan 1
switch1(config-(fspf-config))# region 0
FSPF Issues in a Single-VSAN Environment
Figure 4-1 Single VSAN Topology
For the purpose of this example, assume that all interfaces are located in VSAN 1.
Step 1 Verify that each path is in the FSPF database by entering the following command at the exec prompt:
Switch switch1# show fspf database
The system output looks like this:
Switch switch1# show fspf database
FSPF Link State Database for VSAN 1 Domain 1 -----1
Advertising domain ID = 1 -----2
LSR Incarnation number = 0x80000098 -----4
NbrDomainId IfIndex NbrIfIndex Link Type Cost
------------------------------------------------------------------------------------------
------
237 0x00010002 0x00010001 1 1000 -----5
238 0x00010003 0x00010002 1 1000 -----6
The following is the beginning of another switch's LSR (Link State Record).
FSPF Link State Database for VSAN 1 Domain 237
Advertising domain ID = 237 -----7
LSR Incarnation number = 0x8000000c
NbrDomainId IfIndex NbrIfIndex Link Type Cost
------------------------------------------------------------------------------------------
-------
239 0x00010000 0x00010003 1 1000 -----8
1 0x00010001 0x00010002 1 1000 -----9
The following is the beginning of another switch's LSR (Link State Record)
FSPF Link State Database for VSAN 1 Domain 238
Advertising domain ID = 238
LSR Incarnation number = 0x80000013
NbrDomainId IfIndex NbrIfIndex Link Type Cost
------------------------------------------------------------------------------------------
-------
239 0x00010003 0x00010001 1 1000
1 0x00010002 0x00010003 1 1000
The following is the beginning of another switch's LSR (Link State Record)
FSPF Link State Database for VSAN 1 Domain 239
Advertising domain ID = 239
LSR Incarnation number = 0x80000086
NbrDomainId IfIndex NbrIfIndex Link Type Cost
------------------------------------------------------------------------------------------
----
237 0x00010003 0x00010000 1 1000
238 0x00010001 0x00010003 1 1000
1. Provides the Domain 1 view of the fabric topology.
2. Indicates that Domain 1 is owner of the LSR (Link State Record).
3. This is a 16-bit counter starting at 0x0000, incremented by one for each switch during flooding and by one for each second held in database. This field is used as a tie-breaker if Incarnation numbers are the same.
4. This is a 32-bit value between 0x80000001 and 0x7FFFFFFF. which is incremented by one each time the originating switch transmits an LSR. This is used first before LSR Age.
5. Indicates the path to Domain 237, Switch switch1.
6. Indicates the path to Domain 238, Switch switch5.
7. Indicates that switch1, Domain ID 237 is the owner.
8. Indicates the path to Domain 239, Switch switch3.
9. Indicates the path to Domain 1, Switch switch2
Step 2 Verify that the FSPF parameters are correct for each interface by entering the following command at the exec prompt:
switch1# show fspf vsan 1 interface fc1/2
View the output from this command to verify that the interface is in FSPF "active state." The system output looks like this:
switch1# show fspf vsan 1 interface fc1/2
FSPF interface fc1/2 in VSAN 1
FSPF routing administrative state is active -----1
Interface cost is 1000 -----2
Timer intervals configured, Hello 20 s, Dead 80 s, Retransmit 5 s -----3
FSPF State is FULL -----4
Neighbor Domain Id is 1, Neighbor Interface index is 0x00010002 -----5
Number of packets received : LSU 46 LSA 24 Hello 103 Error packets 0
Number of packets transmitted : LSU 24 LSA 45 Hello 104 Retransmitted LSU 0
Number of times inactivity timer expired for the interface = 0
This displays the number of packets; Hellos should be received every 20 seconds.
1. Indicates that FSPF is active and is not disabled on this interface.
2. Indicates the cost of the path out this interface.
3. Identifies the configured FSPF timers for this interface, which must match on both sides.
4. Indicates Full State or Adjacent. Sent and received all database exchanges and required Acks. Port is now ready to route frames.
5. Provides FSPF neighbor information.
Step 3 Verify that all FC routes are available by entering the following command:
switch1# show fspf internal route v 1
The system output looks like this:
switch1# show fspf internal route v 1
---------------------------
VSAN Number Dest Domain Route Cost Next hops
----------------------------------------------------------------------------------------
This shows the total cost of all links.
The next hop to (238) has two interfaces. This indicates that both paths will be used during load sharing. Up to sixteen paths can be used by FSPF with a Cisco MDS 9000 Family switch.
FSPF Issues in a Multi-VSAN Environment
With the implementation of VSANs used with Cisco MDS 9000 Family switches, a separate instance of FSPF runs within each VSAN, and each instance is independent of the others. For this reason, FSPF issues affecting one VSAN have no effect on FSPF running in other VSANs.
Note For all FSPF configuration statements and diagnostic commands, if the vsan keyword is not specified, VSAN 1 is used by default. When making configuration changes or issuing diagnostic commands in a multi-VSAN environment, be sure to explicitly specify the target VSAN by including the vsan keyword in the statement or command.
Troubleshooting Zoning Issues
This section describes how to identify and resolve zoning issues that may arise in a multiswitch fabric. It includes the following topics:
•Mismatched Active Zonesets Within the Same VSAN
•Importing or Exporting a Zoneset Between Switches
•Deactivating a Zoneset and Restarting the Zone Merge Process
•Misconfigured Zones Within an Active Zoneset in the Same VSAN
If after you verify the proper operation of the fibre channel name server and FSPF problems remain discovering remote switches and their attached resources, the fabric may have zone configuration problems. Examples of zone configuration problems are mismatched active zonesets and misconfigured zones within the active zoneset.
Mismatched Active Zonesets Within the Same VSAN
When merging switch fabrics, you must ensure that the zones in both active zonesets have unique names, or that any zones with the same name have exactly the same members. If either of these conditions is violated the E port connecting the two fabrics will appear in an isolated state.
Figure 4-2 Topology for Zone Merge Failure Example
In this example, two switches have the same zoneset name, and the same zone names, but different zone members. As a result, the VSAN is isolated on the TE port that connects to two switches.
This issue can be resolved by doing one of the following:
1. Modify the zone members on both zonesets to match and elimiate conflict
2. Deactivate the zoneset on one of the switches and restart the zone merge process
3. Explicitly import or export a zoneset between the switches to synchronize them.
To identify this problem, follow these steps:
Step 1 Enter the following command to display the active zoneset configuration of the first switch:
Switch1# show zoneset active v 99
The system output looks like this:
Switch1# show zoneset active v 99
zoneset name ZoneSet1 vsan 99
* fcid 0x7800e2 [pwwn 22:00:00:20:37:04:ea:2b]
* fcid 0x7800d9 [pwwn 22:00:00:20:37:04:f8:a1]
Step 2 Enter the following command to display the active zoneset configuration of the second switch:
Switch2# show zoneset active v 99
The system output looks like this:
Switch2# show zoneset active v 99
zoneset name ZoneSet1 vsan 99
pwwn 22:00:00:20:37:04:f8:a1
pwwn 22:00:00:20:37:0e:65:44
Even though the zones have the same name, their respective members are different.
Step 3 Enter the following command to view information about the TE port:
This command shows all the information about the interface. The system output looks like this:
Hardware is Fibre Channel
Port WWN is 20:08:00:05:30:00:5f:1e
Peer port WWN is 20:05:00:05:30:00:86:9e
Admin port mode is E, trunk mode is auto
Receive B2B Credit is 255
Receive data field size is 2112
Trunk vsans (admin allowed and active) (1,99)
Trunk vsans (isolated) (99)
Trunk vsans (initializing) ()
5 minutes input rate 120 bits/sec, 15 bytes/sec, 0 frames/sec
5 minutes output rate 88 bits/sec, 11 bytes/sec, 0 frames/sec
10845 frames input, 620268 bytes, 0 discards
10842 frames output, 487544 bytes, 0 discards
3 input OLS, 4 LRR, 3 NOS, 0 loop inits
18 output OLS, 2 LRR, 14 NOS, 0 loop inits
From this output, you can see that VSAN 99 is isolated.
Step 4 Enter the following command to get information about why the interface is isolated:
switch2# show int fc1/8 trunk vsan
The system output looks like this:
switch2# show int fc1/8 trunk vsan
Vsan 1 is up, FCID is 0x650000
From this output, you can see the VSAN is isolated due to a zone merge failure:
Step 5 To resolve the isolation problem, do one of the following:
•Change the membership of one of the zones to match the other of the same name.
•Discard one of the zonesets completely by either deactivating it, or by overwriting the active zoneset on one switch using the import or export commands. This method is destructive to one of the active zonesets.
For instructions about changing the membership of a zone, refer to the Cisco MDS 9000 Family Configuration Guide.
Step 6 The following example shows the use of the export command:
In this example, the zoneset configuration is correct on switch1, so you would want to discard the zoneset configuration on switch2. You get the same result by entering the zone import command from switch2.
Step 7 Import the active zoneset for VSAN 99 on interface fc1/8 by entering the following command:
Switch1# zone merge interface fc1/8 import vsan 99
Zoneset export initiated. check zone status
Step 8 Verify that VSAN 99 is no longer isolated by entering the following command:
Switch1# show int fc1/5 trunk v 99
Vsan 99 is up, FCID is 0x780102
If a VSAN does not have an active zoneset, it automatically takes the active zoneset of the other merging switch. So another way to solve this problem is by deactivating the active zoneset on switch2 by entering the following command:
no zoneset activate name ZoneSet1 v 99
This removes the active zoneset on switch2, which will then automatically take the active zoneset from switch1.
Importing or Exporting a Zoneset Between Switches
To import or export a zoneset between switches, follow these steps:
Step 1 Enter the following command <<explain why?>>
The system output looks like this:
Nov 19 09:28:45 switch4 %LOG_PORT_CHANNEL-5-FOP_CHANGED: port-channel 1: first operational
port changed from none to fc1/14
Nov 19 09:28:55 switch4 %LOG_ZONE-2-ZS_MERGE_FAILED: Zone merge failure, Isolating port
port-channel 1 (VSAN 1)
Step 2 Enter the following command <<explain why?>>
switch4# zone merge int port-channel 1 import vsan 1
The system output looks like this:
Zoneset Import initiated. check zone status
switch4# Nov 19 11:43:02 switch4 %LOG_PORT-5-IF_UP: Interface port-channel 1 is up in mode
TE
Nov 19 11:43:02 switch4 %LOG_PORT-5-IF_TRUNK_UP: Interface fc1/14, vsan 1 is up
Nov 19 11:43:02 switch4 %LOG_PORT-5-IF_TRUNK_UP: Interface fc1/15, vsan 1 is up
Nov 19 11:43:02 switch4 %LOG_PORT-5-IF_TRUNK_UP: Interface fc1/16, vsan 1 is up
switch4# show zoneset activ
The system output looks like this:
zone name $default_zone$ vsan 1
When you explicitly import a zoneset for VSAN 1 from switch3 on switch4 over the isolated ISL, the active zoneset is copied from switch3 to switch4. The zone databases are now synchronized, and the VSAN is no longer isolated.
Deactivating a Zoneset and Restarting the Zone Merge Process
To deactivate a zoneset and restart the zone merge process, follow these steps:
Step 1 Remove the zoneset configuration from the switch, as in the following example:
switch4(config)# no zoneset activate name excal2 vsan 1
The system output looks like this:
Zoneset Dectivation initiated. check zone status
Step 2 To confirm that the zoneset has been removed, enter the following command:
switch4# show zoneset active
Step 3 Shut down the connection to the zone to be merged.
Enter configuration commands, one per line. End with CNTL/Z.
switch4(config)# int port-channel 1
The system output looks like this:
Nov 19 10:26:10 switch4 %LOG_PORT-5-IF_DOWN_CHANNEL_ADMIN_DOWN: Interface fc1/14 is down
(Channel admin down)
Nov 19 10:26:10 switch4 %LOG_PORT-5-IF_DOWN_CHANNEL_ADMIN_DOWN: Interface fc1/15 is down
(Channel admin down)
Nov 19 10:26:10 switch4 %LOG_PORT-5-IF_DOWN_CHANNEL_ADMIN_DOWN: Interface fc1/16 is down
(Channel admin down)
Nov 19 10:26:10 switch4 %LOG_PORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface
port-channel 1 is down (No operational members)
Nov 19 10:26:10 switch4 %LOG_PORT-5-IF_DOWN_ADMIN_DOWN: Interface port-channel 1 is down
(Administratively down)
Nov 19 10:26:10 switch4 %LOG_PORT_CHANNEL-5-FOP_CHANGED: port-channel 1: first operational
port changed from fc1/16 to none
Step 4 To reactivate the connection to the zone to be merged, enter the following command:
switch4(config-if)# no shut
The system output looks like this:
switch4(config-if)# Nov 19 10:28:11 switch4 %LOG_PORT_CHANNEL-5-FOP_CHAN
GED: port-channel 1: first operational port changed from none to fc1/15
Nov 19 10:28:21 switch4 %LOG_PORT-5-IF_UP: Interface port-channel 1 is up in mode TE
Nov 19 10:28:21 switch4 %LOG_PORT-5-IF_TRUNK_UP: Interface fc1/14, vsan 1 is up
Nov 19 10:28:21 switch4 %LOG_PORT-5-IF_TRUNK_UP: Interface fc1/15, vsan 1 is up
Nov 19 10:28:21 switch4 %LOG_PORT-5-IF_TRUNK_UP: Interface fc1/16, vsan 1 is up
Nov 19 10:28:21 switch4 %LOG_PORT-5-IF_TRUNK_UP: Interface fc1/14, vsan 1 is up
Nov 19 10:28:21 switch4 %LOG_PORT-5-IF_TRUNK_UP: Interface fc1/15, vsan 1 is up
Nov 19 10:28:21 switch4 %LOG_PORT-5-IF_TRUNK_UP: Interface fc1/16, vsan 1 is up
Step 5 To exit config mode and check the active zone sets, enter the following command:
switch4# show zoneset active
The system output looks like this:
zone name $default_zone$ vsan 1
After deactivating the zoneset on switch4 and performing a shutdown followed by a no shutdown on the ISL that connects it to switch3, the zone merge is processed again. Because switch3 has no active zoneset, it learns the active zoneset from switch4 during the zone merge process.
Misconfigured Zones Within an Active Zoneset in the Same VSAN
Even when the active zonesets contain the same zones for a VSAN on all the switches within a fabric, the members contained within those zones must also match or the zone merge will fail.
In this example, the active zonesets "wall" for VSAN 1 on switches switch3 and switch4 contain the same zone "excal1". However, the members of the zone are different on each switch. When the switches they are connected together to form the switch fabric, a zone merge failure occurs and VSAN 1 becomes isolated.
This topic has already been covered in a previous section. All you need to to do is to modify the membership of the zones so that any zones that have the same names also have the exact same membership.
For instructions about changing the membership of a zone, refer to the Cisco MDS 9000 Family Configuration Guide.