Table Of Contents

Troubleshooting SANTap

Overview

Definitions

Limitations

Interface Restrictions

Initial Troubleshooting Checklist

Common Troubleshooting Tools in Fabric Manager

Common Troubleshooting Commands in the CLI

Messages, Logs and Databases

SANTap Issues

Host Login Problems

ITL Problems

Common Mismatch Problems

Troubleshooting SANTap

---

This chapter describes how to identify and resolve problems that might occur when implementing SANTap. This chapter includes the following sections:

•Overview

•Definitions

•Limitations

•Initial Troubleshooting Checklist

•SANTap Issues

Overview

---

Note SANTap is not supported in Release 3.3.1.

---

SANTap is an Intelligent Storage Services feature supported on the Storage Services Module (SSM). The SSM supports SANTap in Cisco MDS SAN-OS Release 2.0(2b) and later. The SANTap feature allows third-party data storage applications, such as long distance replication and continuous backup, to be integrated into the SAN.

SANTap has a control path and a data path. The control path services requests that create and manipulate replication sessions that are sent by an appliance. The control path is implemented using a SCSI-based protocol. An appliance sends requests to a control virtual target (CVT) which the SANTap process creates and monitors. Responses are sent to the control LUN on the appliance. SANTap also allows LUN mapping to appliance virtual targets (AVTs). You can have a maximum of 512 target LUNs.

When introducing SANTap-based applications, SANTap does not require reconfiguration of either the host or the target, and neither the host initiator nor the target is required to be directly connected to an SSM. This accomplished by assigning Cisco-specific WWNs to the virtual initiators (VIs) and data virtual targets (DVTs).

A host initiator or a target can be connected directly to an SSM. However, you must partition the SAN using VSANs. You must configure the host initiator and the DVT in one VSAN and configure the virtual initiator (VI) and the target in another VSAN.

Definitions

Table 23-1includes brief definitions of some of the common SANTap acronyms and terms.

Table 23-1 SANTap Acronyms 

Acronym / Term	Definition
AVT	Appliance virtual target. The portal through which an appliance can complete its synchronization with the target LUN. AVT can be thought of as a host proxy for the appliance.
CVT	Control virtual target. The portal through which an appliance communicates with SANTap. An initiator port on the appliance sends out SANTap Control Protocol requests to the SANTap process. When the request is processed, the response is sent back by the Cisco VI (virtual initiator) to a target port on the appliance.
DVT	Data virtual target. A DVT is created for every port on a multi-ported target that is included in SANTap-based services. The DVT is created in the host VSAN. Once a DVT is created and a host logs into the DVT, SANTap installs a DVTLUN for every configured LUN on the target for this host.
ITL	Initiator/target/LUN tuple. Uniquely identifies a LUN on a target.
Session	A record/object that is created for every ITL whose WRITE I/Os the appliance is interested in. A session can be thought of as a target LUN that requires SANTap-based services.
VI	Virtual initiator. SANTap creates 9 VIs in the appliance and target VSANs. In the appliance, VIs are used to send responses to SANTap CP requests and also to send copies of WRITE I/Os. In the target, VIs are used when the appliance is down and one of the SANTap recovery tools (ARL, PWL-BPR) is enabled. Note If the appliance implementation chooses not to use these recovery tools, the VIs are not used.

Limitations

This section describes SANTap limitations. Table 23-2 describes the component limitations by SAN-OS and SSI releases.

•One Recover Point Appliance (RPA) cluster can have only one target VSAN.

•SANTap does not support IVR. You cannot run SANTap and IVR together.

Table 23-2 SANTap Attribute Limitations 

Attribute	Limitation	Cisco SAN-OS and SSI Release
Hosts per DVT	16	Prior to Cisco SAN-OS 3.2
	32	Cisco SAN-OS 3.2
LUNs per host	256	All releases
LUNs per DVT	1500	Cisco SAN-OS 3.2
	1024	Prior to Cisco SAN-OS 3.2
DVTs per SSM	32	Cisco SAN-OS 3.1(2) with SSI 3.1(2m) and later
	16	Prior to Cisco SAN-OS 3.1(2) with SSI 3.1(2m)
Sessions per SSM	1024	Cisco SAN-OS 3.0(2) with SSI 3.0(2j)
	2048	Cisco SAN-OS 3.1(2b) with SSI 3.1(2m) and later
LUN ID address	32-bits	Cisco SAN-OS 3.2
	16-bits	Prior to Cisco SAN-OS 3.2
DVT LUNs per SSM	4080	All releases
ITLs per DPP	1500	Cisco SAN-OS 3.2
	1024	Prior to Cisco SAN-OS 3.2
ITLs per SSM	1024	Cisco SAN-OS 3.0(2) with SSI 3.0(2j)
	2048	Cisco SAN-OS 3.1(2b) with SSI 3.1(2m)
	4080	Cisco SAN-OS 3.1(3) with SSI 3.1(3) and later

Interface Restrictions

When enabled on a per-port basis, SANTap must be enabled on a group of four ports on an SSM. The following restrictions apply:

•The fewest number of interfaces that can be SANTap enabled is four.

•The first interfaces in a group must be 1, 5, 9, 13, 17, 21, 25, or 29. (You can specify fc5 through fc8, but not fc7 through fc10.)

•The groups of four interfaces do not need to be consecutive. For example, you can specify fc1 through fc8 and fc17 through fc20.

Initial Troubleshooting Checklist

Troubleshooting a SANTap problem involves gathering information about the configuration and connectivity of the various SANTap components. Begin your troubleshooting activity as follows:

Checklist	Check off
Verify licensing requirements. See Cisco MDS 9000 Family Fabric Manager Configuration Guide.
Verify that SANTap is enabled on the SSM module of the selected switch.
Verify the VSAN configuration and zones for the appliance, using the configuration and verification tools for the specific appliance.
Verify the physical connectivity for any problem ports or VSANs.

Common Troubleshooting Tools in Fabric Manager

Use the following Fabric Manager procedures to verify the VSAN configuration for the SANTap components.

•Choose Fabricxx > VSANxx to view the VSAN configuration in the Information pane.

•Choose Fabricxx > VSANxx and select the Host or Storage tab in the Information pane to view the VSAN members.

Common Troubleshooting Commands in the CLI

Use the show santap module command to display information about SANTap.

Example 23-1 Display SANTap CVT Information

switch# show santap module 2 cvt

CVT Information :

        cvt pwwn      = 23:4f:00:0d:ec:09:3c:02

        cvt nwwn      = 23:9d:00:0d:ec:09:3c:02

        cvt id        = 135895180

        cvt xmap_id   = 135895212

        cvt vsan      = 8

        cvt name      =

Example 23-2 Display SANTap DVT Information

switch# show santap module 2 dvt

DVT Information :

       dvt pwwn        = 50:06:0e:80:03:81:32:36

       dvt nwwn        = 50:06:0e:80:03:81:32:36

       dvt id          = 136773180

       dvt mode        = 3

       dvt vsan        = 12

       dvt if_index    = 0x1080000

       dvt fp_port     = 1

       dvt name        = MYDVT

       dvt tgt-vsan    = 9

       dvt io timeout                 = 10 secs

       dvt lun size handling          = 0

       dvt app iofail behaviour       = 1

       dvt quiesce behavior           = 1

       dvt tgt iofail behavior        = 0

       dvt appio failover time        = 50 secs

       dvt inq data behavior          = 0

Example 23-3 Display SANTap DVT LUN Information

switch# show santap module 2 dvtlun

DVT LUN Information :

       dvt pwwn       = 22:00:00:20:37:88:20:ef

       dvt lun        = 0x0

       xmap id        = 8

       dvt id         = 3

       dvt mode       = 0

       dvt vsan       = 3

       tgt pwwn       = 22:00:00:20:37:88:20:ef

       tgt lun        = 0x0

       tgt vsan       = 1

Example 23-4 Display SANTap Session Information

switch# show santap module 2 session

Session Information :

    session id    = 1

    host pwwn     = 21:00:00:e0:8b:12:8b:7a

    dvt pwwn      = 50:06:0e:80:03:81:32:36

    dvt lun       = 0x0

    tgt pwwn      = 50:06:0e:80:03:81:32:36

    tgt lun       = 0x0

    adt pwwn      = 33:33:33:33:33:33:33:00

    adt lun       = 0x0

    aci pwwn      = 22:22:22:22:22:22:22:22

    cvt pwwn      = 23:4f:00:0d:ec:09:3c:02

    num ranges    = 0

    session state = 5

    redirect mode = 0

    mrl requested 1

    MRL : vsan 8 RegionSize 4806720, DiskPWWN 0x234f000dec093c02, DiskLun 0x 1, startLBA 1

    pwl requested 1

    PWL : type 2, UpdatePol 2, RetirePolicy 4, pwl_start 1

    iol requested 0

Example 23-5 Display SANTap AVT Information

switch# show santap module 2 avt

AVT Information :

      avt pwwn       = 2a:4b:00:05:30:00:22:25

      avt nwwn       = 2a:60:00:05:30:00:22:25

      avt id         = 12

      avt vsan       = 4

      avt if_index   = 0x1080000

      hi pwwn        = 21:00:00:e0:8b:07:61:aa

      tgt pwwn       = 22:00:00:20:37:88:20:ef

      tgt vsan       = 1

Example 23-6 Display SANTap AVT LUN Information

switch# show santap module 2 avtlun

AVT LUN Information :

      avt pwwn       = 2a:4b:00:05:30:00:22:25

      avt lun        = 0x0

      xmap id        = 16

      avt id         = 12

      tgt lun        = 0x0

Example 23-7 Display SANTap Remote Virtual Terminal Information

switch# show santap module 2 rvt

RVT Information :

      rvt pwwn       = 2a:61:00:05:30:00:22:25

      rvt nwwn       = 2a:62:00:05:30:00:22:25

      rvt id         = 17

      rvt vsan       = 4

      rvt if_index   = 0x1080000

Example 23-8 Display SANTap Remote Virtual Terminal LUN Information

switch# show santap module 2 rvtlun

RVT LUN Information :

      rvt pwwn       = 2a:61:00:05:30:00:22:25

      rvt lun        = 0x0

      xmap id        = 22

      rvt id         = 17

      app pwwn       = 22:00:00:20:37:39:b1:00

      app lun        = 0x0

      app vsan       = 1

Use the following commands to display more advanced troubleshooting information for SANTap.

•show tech-support

•show santap module 2 tech-support

•show isapi tech-support

•show santap vttbl dvt dvt-pwwn

Messages, Logs and Databases

The following log files and databases can provide helpful information when troubleshooting SANTap problems.

•Look for SSM_CRIT in Sup syslog messages.

•SANTap logs are available via the show isapi tech-support CLI command. Search for the strings "Error" and "failed" or "failure."

•Review FCNS and FLOGI databases, using the show fcns and show flogi CLI commands.

SANTap Issues

This section includes the following topics:

•Host Login Problems

•ITL Problems

•Common Mismatch Problems

Host Login Problems

Host login problems can be caused by DVT limitations, connectivity, or other issues.

To diagnose host login problems, follow these steps:

---

Step 1 Use the show santap vttbl dvt dvt-pwwn command to display SANTap information for the DVT.

Step 2 Use the show santap vttbl dvt dvt-pwwn host-pwwn command to display SANTap information for the DVT and the host.

Step 3 Enable logging on the appropriate SSM module.

Step 4 Issue the following debug commands:

a. debug vsd vfc-felogin

b. debug partner 0x92810000 d1

c. debug partner 0x92810000 d2

Step 5 Review the logs to determine the problem and then take the appropriate corrective action.

---

ITL Problems

An ITL problem may occur if the number of ITLs exceeds the limitations for the version of Cisco SAN-OS and SSI in use. For specific ITL limitations, see the "Limitations" section

To diagnose and resolve ITL problems, follow these steps:

---

Step 1 Use the show isapi dpp 4 queue command to display DPP queue information.

Step 2 Verify that the number of ITLs on a DPP is within the limitations for the version of Cisco SAN-OS and SSI in use. Use the show isapi dpp 4 queue incl LUN and show isapi dpp 4 queue count commands.

Step 3 Verify that the number of ITLs on an SSM is within the limitations for the version of Cisco SAN-OS and SSI in use. Issue the show isapi dpp all queue command, using the incl LUN and count parameters.

---

Common Mismatch Problems

Problems are often caused by mismatching component versions, or using devices that are not supported by the interoperability matrix.

The following lists common mismatch situations:

Table 23-3 Common SANTap Mismatch Problems 

Problem	Description
Version mismatch between SSM and the RPA.	The version of SSM and the version of the replication appliance are not compatible.
Version mismatch between the Supervisor and the SSI image.	The version of the Sup and the SSI image are not compatible.
Hosts, targets, HBAs or switches are not supported by the interoperability matrix.	•Review the Cisco SANTap interoperability matrix at http://www.cisco.com/en/US/products/ps5989/products_device_support_tables_list.html.
CVT is in the host VSAN.	The CVT is the portal through which the appliance communicates with SANTap, and cannot be in the host VSAN.
IVR and SANTap are being used together.	IVR and SANTap both perform straddling across VSANs and cannot be used together.
VIs in a DVT VSAN (CVT and DVT in the same VSAN).	This results in the creation of one CPP VI and eight DPP VIs in the DVT VSAN. The VIs attempt to login to DVT, resulting in non-deterministic behavior.
Overprovisioning, including: •Too many ITLs per SSM •Too many hosts per DVT •Too many ITLs per DPP	When replication is enabled, AVT LUNs are created, and that can increase the ITL count over the limit. (See "Limitations" section.) If Reservation Support is not enabled on the RPA: •26 AVT LUNs are created at a time. •The appliance completes recovery of these LUNs and then deletes them before creating more. •This behavior does not significantly increase the ITL count. If Reservation Support is enabled on the RPA: •All AVT and AVT LUNs are permanently created. •In RPA 2.3, only half the appliance posts are zoned with AVTs. This does not increase the ITL count significantly. •In RPA 2.4, all appliance ports are zoned with AVTs. This behavior can increase the ITL count significantly.
Same pWWN occurs more than once in the same VSAN.	Misconfiguration can result in two DVTs (or two VIs) with the same WWN in the same VSAN. For example, assume that two DVTs are created on different SSMs or on different DPPs. Both of these DVTs have the same back-end VSAN. When a host HBA logs into both DVTs, an attempt is made to create two VIs with the same WWN in the same back-end VSAN. This results in non-deterministic behavior.
A host is moved from the front-end VSAN to the back-end VSAN.	There is a VI in the back-end VSAN with the same pWWN as the host. Before you can move the host: •Shut the host port. •Purge to remove VI from the back-end VSAN. The host can now be safely moved.
Inaccurate zoning.	Zoning solutions differ, based on the Cisco SAN-OS and SSI versions in use. •With SSI 3.0(2j), you must have default zoning in the back-end VSAN, or zone the target and VIs together in the back-end VSAN. •With SSI 3.1(2), only the host VI and target need to be zoned together in the back-end VSAN.
Adding and removing hosts without performing a purge.	If you have 16 hosts and you remove one and add another, the new host will not see the LUNs. In this situation, perform a purge to clear one of the 16 entries after removing the host. Then you can add the new host to the DVT.