Symptom : Following a switch reload or a software upgrade, the startup configuration occasionally does not display feature FCIP.
Workaround : Once the switch is completely up, copy the running configuration to the startup configuration by entering the copy running-config startup-config command.
Symptom : Following an ISL flap that isolates some switches in the fabric (and the corresponding FCR peers) and then later merges them again, some FCR peers might get out of sync with the SSM switch FCR peer (that is, the FCR peer that owns a configuration) during the peer discovery phase. Consequently, some application flows might not come online, or might even get deleted.
Workaround : Enter the following supervisor commands on the problematic FCR peer, which will trigger this FCR peer to resync with the rest of the fabric, and force it to come out of the error state:
switch# test fc-redirect config sync-with-fabric
switch# test fc-redirect config sync-with-fabric all-peers
If more than one peer is affected by this issue, then additional steps are needed. If the application flows do not come online, then the application CLI commands need to be checked to verify that the flows did not get deleted. If they did, then the flows need to be recreated again through the application CLI command s or Cisco DCNM-SAN interface.
Symptom : After replacing a smart card that reached the maximum number of shares (16), the new smart card gets stuck in the initializing state.
Workaround : None.
Symptom : Periodic FCIP port flaps might occur when Tape Acceleration and IP compression are enabled and the MTU is configured at 9000. This situation might lead to a failure or an IPS core.
This symptom might be seen when the following configuration is applied to an FCIP tunnel:
– Tape Acceleration (TA) is enabled.
– Write Acceleration (required for TA) is enabled.
– IP compression in enabled.
– The MTU size is 9000.
This issue does not occur if TA is not configured and other parameters remain the same.
This issue does not occur if the MTU size of up to 2500 is configured and other parameters remain the same.
Symptom : Activation of a zone set in a VSAN in interop mode 1, fails with the following error messages:
%ZONE-2-ZS_CHANGE_ACTIVATION_FAILED_RESN_DOM: %$VSAN <vsan>%$ Activation failed
: reason Invalid zonset format domain <dom>
%ZONE-2-ZS_CHANGE_SFC_FAILED: %$VSAN <vsan>%$ SFC failed : domain <dom> returns
The failure occurs only if the following conditions exist:
– Zone set activation occurs from a Cisco MDS 9000 switch.
– The switch is running Cisco NX-OS Release 5.2(6) or Release 5.2(6a).
– The VSAN in question is running in interop mode 1.
– McData switches are present in the VSAN that is in interop mode 1.
There are two methods for working around this issue:
– Leave the VSAN in interop mode 1 and do all zoning from the McData side. This method is completely nondisruptive.
– Convert VSAN to interop 4. This method is disruptive to the entire VSAN because it needs to be suspended and the fcdomain domains need to be adjusted to be in the range of 1 to 31.
Symptom : Following an upgrade to NX-OS Release 5.2(6) or Release 5.2(6a), FCIP interfaces are down and the following message appears:
fcip1 is down (Tunnel port src interface unbound)
The licenses for all SAN_EXTN_OVER_IP packages in the grace period might be unnecessarily checked out.The inter-VSAN routing (IVR) process is using those licenses.
IVR is running based on the following license(s)
Conditions : This symptom might be seen when IVR is running over FCIP.
Workaround : To work aorund this issue, follow these steps:
1. Enter the show license usage license package name command to identify the applications that are using the license. For example:
switch# show license usage SAN_EXTN_OVER_IP_18_4
2. Back up the FCIP configuration and other applications from the running configuration on the switch.
3. Disable the features in use by the license. This action is disruptive to any non-FCIP IVR traffic.
4. Copy the license files off the switch as follows:
switch# copy licenses bootflash:lic-backup.tar
switch# copy bootflash:lic-backup.tar tftp://xxx.xxx.xxx.xxx
5. Enter the clear license command to clear the license.
6. Reinstall the license files.
7. Enable FCIP and reconfigure the switch.
8. Enable IVR and reconfigure the switch.
Symptom : Ports on the 24-port, 48-port, or 4/44-port 8-Gbps Fibre Channel switching modules go into a suspended state and packets cannot egress or ingress on the port. This issue can occur on Cisco MDS 9500 Series Directors and Cisco MDS 92221 switches when any of the following modules are installed:
– 24-port 8-Gbps Fibre Channel switching module (DS-X9224-96K9)
– 48-port 8-Gbps Fibre Channel switching module (DS-X9248-96K9)
– 4/44-port Host Optimized 8-Gbps Fibre Channel switching module (DS-X9248-48K9)
Conditions : This issue occurs after a nondisruptive upgrade to Cisco NX-OS Release 5.2(6x) or Release 5.2(8), and a link flap occurs on the ports.
Workaround : To work around this issue, do one of the following:
1. To temporarily recover a port in this state, enter the shut command followed by the no shut command on the port. A port that is recovered in this way can fail again.
2. To permanently work around this issue, reload the affected 24-port, 48-port, or 4/44-port 8-Gbps Fibre Channel switching module. If the switch contains primarily these modules, then reload the entire switch. Following the reload, the issue will not reoccur
3. Downgrade or upgrade the Cisco NX-OS software to a release that is not affected by this bug. If a port is in this suspended state at the time of the upgrade or downgrade, the upgrade or downgrade does not automatically recover the port. Enter the shut command followed by the n o shut command on the port after the upgrade or downgrade is complete to recover the port.
Further Problem Description: The main indications of a port exhibiting this issue are the following:
– The interface is up, there is an F port, there are zero frames per second in and out, there are output discards, and all B2B credits are remaining.
– Logging onboard shows small numbers of OFFLINE and TIMEOUT drops that are incrementing.
Symptom : In an SME or IOA deployment, if the number of H->T flows that are added exceeds 512, then subsequent disruptive platform events can leave an FCR unable to process interprocess (MTS) messages at a rate fast enough to support the scaled configuration. This situation can cause an MTS-buffers full condition in the FCR, which can result in unpredictable behavior. Some of the resulting errors may be unrecoverable, and may require a disruptive restart of the flows.
Workaround : Use CFS regions if a large number (greater than 512) of application flows need to be configured. CFS regions segment the FCR peer topology into manageable proportions.
Symptom : On the MDS 9513 switch, when an MSM-18/4 module boots up, it sends a request to the supervisor module to mount the modflash on the MSM-18/4 module. If there is a timeout or error in response, the following syslog message displays:
2011 Jul 14 01:18:13 sw-dc5-br2-12 %LC_MNT_MGR-SLOT3-2-LC_MNT_MGR_ERROR: SUP mount failed. MTS receive timedout
2011 Jul 14 01:19:06 sw-dc5-br2-12 %PROC_MGR-SLOT3-2-ERR_MSG: ERROR: PID 1144 (lc_mnt_mgr) exited abnormally, exit status (0xa)
2011 Jul 14 01:19:06 sw-dc5-br2-12 %MODULE-2-MOD_MINORSWFAIL: Module 3 (serial: JAE1141ZB43) reported a failure in service lc_mnt_mgr
This issue might be seen when the supervisor module is unusually busy and cannot process the mount request from the MSM-18/4 module, or the actual mount command on the supervisor takes a long time.
Workaround : Reload the MSM-18/4 module in the same slot/module where the modflash mount failed. A request will be sent to the supervisor to mount the modflash.
Symptom : The label for a smart card is not detected correctly.
Workaround : None.
Symptom : After creating an SME cluster, adding disks to a disk group, changing some disk states, and starting master key rekey, the key creation date is invalid.
Workaround : None.
Symptom : After creating an SME cluster in advanced mode, adding 500 disks to the disk group, exporting the keys, and importing the keys again, there is no scroll option in the import manage key window.
Workaround : None.
Symptom : In Generation 4 modules, the following errors might be observed. They are not a cause for concern to the normal operation of the hardware. These errors are logged in the onboard persistent log and can be viewed by entering the show logging onboard exception-log command after attaching to the module.
TBIRD_FWD_EBM_0_SER_PARITY_XXX : Informational
TBIRD_FWD_EBM_0_PACK0_EPR_DROP : Informational
TBIRD_FWD_EBM_1_PACK3_MISS_SOF : Informational
TBIRD_FWD_EBM_1_PACK0_MISS_EOP : Informational
TBIRD_FWD_EBM_1_PACK0_SF_TOO_SMALL : Informational
These errors are reported when there are internal correctable conditions encountered during operation, and the corrective action is taken. They do not necessarily indicated a problem with the switch or module.
The following error can occur in cases of corrupted frames in the ingress path (due to a bad cable or SFP). It does not necessarily indicated a problem with the switch hardware.
THB_IPA_IPA0_INTR_FLD_ERR_FROM_MEM : Warning
Workaround : None.
Symptom : After the primary KMC fails over to the secondary, KMC performance suffers. The system attempts to reacquire connectivity to the primary KMC for each operation, and the system waits for that attempt to time out before using the secondary KMC. Waiting for the timeout creates a significant delay for failed over operations, particularly when scaled.
This symptom occurs when the primary KMC fails (because of loss of connectivity).
Workaround : Restore connectivity to the primary KMC.
Symptom : When a trunk link fails, the 22.214.171.124.126.96.36.199.289.1.3 trap has an incorrect link failure reason of “gracefully down.”
This symptom might be seen only when a link failure occurs on a E, TE, or TF port.
Workaround : Ignore the link down reason because the link down trap is valid.
Symptom : A “VSAN down” trap is sent for VSAN 4094 when a trunk port goes down because the VSAN is not configured on the trunk.
This symptom might be seen only when a link down occurs on a E, TE, or TF port.
Workaround : Ignore the trap.
Symptom : Communication errors that occur during master key rekey are not handled gracefully.
Workaround : None.
Symptom : When creating a cluster, key management servers are not listed in the drop-down list.
Workaround : Manually enter the IP address of the server.
Symptom : For a failed master key rekey, the master key shares were generated and updated in the CKMC.
Workaround : None.
Symptom: During an In Service Software Upgrade (ISSU) or In Service Software Downgrade (ISSD), ports that are administratively up and also in the not-connected state get reinitialized.
Condition: This issue affects all Cisco MDS dual-supervisor systems.
Workaround: If starting an ISSU/ISSD from an affected version of Cisco MDS NX-OS software, ensure that all ports that are administratively up, but not connected, are shut down during the ISSU/ISSD.
More Information: This might cause the Cisco MDs Generation 3 Fibre Channel switching modules to drop all outbound traffic with the OFFLINE and TIMEOUT errors. For more information, see CSCuf31077.
Symptom : The security service crashes when configuring an SSH authentication key.
Configuring SSH keys multiple times within 10 minutes results in a HAP reset that resets the active supervisor.
Condition : This issue intermittently occurs when configuring an SSH authentication key.
Workaround : To avoid the supervisor reset, do not configure more than 2 SSH keys per 10 minutes.
Symptom : The full zoneset database in one or more VSANs may be empty after a supervisor switchover.
Conditions : This issue only occurs after the supervisors fail over or a user-initiated switch over occurs (but not an in service switchover situation, ie ISSU/ISSD). The precondition is created before the switchover by activating zone changes (such as adding or removing zones from a zoneset) and is more likely to occur on systems with very large zone configurations.
The symptom described here can occur if zones are modified while a switch is running any NX-OS release 5.2(2) to 5.2(8c) (inclusive), and 6.2(1) to 6.2(5) (inclusive).
Workaround : To recover from this condition follow these steps:
[i] If the full zoneset db for the affected VSAN contains multiple zonesets (ie, inactive zonesets) follow these steps:
a. add dummy zone to any zoneset in the full zoneset db for the vsan on a *neighbouring* switch, and then
b. if zoning mode is basic distribute the change or if the zoning mode is enhanced commit the change, and then
c. the dummy zone may be removed now and the zoneset redistributed/recommitted.
[ii] If the full zoneset db for the affected VSAN contains only a single zoneset (ie, no inactive zonesets) follow these steps:
a. copy the active zoneset db to the full zoneset db on the *affected* switch (it is only necessary to copy zonesets for VSANs that have empty databases), For example:
switch# zone copy active-zoneset full-zoneset vsan 1-4093
2. In both cases, save the config on the *affected* switch after step 1, For example:
switch# copy running start
To recover from condition 2 (isolated ISL) contact Cisco TAC for assistance.
Symptom : After a supervisor switchover, a subsequent ISL flap results in one or more VSAN becoming isolated on the ISL.
Conditions : These issues only occur in situations after the supervisors fail over or a user-initiated switch over occurs (but not an in service switchover situation, ie ISSU/ISSD). The preconditions are created before the switchover by activating zone changes (such as adding or removing zones from a zoneset) and is more likely to occur on systems with very large zone configurations.
The symptoms described here can occur if zones are modified while a switch is running any NX-OS release 5.2(2) to 5.2(8c) (inclusive), and 6.2(1) to 6.2(5) (inclusive).
Symptom : An ISL does not initialize quickly across a DWDM connection. The link can take minutes, hours or even days to connect. Once connected, it is stable.
Conditions : This issue only applies to DS-X9248-256K9 and DS-X9232-256K9 modules when connecting an ISL over a Tellabs 7100 DWDM path.
Symptom : MDS fabric switch running in NPV mode fails to generate port-monitor alerts.
Conditions : Applies to all MDS fabric switches running in NPV mode using port-monitor.
Applies to all versions prior to NX-OS 6.2(13).
Will occur only in the following conditions:
- After one or more upstream NP or TNP ports goes down and then back up.
- For each (T)NP port that flaps, one F port at the end of the range of ports
will no longer be scanned for port-monitor counter events. For example, if the
(T)NP port fc1/1 flaps then the last F port being used(ex. fc1/48) will no
longer be scanned for port-monitor counter events.
Workaround: There are two workarounds, one temporary and one permanent:
1 - Contact the TAC and they can assist with killing the port-monitor process. Once the port-monitor process restarts, all ports will be once again scanned.
This is only temporary in the sense that if an upstream (T)NP port flaps again the problem will recur.
2 - Move the (T)NP ports to the end of the ports on the switch. For example, if there are four (T)NP uplinks on a MDS 9148 or MDS 9148S, then move them to fc1/45-fc1/48. Once this has been done the problem will not recur.