Cisco Nexus 1000V Troubleshooting Guide, Release 4.2(1) SV1(4)
Ports
Downloads: This chapterpdf (PDF - 202.0KB) The complete bookPDF (PDF - 5.9MB) | Feedback

Ports

Table Of Contents

Ports

Information About Ports

Information About Interface Characteristics

Information About Interface Counters

Information About Link Flapping

Information About Port Security

Port Diagnostic Checklist

Problems with Ports

Cannot Enable an Interface

Port Link Failure or Port Not Connected

Link Flapping

Port ErrDisabled

VM Cannot Ping a Secured Port

Port Security Violations

Port Troubleshooting Commands


Ports


This chapter describes how to identify and resolve problems with ports and includes the following topics:

Information About Ports

Port Diagnostic Checklist

Problems with Ports

Port Troubleshooting Commands

Information About Ports

This section includes the following topics:

Information About Interface Characteristics

Information About Interface Counters

Information About Link Flapping

Information About Port Security

Information About Interface Characteristics

Before a switch can relay frames from one data link to another, you must define the characteristics of the interfaces through which the frames are received and sent. The configured interfaces can be Ethernet (physical) interfaces, virtual Ethernet interfaces, and the management interface (mgmt0),.

Each interface has the following:

Administrative Configuration

The administrative configuration does not change unless you modify it. This configuration has attributes that you can configure in administrative mode.

Operational state

The operational state of a specified attribute, such as the interface speed. This state cannot be changed and is read-only. Some values may not be valid when the interface is down (such as the operation speed).

For a complete description of port modes, administrative states, and operational states, see the Cisco Nexus 1000V Interface Configuration Guide, Release 4.2(1)SV1(4).

Information About Interface Counters

Port counters are used to identify synchronization problems. Counters can show a significant disparity between received and transmitted frames. To display interface counters, use the following command:

show interface ethernet slot number counters

See Example 6-11.

Values stored in counters can be meaningless for a port that has been active for an extended period. Clearing the counters provides a better idea of the actual link behavior at the present time. Create a baseline first by clearing the counters.

clear counters interface ethernet slot-number

Information About Link Flapping

When a port continually goes up and down, it is said to be flapping, sometimes called link flapping. When a port is flapping, it cycles through the following states, in this order, and then starts over again:

1. Initializing - The link is initializing.

2. Offline - The port is offline.

3. Link failure or not connected - The physical layer is not operational and there is no active device connection.

To troubleshoot link flapping, see the "Information About Link Flapping" section.

Information About Port Security

The port security feature allows you to secure a port by limiting and identifying the MAC addresses that can access the port. Secure MACs can be manually configured or dynamically learned.

For detailed information about port security, see the Cisco Nexus 1000V Security Configuration Guide, Release 4.2(1)SV1(4).

Type of port
Is Port Security Supported?

vEthernet access

Yes

vEthernet trunk

Yes

vEthernet SPAN destination

No

Standalone Ethernet interfaces

No

Port channel members

No


To troubleshoot problems with port security, see the following:

"VM Cannot Ping a Secured Port" section

"Port Security Violations" section

Port Diagnostic Checklist

Use the following checklist to diagnose port interface activity.

For more information about port states, see the Cisco Nexus 1000V Interface Configuration Guide, Release 4.2(1)SV1(4).

Table 6-1 Port Diagnostic Checklist  

Checklist 
Example

Verify that the module is active.

show module

See Example 6-1.

 

Verify that the VSM is connected to the vCenter Server.

show svs connections

See Example 6-3.

 

On the vSphere Client connected to vCenter Server, verify that required port profiles are assigned to the physical NICS and the virtual NICS.

   

Verify that the ports have been created.

show interface brief

See Example 6-8.

 

Verify the state of the interface.

show interface ethernet

See Example 6-10.

 

Problems with Ports

This section includes possible causes and solutions for the following symptoms:

Cannot Enable an Interface

Port Link Failure or Port Not Connected

Link Flapping

Port ErrDisabled

VM Cannot Ping a Secured Port

Port Security Violations

Cannot Enable an Interface

Use these guidelines to troubleshoot an interface that cannot be enabled.

Possible Cause
Solution

Layer 2 port is not associated with an access VLAN or the VLAN is suspended.

1. Verify that the interface is configured in a VLAN.
show interface brief

2. If not already, associate the interface with an access VLAN.

3. Determine the VLAN status.
show vlan brief

4. If not already active, configure the VLAN as active.
config t
vlan
vlan-id
state active


Port Link Failure or Port Not Connected

Use these guidelines to troubleshoot a port that remains in link failure or not connected.

Table 6-2 Troubleshooting Ports in Link Failure or Not Connected  

Possible Cause
Solution

Port connection is bad.

1. Verify the port state.

show system internal ethpm info

2. Disable and then enable the port.

shut
no shut

3. Move the connection to a different port on the same module or a different module.

4. Collect the ESX side NIC configuration.

vss-support

Link is stuck in initialization state or the link is in a point-to-point state.

1. Check for a link failure system message.
Link Failure, Not Connected

show logging

2. Disable and then enable the port.

shut
no shut

3. Move the connection to a different port on the same module or a different module.

4. Collect the ESX side NIC configuration.

vss-support


Link Flapping

When troubleshooting unexpected link flapping, it is important to have the following information:

Who initiated the link flap.

The actual reason for the link being down.

For a definition of link flapping, see the "Link Flapping" section.

Table 6-3 Troubleshooting link flapping  

Possible Cause
Solution

The bit rate exceeds the threshold and puts the port into an error disabled state.

Disable and then enable the port.

shut
no shut

The port should return to the normal state.

A hardware failure or intermittent hardware error causes a packet drop in the switch.

A software error causes a packet drop.

A control frame is erroneously sent to the device.

An external device may choose to initialize the link again when encountering the error. If so, the exact method of link initialization varies by device.

1. Determine the reason for the link flap as indicated by the MAC driver.

2. Use the debug facilities on the end device to troubleshoot the problem.

ESX errors, or link flapping on the upstream switch.

Use the troubleshooting guidelines in the documentation for your ESX or upstream switch.


Port ErrDisabled

Use the guidelines in this section to troubleshoot ports that are error disabled.

Table 6-4 Troubleshooting error disabled ports 

Possible Cause
Solution

Defective or damaged cable.

1. Verify the physical cabling.

2. Replace or repair defective cables.

3. Re-enable the port.

shut
no shut

You attempted to add a port to a port channel that was not configured identically; and the port is then errdisabled.

1. Display the switch log file and identify the exact configuration error in the list of port state changes.

show logging logfile

See Example 6-6.

2. Correct the error in the configuration and add the port to the port channel.

3. Re-enable the port.

shut
no shut

VSM application error

1. Identify the component which errored while bringing up the port.

show logging logfile | grep interface_number

See Example 6-7.

2. Identify the error transition.

show system internal ethpm event-history interface interface_number

3. Open a support case and submit the output of the above commands.

For more information see the "Before Contacting Technical Support" section.


VM Cannot Ping a Secured Port

Use these troubleshooting guidelines when you cannot ping a secured port from a VM.

Table 6-5 Troubleshooting VM Cannot Ping a Secured Port  

Possible Cause
Solution

The vEthernet interface is not up.

1. Verify the state of the vEthernet interface.

show interface vethernet number

2. If the interface is down, enable it.

shut
no shut

Drop on Source Miss (DSM) is set.

New MAC addresses cannot be learned by this port.

1. Verify the port security configuration.

module vem 3 execute vemcmd show portsec stats

2. If DSM is set, clear the DSM bit on the VSM.

no port-security stop learning

The packet VLAN is not allowed on the port.

1. Identify the packet VLAN ID.

show svs domain

2. Verify that the packet VLAN is allowed on VEM uplink ports.

show port-profile na uplink-all

3. If the packet VLAN is not allowed on the uplink port profile, add it to the allowed VLAN list.

The packet VLAN is not allowed on the upstream switch port.

1. Identify the upstream neighbors connected to the interface.

show cdp neighbors

2. Log in to the upstream switch and verify that the packet VLAN is allowed on the port.

show running-config interface gigabitEthernet slot/port

3. If the packet VLAN is not allowed on the port, add it to the allowed VLAN list.


Port Security Violations

Use these troubleshooting guidelines when a vEthernet port is disabled because of a security violation.

For detailed information about port security, see the Cisco Nexus 1000V Security Configuration Guide, Release 4.2(1)SV1(4).

Table 6-6 Troubleshooting Port Security Violations  

Possible Cause
Solution

The configured maximum number of secured addresses on the port is exceeded.

A MAC that is already secured on one port is then seen on another secure port.

1. Display the secure addresses.

show port -security address vethernet number
show port-security

2. Identify ports with a security violation.

show logging | inc "PORT-SECURITY-2-ETH_PORT_SEC_SECURITY_VIOLATION_MAX_MAC_VLAN"

3. Correct the security violation.

4. Enable the interface.

shut
no shut


Port Troubleshooting Commands

You can use the commands in this section to troubleshoot problems related to ports.

Table 6-7 Port Troubleshooting Commands  

Command
Purpose

show module module-number

Displays the state of a module.

See Example 6-1.

show svs domain

Displays the domain configuration.

See Example 6-2.

show svs connections

Displays the Cisco Nexus 1000V connections.

See Example 6-3.

show cdp neighbors

Displays the neighbors connected to an interface.

See Example 6-4.

show port internal event-history interface

Displays information about the internal state transitions of the port.

See Example 6-5.

show logging logfile

Displays logged system messages.

See Example 6-6.

show logging logfile

Displays logged system messages for a specified interface.

See Example 6-7.

show interface brief

Displays a table of interface states.

See Example 6-8.

show interface ethernet

Displays the configuration for a named Ethernet interface, including the following:

Administrative state

Speed

Trunk VLAN status

Number of frames sent and received

Transmission errors, including discards, errors, CRCs, and invalid frames

See Example 6-9.

See Example 6-10.

show interface ethernet counters

Displays port counters for identifying synchronization problems.

For information about counters, see "Information About Interface Counters" section.

See Example 6-11.

show interface vethernet

Displays the vEthernet interface configuration.

See Example 6-12.

show interface status

Displays the status of the named interface.

show interface capabilities

Displays tabular view of all configured port profiles.

See Example 6-13.

show interface virtual port mapping

Displays the virtual port mapping for all vEthernet interfaces:

See Example 6-14.

show system internal ethpm errors

 

show system internal ethpm event-history

 

show system internal ethpm info

 

show system internal ethpm mem-stats

 

show system internal ethpm msgs

 

show system internal vim errors

 

show system internal vim event-history

 

show system internal vim info

 

show system internal vim mem-stats

 

show system internal vim msgs

 

module vem execute vemcmd show portsec status

Displays the port security status of the port. If enabled, the output shows an LTL connected to the VM network adapter.

See Example 6-15

show port-security

Displays information about the secured MAC addresses in the system.

See Example 6-16.

show port-security interface veth

Displays secure vEthernet interfaces.

show port -security address vethernet

Displays information about secure addresses on an interface.

See Example 6-17.

show system internal port-security msgs

 

show system internal port-security errors

 

show system internal l2fm msgs

 

show system internal l2fm errors

 

show system internal l2fm info detail

 

show system internal pktmgr interface brief

 

show system internal pktmgr client detail

 

For detailed information about show command output, see the Cisco Nexus 1000V Command Reference, Release 4.2(1)SV1(4).

EXAMPLES

Example 6-1 show module

n1000v# show mod 3
Mod  Ports  Module-Type                      Model              Status
---  -----  -------------------------------- ------------------ ------------
3    248    Virtual Ethernet Module                             ok
 
Mod  Sw              Hw
---  --------------  ------
3    NA              0.0
 
Mod  MAC-Address(es)                         Serial-Num
---  --------------------------------------  ----------
3    02-00-0c-00-03-00 to 02-00-0c-00-03-80  NA
 
Mod  Server-IP        Server-UUID                           Server-Name
---  ---------------  ------------------------------------  --------------------
3    192.168.48.20    496e48fa-ee6c-d952-af5b-001517136344  frodo

Example 6-2 show svs domain

n1000v# show svs domain
SVS domain config:
  Domain id:    559
  Control vlan: 3002
  Packet vlan:  3003
  L2/L3 Aipc mode: L2
  L2/L3 Aipc interface: mgmt0
  Status: Config push to VC successful.
n1000v#

Example 6-3 show svs connections

n1000v# show svs connections 
connection VC:
    ip address: 192.168.0.1
    protocol: vmware-vim https
    certificate: default
    datacenter name: Hamilton-DC
    DVS uuid: ac 36 07 50 42 88 e9 ab-03 fe 4f dd d1 30 cc 5c
    config status: Enabled
    operational status: Connected
n1000v#

Example 6-4 show cdp neighbors

n1000V#show cdp neighbors
Capability Codes: R - Router, T - Trans-Bridge, B - Source-Route-Bridge
                  S - Switch, H - Host, I - IGMP, r - Repeater,
                  V - VoIP-Phone, D - Remotely-Managed-Device,
                  s - Supports-STP-Dispute
 
   
Device ID              Local Intrfce   Hldtme  Capability  Platform      Port ID
swordfish-6k-2         Eth3/2        149    R S I       WS-C6506-E    Gig1/38
n1000V#

Example 6-5 show port internal event-history interface

n1000v# show port internal event-history interface e1/7
>>>>FSM: <e1/7> has 86 logged transitions<<<<<
1) FSM:<e1/7> Transition at 647054 usecs after Tue Jan  1 22:44..
    Previous state: [PI_FSM_ST_IF_NOT_INIT]
    Triggered event: [PI_FSM_EV_MODULE_INIT_DONE]
    Next state: [PI_FSM_ST_IF_INIT_EVAL]
2) FSM:<e1/7> Transition at 647114 usecs after Tue Jan  1 22:43..
    Previous state: [PI_FSM_ST_IF_INIT_EVAL]
    Triggered event: [PI_FSM_EV_IE_ERR_DISABLED_CAP_MISMATCH] 
    Next state: [PI_FSM_ST_IF_DOWN_STATE] 

Example 6-6 show logging logfile

n1000v# show logging logfile
 . . .
Jan  4 06:54:04 switch %PORT_CHANNEL-5-CREATED: port-channel 7 created
Jan  4 06:54:24 switch %PORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel 7 
is down (No operational members)
Jan  4 06:54:40 switch %PORT_CHANNEL-5-PORT_ADDED: e1/8 added to port-channel 7
Jan  4 06:54:56 switch %PORT-5-IF_DOWN_ADMIN_DOWN: Interface e1/7 is down (Admnistratively 
down)
Jan  4 06:54:59 switch %PORT_CHANNEL-3-COMPAT_CHECK_FAILURE: speed is not compatible
Jan 4 06:55:56 switch%PORT_CHANNEL-5-PORT_ADDED: e1/7 added to port-channel 7
n1000v#

Example 6-7 show logging logfile | grep interface_number

n1000v# show logging logfile | grep Vethernet3626
2011 Mar 25 10:56:03 n1k-bl %VIM-5-IF_ATTACHED: Interface Vethernet3626
is attached to Network Adapter 8 of gentoo-pxe-520 on port 193 of module
13 with dvport id 6899
2011 Mar 25 11:10:06 n1k-bl %ETHPORT-2-IF_SEQ_ERROR: Error ("Client data
inconsistency") while communicating with component MTS_SAP_ACLMGR for
opcode MTS_OPC_ETHPM_PORT_PRE_CFG (RID_PORT: Vethernet3626)
2011 Mar 25 11:10:06 n1k-bl %ETHPORT-2-IF_DOWN_ERROR_DISABLED: Interface
Vethernet3626 is down (Error disabled. Reason:Client data inconsistency)

Example 6-8 show interface brief

n1000v# show int brief
--------------------------------------------------------------------------------
Port VRF Status IP Address Speed MTU
--------------------------------------------------------------------------------
mgmt0 -- up 172.23.232.141 1000 1500
--------------------------------------------------------------------------------
Ethernet VLAN Type Mode Status Reason Speed Port
Interface Ch #
--------------------------------------------------------------------------------
Eth3/2 1 eth trunk up none 1000(D) --
Eth3/3 1 eth access up none 1000(D) --
n1000v#

Example 6-9 show interface ethernet

n1000v# show interface e1/14
e1/7 is down (errDisabled)

Example 6-10 show interface ethernet

Example: 
n1000v# show interface eth3/2
Ethernet3/2 is up
  Hardware: Ethernet, address: 0050.5653.6345 (bia 0050.5653.6345)
  MTU 1500 bytes, BW -598629368 Kbit, DLY 10 usec,
     reliability 0/255, txload 0/255, rxload 0/255
  Encapsulation ARPA
  Port mode is trunk
  full-duplex, 1000 Mb/s
  Beacon is turned off
  Auto-Negotiation is turned off
  Input flow-control is off, output flow-control is off
  Auto-mdix is turned on
  Switchport monitor is off
    Rx
    18775 Input Packets 10910 Unicast Packets
    862 Multicast Packets 7003 Broadcast Packets
    2165184 Bytes
    Tx
    6411 Output Packets 6188 Unicast Packets
    216 Multicast Packets 7 Broadcast Packets 58 Flood Packets
    1081277 Bytes
    1000 Input Packet Drops 0 Output Packet Drops
    1 interface resets
n1000v# 

Example 6-11 show interface ethernet counters

n1000v# show interface eth3/2 counters
 
--------------------------------------------------------------------------------
Port                InOctets       InUcastPkts      InMcastPkts      InBcastPkts
--------------------------------------------------------------------------------
Eth3/2             2224326         11226           885          7191
 
--------------------------------------------------------------------------------
Port               OutOctets      OutUcastPkts     OutMcastPkts     OutBcastPkts
--------------------------------------------------------------------------------
Eth3/2             1112171          6368           220             7

Example 6-12 show interface vEthernet

n1000v# show interface veth1
Vethernet1 is up
    Port description is gentoo1, Network Adapter 1
    Hardware is Virtual, address is 0050.56bd.42f6
    Owner is VM "gentoo1", adapter is Network Adapter 1
    Active on module 33
    VMware DVS port 100
    Port-Profile is vlan48
    Port mode is access
    Rx
    491242 Input Packets 491180 Unicast Packets
    7 Multicast Packets 55 Broadcast Packets
    29488527 Bytes
    Tx
    504958 Output Packets 491181 Unicast Packets
    1 Multicast Packets 13776 Broadcast Packets 941 Flood Packets
    714925076 Bytes
    11 Input Packet Drops 0 Output Packet Drops
n1000v# 

Example 6-13 show interface capabilities

n1000v# show interface capabilities
mgmt0
  Model:                 --
  Type:                  --
  Speed:                 10,100,1000,auto
  Duplex:                half/full/auto
  Trunk encap. type:     802.1Q
  Channel:               no
  Broadcast suppression: none
  Flowcontrol:           rx-(none),tx-(none)
  Rate mode:             none
  QOS scheduling:        rx-(none),tx-(none)
  CoS rewrite:           yes
  ToS rewrite:           yes
  SPAN:                  yes
  UDLD:                  yes
  Link Debounce:         no
  Link Debounce Time:    no
  MDIX:                  no
  Port Group Members:    none
 
   
port-channel1
  Model:                 unavailable
  Type:                  unknown
  Speed:                 10,100,1000,10000,auto
  Duplex:                half/full/auto
  Trunk encap. type:     802.1Q
  Channel:               yes
  Broadcast suppression: percentage(0-100)
  Flowcontrol:           rx-(off/on/desired),tx-(off/on/desired)
  Rate mode:             none
  QOS scheduling:        rx-(none),tx-(none)
  CoS rewrite:           yes
  ToS rewrite:           yes
  SPAN:                  yes
  UDLD:                  no
  Link Debounce:         no
  Link Debounce Time:    no
  MDIX:                  no
  Port Group Members:    none
 
   
port-channel2
  Model:                 unavailable
  Type:                  unknown
  Speed:                 10,100,1000,10000,auto
  Duplex:                half/full/auto
  Trunk encap. type:     802.1Q
  Channel:               yes
  Broadcast suppression: percentage(0-100)
  Flowcontrol:           rx-(off/on/desired),tx-(off/on/desired)
  Rate mode:             none
  QOS scheduling:        rx-(none),tx-(none)
  CoS rewrite:           yes
  ToS rewrite:           yes
  SPAN:                  yes
  UDLD:                  no
  Link Debounce:         no
  Link Debounce Time:    no
  MDIX:                  no
  Port Group Members:    none
 
   
port-channel12
  Model:                 unavailable
  Type:                  unknown
  Speed:                 10,100,1000,10000,auto
  Duplex:                half/full/auto
  Trunk encap. type:     802.1Q
  Channel:               yes
  Broadcast suppression: percentage(0-100)
  Flowcontrol:           rx-(off/on/desired),tx-(off/on/desired)
  Rate mode:             none
  QOS scheduling:        rx-(none),tx-(none)
  CoS rewrite:           yes
  ToS rewrite:           yes
  SPAN:                  yes
  UDLD:                  no
  Link Debounce:         no
  Link Debounce Time:    no
  MDIX:                  no
  Port Group Members:    none
 
   
control0
  Model:                 --
  Type:                  --
  Speed:                 10,100,1000,auto
  Duplex:                half/full/auto
  Trunk encap. type:     802.1Q
  Channel:               no
  Broadcast suppression: none
  Flowcontrol:           rx-(none),tx-(none)
  Rate mode:             none
  QOS scheduling:        rx-(none),tx-(none)
  CoS rewrite:           yes
  ToS rewrite:           yes
  SPAN:                  yes
  UDLD:                  yes
  Link Debounce:         no
  Link Debounce Time:    no
  MDIX:                  no
  Port Group Members:    none
 
   
n1000v#

Example 6-14 show interface virtual port-mapping

n1000v# show interface virtual port-mapping 
 
   
-------------------------------------------------------------------------------
Port    Hypervisor Port    Binding Type    Status    Reason 
-------------------------------------------------------------------------------
Veth1   DVPort5747         static          up        none
Veth2   DVPort3361         static          up        none
n1000v# 

Example 6-15 module vem execute vemcmd show portsec status

n1000V# module vem 3 execute vemcmd show portsec stats
  LTL   if_index  cp-cnt  Max         Aging   Aging     DSM  Sticky   VM
                          Secure      Time    Type      Bit  Enabled  Name
                          Addresses
   47   1b020000       0          1       0   Absolute  Clr       No  VM-Pri.eth1

n1000V#

Example 6-16 show port security

n1000V# show port-security
Total Secured Mac Addresses in System (excluding one mac per port)     : 0
Max Addresses limit in System (excluding one mac per port) : 8192
 
   
----------------------------------------------------------------------------
Secure Port  MaxSecureAddr  CurrentAddr  SecurityViolation  Security Action
                (Count)       (Count)          (Count)
----------------------------------------------------------------------------
Vethernet1            1              0              0              Shutdown
==========================================================================

Example 6-17 show port security address interface vethernet

n1000v#show port-security address interface vethernet 1
Total Secured Mac Addresses in System (excluding one mac per port)     : 0
Max Addresses limit in System (excluding one mac per port) : 8192
 
   
----------------------------------------------------------------------
                    Secure Mac Address Table
----------------------------------------------------------------------
Vlan    Mac Address             Type             Ports      Remaining age
                                                              (mins)
----    -----------            ------            -----     ---------------
  65    0050.56B7.7DE2        DYNAMIC        Vethernet1        0
======================================================================