UCS Appliance Port Connectivity and Troubleshooting
Available Languages
Download Options
Bias-Free Language
The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
Contents
Introduction
This document is intended to assist Unified Computing Systems (UCS) administrators who configure direct attached storage on the Cisco UCS platform.
Contributed by Dmitri Filenko and Andreas Nikas, Cisco TAC Engineers.
Prerequisites
Requirements
There are no specific requirements for this document.
Components Used
This document is not restricted to specific software and hardware versions.
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
Conventions
Refer to Cisco Technical Tips Conventions for information on document conventions.
Background Information
UCS appliance ports are used to directly connect a storage appliance to UCS fabric interconnects.
An appliance port behave similar to virtual ethernet (vEthernet or vEth) ports:
- It contains a list of allowed virtual LANs (VLANs).
- MAC addresses on these interfaces are learned by the fabric interconnect.
- It requires an uplink for pinning.
When a VLAN is created for an appliance port in the Appliances section of LAN tab, make sure to create the same VLAN in the LAN Cloud tab.
Why Appliance Port VLANs Should be Allowed on Uplinks
There are number of reasons why an upstream switch should allow storage appliance port traffic. These include:
- If storage must be accessed outside of the UCS domain.
- If storage and servers are located in different subnets.
- If storage is configured in Active/Passive mode and both fabric interconnects require communication to the same controller.
- In certain failover scenarios.
Definition of a Unified Storage Port
Prior to UCS Release 2.1(1a), appliance ports only worked for IP-based storage traffic, such as the Network File System (NFS) and the Internet Small Computer System Interfaces (iSCSIs). In UCS Releases 2.1(1a) and later, the ability for both IP-based storage and Fibre Channel over Ethernet (FCoE) to be used on the same interface was added. This type of interface is called a Unfied Storage port. In order to use this feature, the storage controller must have a Converged Network Adapter (CNA) that is capable of FCoE and traditional Ethernet on the same port.
Tip: For information about how to configure a Unified Storage port, refer to the Configuring an Appliance Port as a Unified Storage Port section of the Cisco UCS Manager GUI Configuration Guide, Release 2.1.
In order to verify that the port is configured as a unified storage port, login to the Cisco NX-OS shell and verify the port's current configuration with this command:
ucs01-A(nxos)# show running-config interface eth 1/5
interface Ethernet1/5
description AF: UnifiedStorage
...
Appliance Port Port-Channel
UCS supports Static and Link Aggregation Control Protocol (LACP) port channels for appliance port configuration. However, there is no virtual Port-Channel (vPC) support.
When to Use Trunk or Access Mode
The decision about whether you should configure your appliance ports in Trunk or Access mode depends on the capabilities of the storage appliance. If the storage appliance has the capability to add VLAN tags, then Cisco recommends that you configure the appliance ports in Trunk mode and configure the VLAN tagging on the storage side for maximum flexibility. In this case, multiple VLANs can be used on the same link, which allows for isolation of different protocols on the wire. If the storage controller is not capable of VLAN tagging, then the use of an access port is required.
Situations to Avoid
- Configuration of the VLAN tagging on both the storage side and on the UCS sidesimultaneously.
Referred to as double VLAN tagging, this configuration breaks communication over the VLAN. If a VLAN tag is added in the appliance port configuration on the UCS side (done by configuring the appliance port in either Access mode or when using the Native VLAN while in Trunk mode), then do not configure tagging for the same VLAN on the storage controller side.
- Use of the same VLAN for multiple storage protocol traffic
As a best practice, each protocol should be placed into a separate VLAN. For example, when you access both the NFS Shares and the iSCSI LUNs via the same appliance port, configure the appliance port in Trunk mode with two different VLANs allowed across the link (one for the NFS and another for the iSCSI).Note: The native VLAN on the appliance port is not required in this configuration.
Appliance Port Failover
Failover cannot be configured in the UCS side for appliance ports. By design, UCS fabric interconnects operate as two independent fabrics. Failover must be configured on the storage side and must be implemented with the correct network design while specific failover behavior for storage controllers are kept in mind, dependent on the storage controller model.
Network Uplink Failure
With default behavior, appliance ports are shut down if the uplink that is pinned to it goes down.
In order to change this behavior, configure a Network Control Policy and set the action to be taken upon uplink failure to Warning. For more information, refer to the Network Control Policy section of the Cisco UCS Manager GUI Configuration Guide, Release 2.2.
Appliance Port Troubleshooting
In the majority of implementations, communication between the blade servers and storage controllers that are plugged into the appliance ports are over the same broadcast domain (Layer 2 of the Open Systems Interconnect (OSI) model). In order to verify that this Layer 2 communication works correctly, you must verify whether the UCS Fabric Interconnect has learned the MAC address of the storage controller on the appliance port and on the correct VLAN.
In order to check the MAC address table, log into the UCS CLI console, connect to the NX-OS shell,and verify the list of allowed VLANs on the appliance port. You can then view the MAC address entries for the VLAN that are to be used for communication with the appliance port.
Here are the commands and output for verification:
F340-31-14-UCS-2-A# connect nxos a
F340-31-14-UCS-2-A(nxos)# show run int eth 1/11
!! Command: show running-config interface Ethernet1/11
!! Time: Fri Mar 29 07:02:29 2013
version 5.0(3)N2(2.11b)
interface Ethernet1/11
description A: Appliance
no pinning server sticky
pinning server pinning-failure link-down
no cdp enable
switchport mode trunk
switchport trunk allowed vlan 170
F340-31-14-UCS-2-A(nxos)# show mac address-table vlan 170
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
* 170 0025.b500.004f static 0 F F Veth780
* 170 0025.b500.005f static 0 F F Veth779
* 170 010a.84ff.e4fe dynamic 0 F F Eth1/11
In the this output, there are two blade servers, Veth780 and Veth779, and the MAC address of the storage controller is learned on Eth1/11. These devices should be able to communicate with each other if there are no other configuration issues on the end devices.
If no MAC addresses are learned on the appliance port while the correct VLAN is specified, then return to the appliance port configuration and re-confirm the trunk configuration. Also, ensure that the communication link on the storage appliance is in Active mode in the case of an Active/Passive link configuration. You can also check the MAC address table on fabric interconnect B, dependent on the link that is active on the storage controller side.
When the MAC addresses of the server and the storage controller are learned on the fabric interconnect within the same VLAN, the fabric interconnect switches the traffic locally without the use of the upstream switches. At this juncture, you can use an Internet Control Message Protocol (ICMP) request (ping) in order to test the communication between the end points.
Related Information
Contributed by Cisco Engineers
- Dmitri FilenkoCisco TAC Engineer
- Andreas NikasCisco TAC Engineer
FeedbackContact Cisco
- Open a Support Case
- (Requires a Cisco Service Contract)