Quality of Service

Quality of Service

Cisco UCS provides the following methods to implement quality of service:

  • System classes that specify the global configuration for certain types of traffic across the entire system

  • QoS policies that assign system classes for individual vNICs

  • Flow control policies that determine how uplink Ethernet ports handle pause frames

Global QoS changes made to the QoS system class may result in brief data-plane interruptions for all traffic. Some examples of such changes are:

  • Changing the MTU size for an enabled class

  • Changing packet drop for an enabled class

  • Changing the CoS value for an enabled class

Guidelines and Limitations for Quality of Service on Cisco UCS 6454 Fabric Interconnects

  • Multicast optimization is not supported.

  • MTU is not configurable for drop type QoS system classes and is always set to 9216. MTU is only configurable for no-drop type QoS system classes, except for the fibre channel class.

  • The default MTU size for the no-drop class is 1500 and the maximum supported size for this class is 9216.

  • The MTU size for fibre channel is always 2240.

Guidelines and Limitations for Quality of Service on Cisco UCS 6300 Series Fabric Interconnect

  • Cisco UCS 6300 Series Fabric Interconnect uses a shared buffer for all system classes.

  • Multicast optimization is not supported.

  • When you change the QoS parameters for any class causes traffic disruption to all classes. The following table lists the changes in the QoS system class and the conditions that trigger a system reboot.

    QoS System class status

    Condition

    FI Reboot Status

    Enabled

    Change between drop and no drop

    Yes

    No-drop

    Change between enable and disable

    Yes

    Enable and no-drop

    Change in MTU size

    Yes

  • The subordinate FI reboots first as a result of the change in the QoS system class. The primary FI reboots only after you acknowledge it in Pending Activities.

Guidelines and Limitations for Quality of Service on Cisco UCS Mini

  • Cisco UCS Mini uses a shared buffer for all system classes.

  • The bronze class shares the buffer with SPAN. We recommend using either SPAN or the bronze class.

  • Multicast optimization is not supported.

  • Changing the QoS parameters for any class causes traffic disruption to all classes.

  • When mixing Ethernet and FC or FCoE traffic, the bandwidth distribution is not equal.

  • Multiple streams of traffic from the same class may not be distributed equally.

  • Use the same CoS values for all no-drop policies to avoid any FC or FCoE performance issues.

  • Only the platinum and gold classes support no-drop policies.

Configuring System Classes

System Classes

Cisco UCS uses Data Center Ethernet (DCE) to handle all traffic inside a Cisco UCS domain. This industry standard enhancement to Ethernet divides the bandwidth of the Ethernet pipe into eight virtual lanes. Two virtual lanes are reserved for internal system and management traffic. You can configure quality of service (QoS) for the other six virtual lanes. System classes determine how the DCE bandwidth in these six virtual lanes is allocated across the entire Cisco UCS domain.

Each system class reserves a specific segment of the bandwidth for a specific type of traffic, which provides a level of traffic management, even in an oversubscribed system. For example, you can configure the Fibre Channel Priority system class to determine the percentage of DCE bandwidth allocated to FCoE traffic.

The following table describes the system classes that you can configure.

Table 1. System Classes

System Class

Description

Platinum

Gold

Silver

Bronze

A configurable set of system classes that you can include in the QoS policy for a service profile. Each system class manages one lane of traffic.

All properties of these system classes are available for you to assign custom settings and policies.

For Cisco UCS Mini, packet drop can only be disabled on the platinum and gold classes. Only one platinum and one gold class can be configured as a no drop class at a time.

Best Effort

A system class that sets the quality of service for the lane reserved for basic Ethernet traffic.

Some properties of this system class are preset and cannot be modified. For example, this class has a drop policy that allows it to drop data packets if required. You cannot disable this system class.

Fibre Channel

A system class that sets the quality of service for the lane reserved for Fibre Channel over Ethernet traffic.

Some properties of this system class are preset and cannot be modified. For example, this class has a no-drop policy that ensures it never drops data packets. You cannot disable this system class.

Note 

FCoE traffic has a reserved QoS system class that should not be used by any other type of traffic. If any other type of traffic has a CoS value that is used by FCoE, the value is remarked to 0.

Configuring QoS System Classes

The type of adapter in a server might limit the maximum MTU supported. For example, network MTU above the maximums might cause the packet to drop for the following adapter:

  • The Cisco UCS 82598KR-CI adapter that supports a maximum MTU of 140009.


Note

Under the network QoS policy, the MTU is used only for buffer carving when no-drop classes are configured. No additional MTU adjustments are required under the network QoS policy to support jumbo MTU.


Note

For VIC 14xx adapters, you can change the MTU size of the vNIC from the host interface settings. When the Overlay network is configured, make sure that the overall MTU size does not exceed the MTU value in the QoS system class. If this MTU value exceeds the MTU value in the QoS system class, packets could be dropped during data transmission.


Important

Use the same CoS (Class of Service) values on UCS and N5K for all the no-drop policies. To insure that end-to-end PFC works correctly, have the same QoS policy configured on all intermediate switches.

Procedure

Step 1

In the Navigation pane, click LAN.

Step 2

Expand LAN > LAN Cloud.

Step 3

Select the QoS System Class node. Packet drop should be unchecked to configure MTU.

MTU is not configurable for drop type QoS system classes and is always set to 9216. MTU is only configurable for no-drop type QoS system classes

Step 4

In the Work pane, click the General tab.

Step 5

Update the properties for the system class that you want to configure to meet the traffic management needs of the system.

Note 

Some properties may not be configurable for all system classes. The maximum value for MTU is 9216.

Step 6

Click Save Changes.

Enabling a QoS System Class

The Best Effort or Fibre Channel system classes are enabled by default.

Procedure

Step 1

In the Navigation pane, click LAN.

Step 2

Expand LAN > LAN Cloud.

Step 3

Select the QoS System Class node.

Step 4

In the Work pane, click the General tab.

Step 5

Check the Enabled check box for the QoS system that you want to enable.

Step 6

Click Save Changes.

Disabling a QoS System Class

You cannot disable the Best Effort or Fibre Channel system classes.

All QoS policies that are associated with a disabled system class default to Best Effort or, if the disabled system class is configured with a Cos of 0, to the Cos 0 system class.

Procedure

Step 1

In the Navigation pane, click LAN.

Step 2

Expand LAN > LAN Cloud.

Step 3

Select the QoS System Class node.

Step 4

In the Work pane, click the General tab.

Step 5

Uncheck the Enabled check box for the QoS system that you want to disable.

Step 6

Click Save Changes.

Configuring Quality of Service Policies

Quality of Service Policy

A quality of service (QoS) policy assigns a system class to the outgoing traffic for a vNIC or vHBA. This system class determines the quality of service for that traffic. For certain adapters, you can also specify additional controls on the outgoing traffic, such as burst and rate.

You must include a QoS policy in a vNIC policy or vHBA policy and then include that policy in a service profile to configure the vNIC or vHBA.

Creating a QoS Policy

Procedure

Step 1

In the Navigation pane, click LAN.

Step 2

Expand LAN > Policies.

Step 3

Expand the node for the organization where you want to create the pool.

If the system does not include multi tenancy, expand the root node.

Step 4

Right-click QoS Policy and select Create QoS Policy.

Step 5

In the Create QoS Policy dialog box, complete the required fields.

Step 6

Click OK.

What to do next

Include the QoS policy in a vNIC or vHBA template.

Deleting a QoS Policy

If you delete a QoS policy that is in use or you disable a system class that is used in a QoS policy, any vNIC or vHBA that uses that QoS policy is assigned to the Best Effort system class or to the system class with a CoS of 0. In a system that implements multitenancy, Cisco UCS Manager first attempts to find a matching QoS policy in the organization hierarchy.

Procedure

Step 1

In the Navigation pane, click LAN.

Step 2

Expand Servers > Policies > Organization_Name.

Step 3

Expand the QoS Policies node.

Step 4

Right-click the QoS policy you want to delete and select Delete.

Step 5

If a confirmation dialog box displays, click Yes.

Configuring Flow Control Policies

Flow Control Policy

Flow control policies determine whether the uplink Ethernet ports in a Cisco UCS domain send and receive IEEE 802.3x pause frames when the receive buffer for a port fills. These pause frames request that the transmitting port stop sending data for a few milliseconds until the buffer clears.

For flow control to work between a LAN port and an uplink Ethernet port, you must enable the corresponding receive and send flow control parameters for both ports. For Cisco UCS, the flow control policies configure these parameters.

When you enable the send function, the uplink Ethernet port sends a pause request to the network port if the incoming packet rate becomes too high. The pause remains in effect for a few milliseconds before traffic is reset to normal levels. If you enable the receive function, the uplink Ethernet port honors all pause requests from the network port. All traffic is halted on that uplink port until the network port cancels the pause request.

Because you assign the flow control policy to the port, changes to the policy have an immediate effect on how the port reacts to a pause frame or a full receive buffer.

Creating a Flow Control Policy

Before you begin

Configure the network port with the corresponding setting for the flow control that you need. For example, if you enable the send setting for flow-control pause frames in the policy, ensure that the receive parameter in the network port is set to on or to desired. If you want the Cisco UCS port to receive flow-control frames, ensure that the send parameter is set to on or to desire on the network port. If you do not want to use flow control, you can set the send and receive parameters on the network port to off.

Procedure

Step 1

In the Navigation pane, click LAN.

Step 2

Expand LAN > Policies.

Step 3

Expand the root node.

You can only create a flow control policy in the root organization. You cannot create a flow control policy in a sub-organization.

Step 4

Right-click the Flow Control Policies node and select Create Flow Control Policy.

Step 5

In the Create Flow Control Policy wizard, complete the required fields.

Step 6

Click OK.

What to do next

Associate the flow control policy with an uplink Ethernet port or port channel.

Deleting a Flow Control Policy

Procedure

Step 1

In the Navigation pane, click LAN.

Step 2

Expand LAN > Policies > Organization_Name.

Step 3

Expand the Flow Control Policies node.

Step 4

Right-click the policy you want to delete and select Delete.

Step 5

If a confirmation dialog box displays, click Yes.

Configuring Slow Drain

QoS Slow Drain Device Detection and Mitigation

All data traffic between end devices in the fabric is carried by Fibre Channel services that use link-level, per-hop-based, and buffer-to-buffer flow control. These classes of service do not support end-to-end flow control. When slow devices are attached to the fabric, the end devices do not accept the frames at the configured or negotiated rate. The slow devices lead to an Inter-Switch Link (ISL) credit shortage in the traffic that is destined for these devices, and they congest the links. The credit shortage affects the unrelated flows in the fabric that use the same ISL link even though destination devices do not experience a slow drain.

Similarly, in End-Host Mode, if a server that is directly attached to the Fabric Interconnect receives traffic slowly, it may congest the uplink port shared by other servers. If a slow server is attached to a HIF port on FEX/IOM, it may congest the fabric port and/or uplink port.

Cisco UCS Manager Release 4.0(2) introduces the QoS Slow Drain Detection and Mitigation feature on Cisco UCS 6454 Fabric Interconnects. This feature provides various enhancements that enable you to detect slow drain devices that cause congestion in the network, and also mitigate it. The enhancements are mainly on the edge ports and core ports that connect to the slow drain devices. This is done to minimize the frames stuck condition in the edge and core ports due to slow drain devices that are causing an ISL blockage. To avoid or minimize the stuck condition, you can configure smaller frame timeout for the ports. A smaller frame timeout value helps to alleviate the slow drain condition that affects the fabric by dropping the packets on the edge ports sooner than the time they actually get timed out. This function frees the buffer space in ISL, which can be used by other unrelated flows that do not experience the slow drain condition.

In this release, slow drain detection and mitigation is supported on the following ports:

  • FCoE

  • Back-plane

Configuring Slow Drain

While configuring slow drain timeout timers, you can select the timeout value from the list of allowed values. You cannot configure custom timeout values.


Note

  • We recommend to change the default timeout values of the Core FCoE port and Edge FCoE Port only when the current default timeout values result in the dropping of packets during high traffic load.

  • Slow drain and the watchdog timer cannot be used simultaneously. Attempting to do so will result in an error.

Procedure

Step 1

In the Navigation pane, click LAN.

Step 2

Expand LAN > LAN Cloud.

Step 3

In the Work pane, click the QoS tab.

Step 4

Click Configure Slow Drain.

Step 5

In the Configure Slow Drain Timers dialog box that appears, configure the following fields:

Name

Description

FCoE Port radio button

Whether slow drain timers are enabled on FCoE ports.

  • Disabled—Configuration of slow drain timers is disabled. This is the default option till the previous version of Cisco UCS Manager Release 4.1(3a).

  • Enabled—Configuration of slow drain timers is enabled. Starting with Cisco UCS Manager Release 4.1(3a), this is the default option.

Core FCoE port (ms) drop-down list

The time in milliseconds (ms) after which frames timeout on core FCoE ports. You can select from the following values:

  • 100

  • 200

  • 300

  • 400

  • 500—This is the default value

  • 600

  • 700

  • 800

  • 900

  • 1000

Edge FCoE Port (ms) drop-down list

The time in milliseconds (ms) after which frames timeout on edge FCoE ports. You can select from the following values:

  • 100

  • 200

  • 300

  • 400

  • 500—This is the default value

  • 600

  • 700

  • 800

  • 900

  • 1000
Step 6

Click OK.

Step 7

Click Save Changes.

Correcting a Slow Drain Condition

Correcting a slow drain condition will work only on those ports that are designated to be in the ‘error-disabled’ state because of ‘slow-drain’.

Procedure

Step 1

In the Navigation pane, click Equipment.

Step 2

Expand Equipment > Chassis > Chassis Number > IO Modules.

Step 3

Choose the I/O module on which you want to recover backplane ports that are in the error-disabled state.

Step 4

In the Work pane, click the General tab.

Step 5

In the Actions area, click Correct Slow Drain Condition.

Step 6

If a confirmation dialog box displays, click Yes.