Configuring Quality of Service

This chapter includes the following sections:

Quality of Service

Cisco UCS provides the following methods to implement quality of service:

  • System classes that specify the global configuration for certain types of traffic across the entire system

  • QoS policies that assign system classes for individual vNICs

  • Flow control policies that determine how uplink Ethernet ports handle pause frames

Global QoS changes made to the QoS system class may result in brief data-plane interruptions for all traffic. Some examples of such changes are:

  • Changing the MTU size for an enabled class

  • Changing packet drop for an enabled class

  • Changing the CoS value for an enabled class

Guidelines and Limitations for Quality of Service on Cisco UCS 6300 Series Fabric Interconnect

  • Cisco UCS 6300 Series Fabric Interconnect uses a shared buffer for all system classes.

  • Multicast optimization is not supported.

  • When you change the QoS parameters for any class causes traffic disruption to all classes. The following table lists the changes in the QoS system class and the conditions that trigger a system reboot.

    QoS System class status

    Condition

    FI Reboot Status

    Enabled

    Change between drop and no drop

    Yes

    No-drop

    Change between enable and disable

    Yes

    Enable and no-drop

    Change in MTU size

    Yes

  • The subordinate FI reboots first as a result of the change in the QoS system class. The primary FI reboots only after you acknowledge it in Pending Activities.

Guidelines and Limitations for Quality of Service on Cisco UCS Mini

  • Cisco UCS Mini uses a shared buffer for all system classes.

  • The bronze class shares the buffer with SPAN. We recommend using either SPAN or the bronze class.

  • Multicast optimization is not supported.

  • Changing the QoS parameters for any class causes traffic disruption to all classes.

  • When mixing Ethernet and FC or FCoE traffic, the bandwidth distribution is not equal.

  • Multiple streams of traffic from the same class may not be distributed equally.

  • Use the same CoS values for all no-drop policies to avoid any FC or FCoE performance issues.

  • Only the platinum and gold classes support no-drop policies.

Configuring System Classes

System Classes

Cisco UCS uses Data Center Ethernet (DCE) to handle all traffic inside a Cisco UCS domain. This industry standard enhancement to Ethernet divides the bandwidth of the Ethernet pipe into eight virtual lanes. Two virtual lanes are reserved for internal system and management traffic. You can configure quality of service (QoS) for the other six virtual lanes. System classes determine how the DCE bandwidth in these six virtual lanes is allocated across the entire Cisco UCS domain.

Each system class reserves a specific segment of the bandwidth for a specific type of traffic, which provides a level of traffic management, even in an oversubscribed system. For example, you can configure the Fibre Channel Priority system class to determine the percentage of DCE bandwidth allocated to FCoE traffic.

The following table describes the system classes that you can configure.

Table 1 System Classes

System Class

Description

Platinum

Gold

Silver

Bronze

A configurable set of system classes that you can include in the QoS policy for a service profile. Each system class manages one lane of traffic.

All properties of these system classes are available for you to assign custom settings and policies.

For Cisco UCS Mini, packet drop can only be disabled on the platinum and gold classes. Only one platinum and one gold class can be configured as a no drop class at a time.

Best Effort

A system class that sets the quality of service for the lane reserved for basic Ethernet traffic.

Some properties of this system class are preset and cannot be modified. For example, this class has a drop policy that allows it to drop data packets if required. You cannot disable this system class.

Fibre Channel

A system class that sets the quality of service for the lane reserved for Fibre Channel over Ethernet traffic.

Some properties of this system class are preset and cannot be modified. For example, this class has a no-drop policy that ensures it never drops data packets. You cannot disable this system class.

Note   

FCoE traffic has a reserved QoS system class that should not be used by any other type of traffic. If any other type of traffic has a CoS value that is used by FCoE, the value is remarked to 0.

Configuring a System Class

The type of adapter in a server might limit the maximum MTU supported. For example, network MTU above the maximums might cause the packet to be dropped for the following adapters:

  • The Cisco UCS M71KR CNA adapter, which supports a maximum MTU of 9216.

  • The Cisco UCS 82598KR-CI adapter, which supports a maximum MTU of 14000.

Procedure
     Command or ActionPurpose
    Step 1UCS-A# scope eth-server  

    Enters Ethernet server mode.

     
    Step 2UCS-A /eth-server # scope qos  

    Enters Ethernet server QoS mode.

     
    Step 3UCS-A /eth-server/qos # scope eth-classified {bronze | gold | platinum | silver}  

    Enters Ethernet server QoS Ethernet classified mode for the specified system class.

     
    Step 4UCS-A /eth-server/qos/eth-classified # enable  

    Enables the specified system class.

     
    Step 5UCS-A /eth-server/qos/eth-classified # set cos cos-value  

    Specifies the class of service for the specified system class. Valid class of service values are 0 to 6.

    Important:

    Use the same CoS values on UCS and N5K for all the no-drop policies. To insure that end-to-end PFC works correctly, have the same QoS policy configured on all intermediate switches.

     
    Step 6UCS-A /eth-server/qos/eth-classified # set drop {drop | no-drop}  

    Specifies whether the channel can drop packets or not.

    Note   

    Changes saved to the drop displays the following warning message: Warning: The operation will cause momentary disruption to traffic forwarding.

     
    Step 7UCS-A /eth-server/qos/eth-classified # set mtu {mtu-value | fc | normal}   The maximum transmission unit, or packet size to be used. The maximum value for MTU is 9216.
    Note   

    If the vNIC has an associated QoS policy, the MTU specified here must be equal to or less than the MTU specified in the associated QoS system class. If this MTU value exceeds the MTU value in the QoS system class, packets might get dropped during data transmission.

    Changes saved to the MTU displays the following warning message: Warning: The operation will cause momentary disruption to traffic forwarding.

     
    Step 8UCS-A /eth-server/qos/eth-classified # set multicast-optimize {no | yes}  

    Specifies whether the class is optimized to for sending multicast packets.

     
    Step 9UCS-A /eth-server/qos/eth-classified # set weight {weight-value | best-effort | none}  

    Specifies the relative weight for the specified system class. Valid weight values are 0 to 10.

     
    Step 10UCS-A /eth-server/qos/eth-classified # commit-buffer  

    Commits the transaction to the system configuration.

     

    The following example shows how to enable the platinum system class, allow the channel to drop packets, set the class of service to 6, set the MTU to normal, set the relative weight to 5, and commit the transaction: 

    UCS-A# scope eth-server
UCS-A /eth-server # scope qos
UCS-A /eth-server/qos # scope eth-classified platinum
UCS-A /eth-server/qos/eth-classified # enable
UCS-A /eth-server/qos/eth-classified* # set drop drop
Warning: The operation will cause momentary disruption to traffic forwarding
UCS-A /eth-server/qos/eth-classified* # set cos 6
UCS-A /eth-server/qos/eth-classified* # set mtu normal
Warning: The operation will cause momentary disruption to traffic forwarding
UCS-A /eth-server/qos/eth-classified* # set weight 5
UCS-A /eth-server/qos/eth-classified* # commit-buffer
UCS-A /eth-server/qos/eth-classified #

    Disabling a System Class

    If you disable a system class that is used in a QoS policy, Cisco UCS Manager uses the system class configured with CoS 0 for traffic on servers that are configured with the QoS policy. If no system class is configured as CoS 0, the Best Effort system class is used. You cannot disable the Best Effort or Fibre Channel system classes.

    Procedure
       Command or ActionPurpose
      Step 1UCS-A# scope eth-server  

      Enters Ethernet server mode.

       
      Step 2UCS-A /eth-server # scope qos  

      Enters Ethernet server QoS mode.

       
      Step 3UCS-A /eth-server/qos # scope eth-classified {bronze | gold | platinum | silver}  

      Enters Ethernet server QoS Ethernet classified mode for the specified system class.

       
      Step 4UCS-A /eth-server/qos/eth-classified # disable  

      Disables the specified system class.

       
      Step 5UCS-A /eth-server/qos/eth-classified # commit-buffer  

      Commits the transaction to the system configuration.

       

      The following example disables the platinum system class and commits the transaction: 

      UCS-A# scope eth-server
UCS-A /eth-server # scope qos
UCS-A /eth-server/qos # scope eth-classified platinum
UCS-A /eth-server/qos/eth-classified # disable
UCS-A /eth-server/qos/eth-classified* # commit-buffer
UCS-A /eth-server/qos/eth-classified #

      Configuring Quality of Service Policies

      Quality of Service Policy

      A quality of service (QoS) policy assigns a system class to the outgoing traffic for a vNIC or vHBA. This system class determines the quality of service for that traffic. For certain adapters, you can also specify additional controls on the outgoing traffic, such as burst and rate.

      You must include a QoS policy in a vNIC policy or vHBA policy and then include that policy in a service profile to configure the vNIC or vHBA.

      Configuring a QoS Policy

      Procedure
         Command or ActionPurpose
        Step 1Switch-A# scope org org-name  

        Enters org mode for the specified organization. To enter the default org mode, type / as the org-name .

         
        Step 2Switch-A /org # create qos-policy policy-name  

        Creates the specified QoS policy, and enters org QoS policy mode.

         
        Step 3Switch-A /org/qos-policy # create egress-policy  

        Creates the egress policy (for both vNICs and vHBAs) to be used by the QoS policy, and enters org QoS policy egress policy mode.

         
        Step 4Switch-A /org/qos-policy/egress-policy # set host-cos-control {full | none}  

        (Optional) Specifies whether the host or Cisco UCS Manager controls the class of service (CoS) for a vNIC. This setting has no effect on a vHBA.

        Use the full keyword to have the host control the CoS. If the packet has a valid CoS value, the host uses that value. Otherwise, it uses the CoS value associated with the specified class priority. Use the none keyword to have Cisco UCS Manager use the CoS value associated with the specified priority.

         
        Step 5Switch-A /org/qos-policy/egress-policy # set prio sys-class-name  

        Specifies the system class to be used for the egress policy. The sys-class-name argument can be one of the following class keywords:

        • Fc—Use this priority for QoS policies that control vHBA traffic only.

        • Platinum—Use this priority for QoS policies that control vNIC traffic only.

        • Gold—Use this priority for QoS policies that control vNIC traffic only.

        • Silver—Use this priority for QoS policies that control vNIC traffic only.

        • Bronze—Use this priority for QoS policies that control vNIC traffic only.

        • Best Effort—Do not use this priority. It is reserved for the Basic Ethernet traffic lane. If you assign this priority to a QoS policy and configure another system class as CoS 0, Cisco UCS Manager does not default to this system class. It defaults to the priority with CoS 0 for that traffic.

         
        Step 6Switch-A /org/qos-policy/egress-policy # set rate {line-rate | kbps} burst bytes  

        Specifies the expected average rate of traffic. Traffic that falls under this rate will always conform. The default is line-rate, which equals a value of 10,000,000. The minimum value is 8, and the maximum value is 40,000,000.

        Rate limiting is supported only on vNICs on the Cisco UCS VIC-1240 Virtual Interface Card and Cisco UCS VIC-1280 Virtual Interface Card. The Cisco UCS M81KR Virtual Interface Card supports rate limiting on both vNICs and vHBAs.

         
        Step 7Switch-A /org/qos-policy/egress-policy # commit-buffer  

        Commits the transaction to the system configuration.

         

        The following example creates a QoS policy for vNIC traffic, assigns the platinum system class and sets the rate limit (traffic rate and burst size) for the egress policy, and commits the transaction: 

        Switch-A# scope org /
Switch-A /org # create qos-policy VnicPolicy34
Switch-A /org/qos-policy* # create egress-policy
Switch-A /org/qos-policy/egress-policy* # set prio platinum
Switch-A /org/qos-policy/egress-policy* # set rate 5000000 burst 65000
Switch-A /org/qos-policy/egress-policy* # commit-buffer
Switch-A /org/qos-policy/egress-policy #

        The following example creates a QoS policy for vHBA traffic, assigns the fc (Fibre Channel) system class and sets the rate limit (traffic rate and burst size) for the egress policy, and commits the transaction: 

        Switch-A# scope org /
Switch-A /org # create qos-policy VhbaPolicy12
Switch-A /org/qos-policy* # create egress-policy
Switch-A /org/qos-policy/egress-policy* # set prio fc
Switch-A /org/qos-policy/egress-policy* # set rate 5000000 burst 65000
Switch-A /org/qos-policy/egress-policy* # commit-buffer
Switch-A /org/qos-policy/egress-policy #
        What to Do Next

        Include the QoS policy in a vNIC or vHBA template.

        Deleting a QoS Policy

        If you delete a QoS policy that is in use or you disable a system class that is used in a QoS policy, any vNIC or vHBA that uses that QoS policy is assigned to the Best Effort system class or to the system class with a CoS of 0. In a system that implements multitenancy, Cisco UCS Manager first attempts to find a matching QoS policy in the organization hierarchy.

        Procedure
           Command or ActionPurpose
          Step 1UCS-A# scope org org-name  

          Enters organization mode for the specified organization. To enter the root organization mode, type / as the org-name.

           
          Step 2UCS-A /org # delete qos-policy policy-name  

          Deletes the specified QoS policy.

           
          Step 3UCS-A /org # commit-buffer  

          Commits the transaction to the system configuration.

           

          The following deletes the QoS policy named QosPolicy34 and commits the transaction:

          UCS-A# scope org /
UCS-A /org # delete qos-policy QosPolicy34
UCS-A /org* # commit-buffer
UCS-A /org #

          Configuring Flow Control Policies

          Flow Control Policy

          Flow control policies determine whether the uplink Ethernet ports in a Cisco UCS domain send and receive IEEE 802.3x pause frames when the receive buffer for a port fills. These pause frames request that the transmitting port stop sending data for a few milliseconds until the buffer clears.

          For flow control to work between a LAN port and an uplink Ethernet port, you must enable the corresponding receive and send flow control parameters for both ports. For Cisco UCS, the flow control policies configure these parameters.

          When you enable the send function, the uplink Ethernet port sends a pause request to the network port if the incoming packet rate becomes too high. The pause remains in effect for a few milliseconds before traffic is reset to normal levels. If you enable the receive function, the uplink Ethernet port honors all pause requests from the network port. All traffic is halted on that uplink port until the network port cancels the pause request.

          Because you assign the flow control policy to the port, changes to the policy have an immediate effect on how the port reacts to a pause frame or a full receive buffer.

          Configuring a Flow Control Policy

          Before You Begin

          Configure the network port with the corresponding setting for the flow control that you need. For example, if you enable the send setting for flow-control pause frames in the policy, ensure that the receive parameter in the network port is set to on or to desired. If you want the Cisco UCS port to receive flow-control frames, ensure that the send parameter is set to on or to desire on the network port. If you do not want to use flow control, you can set the send and receive parameters on the network port to off.

          Procedure
             Command or ActionPurpose
            Step 1UCS-A# scope eth-uplink  

            Enters Ethernet uplink mode.

             
            Step 2UCS-A /eth-uplink # scope flow-control  

            Enters Ethernet uplink flow control mode.

             
            Step 3UCS-A /eth-uplink/flow-control # create policy policy-name  

            Creates the specified flow control policy.

             
            Step 4UCS-A /eth-uplink/flow-control/policy # set prio prio-option  

            Specifies one of the following flow control priority options:

            • auto —The Cisco UCS system and the network negotiate whether PPP will be used on this fabric interconnect.

            • on —PPP is enabled on this fabric interconnect.

             
            Step 5UCS-A /eth-uplink/flow-control/policy # set receive receive-option  

            Specifies one of the following flow control receive options:

            • off —Pause requests from the network are ignored and traffic flow continues as normal.

            • on —Pause requests are honored and all traffic is halted on that uplink port until the network cancels the pause request.

             
            Step 6UCS-A /eth-uplink/flow-control/policy # set send send-option  

            Specifies one of the following flow control send options:

            • off —Traffic on the port flows normally regardless of the packet load.

            • on —The Cisco UCS system sends a pause request to the network if the incoming packet rate becomes too high. The pause remains in effect for a few milliseconds before traffic is reset to normal levels.

             
            Step 7UCS-A /eth-uplink/flow-control/policy # commit-buffer  

            Commits the transaction to the system configuration.

             

            The following configures a flow control policy and commits the transaction: 

            UCS-A# scope eth-uplink
UCS-A /eth-uplink # scope flow-control
UCS-A /eth-uplink/flow-control # create policy FlowControlPolicy23
UCS-A /eth-uplink/flow-control/policy* # set prio auto
UCS-A /eth-uplink/flow-control/policy* # set receive on
UCS-A /eth-uplink/flow-control/policy* # set send on
UCS-A /eth-uplink/flow-control/policy* # commit-buffer
UCS-A /eth-uplink/flow-control/policy #
            What to Do Next

            Associate the flow control policy with an uplink Ethernet port or port channel.

            Deleting a Flow Control Policy

            Procedure
               Command or ActionPurpose
              Step 1UCS-A# scope eth-uplink  

              Enters Ethernet uplink mode.

               
              Step 2UCS-A /eth-uplink # scope flow-control  

              Enters Ethernet uplink flow control mode.

               
              Step 3UCS-A /eth-uplink/flow-control # delete policy policy-name  

              Deletes the specified flow control policy.

               
              Step 4UCS-A /eth-uplink/flow-control # commit-buffer  

              Commits the transaction to the system configuration.

               

              The following example deletes the flow control policy named FlowControlPolicy23 and commits the transaction:

              UCS-A# scope eth-uplink
UCS-A /eth-uplink # scope flow-control
UCS-A /eth-uplink/flow-control # delete policy FlowControlPolicy23
UCS-A /eth-uplink/flow-control* # commit-buffer
UCS-A /eth-uplink/flow-control #