Cannot pass frame size larger than 2300 bytes through switch
Although the jumbo MTU has been configured for class-default, you cannot pass a frame size larger than 2300 bytes through the Nexus 5000 switch and the Nexus 2000 FEX.
The CoS value may conflict with the existing MTU value.
CoS 7 is used internally for controlling traffic between the Nexus 5000 switch and the Nexus 2000 FEX. The MTU value for the traffic with CoS 7 is set to a fixed value. You must check that the incoming traffic is marked with CoS 7. Use any CoS value other than 7 to avoid this limitation.
MTU for “class-default” value is 1500 when jumbo MTU configured
When the configuration for the network-qos policy-map sets the class-default to jumbo MTU, the
show queuing interface
command indicates that the MTU for class-default is 1500.
An incorrect startup configuration may exist after an upgrade.
If the switch has been upgraded to the 4.2(1)N1(1) release, make sure that you have used the
command to delete the startup configuration. You can save the configuration first to another file name.
After the Nexus 5000 switch boots up with an empty configuration, reapply the original configuration. You might lose your connectivity to the Nexus 5000 if you are using Telnet or SSH. It is recommended that you use the console for this procedure.
Traffic not queued or prioritized correctly on Nexus 2148, Nexus 2232, and Nexus 2248
After configuring all three types of policy maps (QoS, Network-QoS, and Queuing), the traffic is not queued or prioritized correctly on Nexus 2148, Nexus 2232, and Nexus 2248 switches.
The Nexus 2148, Nexus 2232, and Nexus 2248 FEX can only support CoS-based traffic classification. The QoS service policy type configured under System QoS is populated from the Nexus 5000 to FEX only when all the matching criteria are match cos. If other match clauses exist, such as match dscp or match ip access-group in the QoS policy map, then the FEX does not accept the service policy. As a result, all the traffic is placed into the default queue.
Note Use the show queuing interface command to ensure that the queues have been created properly.
For the ingress traffic (from server to network) that is not marked with a CoS value, the traffic is placed into the default queue on FEX. Once the traffic is received on the Nexus 5000, it is classified based on a configured rule and are placed in the proper queue.
For the egress traffic (from Nexus 5000 to FEX, and then FEX to server), it is recommended that you mark mark the traffic with a CoS value on the Nexus 5000 so that the FEX can classify and queue the traffic properly.
The following example is a complete Nexus 5000 and Nexus 2232/Nexus 2248 configuration that classifies the traffic and configures the proper bandwidth for each type of traffic. This example applies only to the Nexus 5000 and Nexus 2248. The configuration for the Nexus 2148 is slightly different due to the fact that Nexus 2148 has only two queues for user data. The Nexus 2232/Nexus 2248 has six hardware queues for user data, which is the same as Nexus 5000.
//class-map for global qos policy-map, which will be used to create CoS-queue mapping.// class-map type qos voice-global class-map type qos critical-global class-map type qos scavenger-global class-map type qos video-signal-global //This qos policy-map will be attached under “system qos”. It will be downloaded to 2248 to create CoS to queue mapping.// policy-map type qos classify-5020-global class video-signal-global class-map type qos Control class-map type qos BulkData class-map type qos Scavenger class-map type qos Signalling class-map type qos CriticalData //This qos policy-map will be applied under all N5k and 2248 interfaces to classify all incoming traffic based on DSCP marking. Please note that even the policy-map will be applied under Nexus 2248 interfaces the traffic will be classified on N5k// policy-map type qos Classify-5020 class-map type network-qos Voice class-map type network-qos Critical class-map type network-qos Scavenger class-map type network-qos Video-Signalling //This policy-map type network-qos will be applied under “system qos” to define the MTU, marking and queue-limit(not configured here).// policy-map type network-qos NetworkQoS-5020 class type network-qos Voice class type network-qos Video-Signalling class type network-qos Scavenger class type network-qos Critical class type network-qos class-default class-map type queuing Voice class-map type queuing Critical class-map type queuing Scavenger class-map type queuing Video-Signalling //The queuing interface will be applied under “system qos” to define the priority queue and how bandwidth is shared among non-priority queues.// policy-map type queuing Queue-5020 class type queuing Scavenger class type queuing Critical class type queuing Video-Signalling class type queuing class-fcoe class type queuing class-default //The input queuing policy determines how bandwidth are shared for FEX uplink in the direction from FEX to N5k. The output queueing policy determines the bandwidth allocation for both N5k interfaces and FEX host interfaces.// service-policy type qos input classify-5020-global service-policy type network-qos NetworkQoS-5020 service-policy type queuing input Queue-5020 service-policy type queuing output Queue-5020 //Apply service-policy type qos under physical interface in order to classify traffic based on DSCP. Please note that for portchannel member the service-policy needs to be configured under interface port-channel.// service-policy type qos input Classify-5020 service-policy type qos input Classify-5020
show queuing interface
command can be used to ensure that the CoS-to-queue mapping is properly configured under the FEX interfaces. It can also be used to check the bandwidth and MTU configuration.
This same command can be used to check the QoS configuration for the Nexus 5000 interfaces.
The following is the output from the
show queuing interface
command for the Nexus 2248 interfaces when the above configrations are applied:
switch# sh queuing interface ethernet 100/1/1 Ethernet100/1/1 queuing information: Qos-group: 0 2 3 4 5 (shared) ------+---------+----------- queue qos-group cos priority bandwidth mtu -----+----------+------------+-----------+--------------+------ ------+-------------+--------------- rx drop rx mcast drop rx error tx drop ------------+-----------------+---------------+--------------- Priority-flow-control enabled: no cos qos-group rx pause tx pause masked rx pause -------+-----------+---------+---------+---------------
The Nexus 2148 has two queues in both the ingress and egress directions. One queue is mapped to the no-drop system class and another queue is mapped to the drop system class. For the ingress direction, the two queues are scheduled using WRR (Weight Round Robin). For the egress direction, the queue for the no-drop system class is the priority queue.
In order to separate traffic for the two queues, the user has to create a no-drop system class. All no-drop system classes created on the Nexus 5000 are mapped to the no-drop queue on the Nexus 2148.
command is added to the Network-QoS in order for the Nexus 2148 to place voice in the priority queue at the FEX egress direction.
policy-map type network-qos NetworkQoS-5020 class type network-qos Voice class type network-qos Video-Signalling class type network-qos Scavenger class type network-qos Critical class type network-qos class-default
The configuration classifies the incoming voice traffic based on DSCP and marks the voice traffic to CoS 5. At the Nexus 2148 egress direction, the FEX assigns voice traffic to the priority queue.
The following is example output from the
show queuing interface
command for the Nexus 2148 with the above configuration.
switch# sh queuing interface ethernet 199/1/1 Ethernet199/1/1 queuing information: Qos-group: 0 2 3 4 (shared) ---------+---------+----------- ---------+---------+----------- queue qos-group cos priority bandwidth mtu ------+------------+--------------+---------+----------+---- 3 0 2 3 4 0 1 2 3 4 6 WRR 100 9280 Buffer threshold: 271360 bytes Priority-flow-control enabled: no cos qos-group rx pause tx pause masked rx pause -------+-----------+---------+---------+---------------