Guest

Cisco 3800 Series Integrated Services Routers

NM-1M-OC3-POM Watermark CLI

Document ID: 71882

Updated: Nov 01, 2006

   Print

Introduction

This document discusses the use of the queue-depth and queue-depth low-latency commands on the NM-1A-OC3-POM network module on the Cisco 3800 platforms to reduce or increase latency out of an ATM Permanent Virtual Circuit (PVC). The queue-depth command is introduced in Cisco IOS® Software Release 12.4(7.24)T and later. The latency issue arises when there is a low bandwidth and a burst in traffic occurs during a surge. Refer to Cisco bug ID CSCsd73749 (registered customers only) and Cisco bug ID CSCsj97952 (registered customers only) for more information on the two different types of latencies can occur in a customer scenario.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

The information in this document is based on the NM-1A-OC3-POM on the Cisco 3800 platforms with Cisco IOS Software Release 12.4(7.24)T and later.

Conventions

Refer to Cisco Technical Tips Conventions for more information on document conventions.

Background Information

On ATM line cards, the Segmentation and Reassembly (SAR) mechanism has a queue for each PVC. These two thresholds are associated with each PVC queue:

  • high watermark

  • low watermark

The high watermark defines the number of cells that the queue can hold. The watermark values are used to apply a flow control mechanism between the host and the SAR on the NM-1A-OC3-POM network module. When cells start to back up in the SAR, the SAR sends a notification to the host as soon as the queue inside the SAR builds up to a high watermark. At this point, the VC is marked as throttled and packets start to back up in the Cisco IOS software hold queues. At the same time, the SAR drains out the packets. When the SAR reaches the low watermark, another notification is sent to the host. The VC is marked as "Open" and traffic to the VC resumes. The problem is caused by the low values that are configured for the high and low watermarks on the SAR.

Problem

Traffic that is processed by PVCs with bandwidth values lower than 10 Mbps on an NM-1M-OC3-POM network module might encounter large latencies. In such cases packets might be dropped from the output queue.

Solution

Applications of the queue-depth Command

When you want to better control priority queuing latency or have better TCP performance, modify the watermark values for each ATM variable bit rate (VBR) VC by using the queue-depth command. If you need to change the watermark values, follow these guidelines:

  • A higher value of high watermark translates to a higher queue build-up within the SAR and results in a higher latency for latency sensitive (LLQ) type traffic.

  • Once the packets are queued in the SAR, they are all treated the same.

  • Higher queues inside the SAR give IOS less chance to load latency sensitive traffic to the SAR. This increases overall latency experienced by latency sensitive traffic. Hence, in the case of LLQ, higher values of high watermark are not desirable. However if the high watermark value is too low, you sometimes end up in situations where each incoming packet causes high and low watermarks to be hit which causes the VC to toggle between the Open and Throttled states and too frequently gives rise to large latencies (Cisco bug ID CSCsd73749 (registered customers only) ). See the Example Scenario section for more information.

Setting the CLI - Commands and Parameter Suggestions

Do not configure the low watermark value to be equal to the high watermark value because this defeats the purpose of the flow control mechanism. Even though the queue-depth command allows a high watermark value up to 65535, it is not recommend that you configure such a high watermark value. A high watermark value translates into queues within the SAR. How high the value of the high watermark can be is defined by the SAR memory. For example, with 1024 VCs, when the high watermark is configured for more than 400 cells, the SAR might run out of memory. This causes packet drops to occur. As a rough guideline, default values of high and low watermarks for PVCs with a bandwidth of less than 1 Mbps are 50 and 10. Latency / drop issues can occur with these values. However, when you multiply these values by a factor of 4 via the queue-depth command such that the new values are 200 and 40, the symptom no longer occurs.

Router#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
Router(config)#interface atm 1/0
Router(config-if)#pvc 1/1
Router(config-if-atm-vc)#queue-depth ?
  <1-65535> queue depth high watermark, in cells

Router(config-if-atm-vc)#queue-depth 200 ?
  <1-200> queue depth low watermark, in cells

Router(config-if-atm-vc)#queue-depth 200 100 ?
  <cr>

Router(config-if-atm-vc)#queue-depth 200 100 
Router(config-if-atm-vc)#end
Router#

Example Scenario

Before You use the queue-depth Command

This command output shows the default behavior. In this case the watermarks are 50/10 for a PVC with PCR=1MEG.

Router(config)#interface atm 1/0.1 point-to-point 
Router(config-subif)#ip address 10.10.11.1 255.255.255.0
Router(config-subif)#pvc 1/2
Router(config-if-atm-vc)#cbr 1000
Router(config-if-atm-vc)#protocol ip 10.10.11.2 broadcast 
Router(config-if-atm-vc)#end
Router#
*Apr  1 19:48:56.551: ATM1/0: Setup_VC: vc:3 vpi:1 vci:2
*Apr  1 19:48:56.551: ATM1/0: Open_Channel(RSY): CH (1), VPI (1), VCI (2)
*Apr  1 19:48:56.555: ATM1/0: HI/LO watermarks: 50/10; PeakRate: 1000
*Apr  1 19:48:56.555: ATM1/0: Open_Channel(SEG): CH (1), VPI (1), VCI (2)
*Apr  1 19:48:56.555: ATM1/0: Setup_Cos: vc:3 wred_name:- max_q:0
*Apr  1 19:48:56.555: %SYS-5-CONFIG_I: Configured from console by console
Router#
Router#ping 10.10.11.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.11.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

This output shows an attempt to ping with large packets and a default timeout.

Router#ping
Protocol [ip]: 
Target IP address: 10.10.11.2
Repeat count [5]: 
Datagram size [100]: 18000
Timeout in seconds [2]: 
Extended commands [n]: 
Sweep range of sizes [n]: 
Type escape sequence to abort.
Sending 5, 18000-byte ICMP Echos to 10.10.11.2, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

This output shows a ping to large packets after you increase the default timeout to 10 seconds.

Router#ping
Protocol [ip]: 
Target IP address: 10.10.11.2
Repeat count [5]: 
Datagram size [100]: 18000
Timeout in seconds [2]: 10
Extended commands [n]: 
Sweep range of sizes [n]: 
Type escape sequence to abort.
Sending 5, 18000-byte ICMP Echos to 10.10.11.2, timeout is 10 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 2976/2995/3000 ms

You can see from the output of this ping command that the round trip time taken for the ping is almost three seconds.

After the queue-depth Command is used to Change the Watermarks to 200/40

Router#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
Router(config)#interface atm 1/0.1
Router(config-subif)#pvc 1/2
Router(config-if-atm-vc)#queue-depth 200 40
Router(config-if-atm-vc)#end
Router#
*Apr  1 19:51:22.403: ATM1/0: Sent pending EOP successfully
*Apr  1 19:51:22.403: ATM1/0: Close_Channel(RSY): Chan_ID (0x84)
*Apr  1 19:51:22.403: ATM1/0: Close_Channel(RSY): Chan_ID (0x84) CLOSE
*Apr  1 19:51:22.403: ATM1/0: Close_Channel: CLOSE_PENDING
*Apr  1 19:51:22.403: ATM1/0: Close_Channel(SEG): Chan_ID (0x85)
*Apr  1 19:51:22.403: ATM1/0: Close_Channel: CLOSE
*Apr  1 19:51:22.403: ATM1/0: Setup_VC: vc:3 vpi:1 vci:2
*Apr  1 19:51:22.403: ATM1/0: Open_Channel(RSY): CH (1), VPI (1), VCI (2)
*Apr  1 19:51:22.403: ATM1/0: HI/LO watermarks: 200/40; PeakRate: 1000
*Apr  1 19:51:22.403: ATM1/0: Open_Channel(SEG): CH (1), VPI (1), VCI (2)
*Apr  1 19:51:22.403: ATM1/0: Setup_Cos: vc:3 wred_name:- max_q:0
*Apr  1 19:51:22.403: %SYS-5-CONFIG_I: Configured from console by console
Router#ping 10.10.11.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.11.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms
Router#ping
Protocol [ip]: 
Target IP address: 10.10.11.2
Repeat count [5]: 
Datagram size [100]: 18000
Timeout in seconds [2]: 
Extended commands [n]: 
Sweep range of sizes [n]: 
Type escape sequence to abort.
Sending 5, 18000-byte ICMP Echos to 10.10.11.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 324/324/324 ms

You see from this output that the round trip time has now reduced to 300 ms.

Problem

When a large file of size greater than 60 MB is copied with the use of the Windows file transfer (PC-to-PC drag and drop / best- effort traffic), the priority class traffic gets delayed and experiences high latency values. At the maximum, the latency can reach 100 ms with average hovering around 60 ms. Refer to Cisco bug ID CSCsj97952 (registered customers only) for more information

Solution

Use The CLI - queue-depth low-latency

A new shaping mechanism is introduced in the driver level in order to fix this issue. Each VC is given a credit in bytes and whenever a packet is sent the credit is decremented. The credit is replenished every 16ms and the credit value is set as the number of bytes that can be transmitted in 25ms (scr *25/8). The value 25ms is arrived after the testing of various PCR values and credit values. A new CLI queue-depth low-latency is introduced in order to enable this feature. This is only available on CBR and VBR class VCs and the bandwidth should not be greater than 10000kbps (10GB).

Note: When queue-depth low-latency is configured, the queue-depthsis set to 50 and 10. Even if the user configures other values it does not take effective. Once the user removes the queue-depth low-latency command the previous configured values are set. If user has not configured any values, the default values are set.

Frequently Asked Questions

Is there ever a need to set the watermark if the PVC is larger than 10 Mbps?

No.

How do I verify my watermark configuration?

See the Example Scenario section in this document.

How do I verify that the queue-depth command is actually the cause of drops, as against a valid oversubscription on the ATM PVC?

If the traffic carried consists of very large packets or is bursty, this problem is more likely to happen. If the problem persists even after you increase the high and low watermarks, then it is probably due to oversubscription.

Related Information

Updated: Nov 01, 2006
Document ID: 71882