Document ID: 17845
Contents
Introduction
Prerequisites
Requirements
Components Used
Conventions
Soft-VC Support
Traffic-Shaping Support: Configuring Non-UBR Virtual Circuits (VCs)
Sample Configuration for Soft-VCs Between ARMs with Traffic Shaping
Task
Step-by-Step Instructions
Caveat
Conclusion
Verify
Troubleshoot
Related Information
Introduction
The purpose of this document is to describe two new features available to the ATM Route Module (ARM) and Enhanced ARM (also known as ARM II) which can be found in the Cisco Catalyst 8540/8510 MSR as well as in the LightStream LS1010 MSR.
The two new features that were introduced via the integration of bug ID CSCdt58332 as of 12.1(7)E are:
-
Support for soft-virtual circuits (soft-VCs) on the ARM.
-
Support for Traffic Shaping (the ability to configure non-UBR Virtual Circuits (VCs) on the ARM).
Prerequisites
Requirements
Before attempting this configuration, ensure that you meet these requirements:
-
Knowledge of basic layer 3 functionality provided by the ARM. Refer to Configuring ATM Router Module Interfaces for detailed information.
-
Knowledge of configuration of soft-VCs on the Catalyst 8500 and LightStream LS1010 series. Refer to Configuring Virtual Circuits for detailed information.
-
Knowledge of basic resource management on the Catalyst 8500 and LightStream LS1010 series. Refer to Configuring Resource Management for detailed information.
Components Used
The information in this document is based on these software and hardware versions:
-
Cisco IOS® Software Release 12.1(10)E
-
A Cisco Catalyst 8540 MSR with an ARM II and a LightStream LS1010 with an ARM
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
Conventions
For more information on document conventions, refer to Cisco Technical Tips Conventions.
Soft-VC Support
As illustrated in the following example, the ability to configure a soft-VC is very desirable on the ARM:
The primary benefit of a soft-VC is that it allows devices (in this case, the routers) attached to ATM switches to be interconnected via VCs which will automatically be rerouted around failures in the ATM network.
Prior to bug CSCdt58332, VCs configured on an ARM module in the Catalyst 8540 MSR had to be Permanent Virtual Circuits (PVCs). If the destination of this VC was across the ATM cloud, we had to explicitly configure a PVC to either use link A or link B. If this link was to fail, we had no redundancy. With the soft-VC support, we simply configure the destination Network Services Access Point (NSAP) address and an SVC is dynamically created using link A or link B. If either link fails, the soft-VC is automatically reestablished using the other link.
Configuring the soft-VC on the ARM is straightforward and will be explained using the sample configuration .
Traffic-Shaping Support: Configuring Non-UBR Virtual Circuits (VCs)
The ability to configure non-unspecified bit rate (non-UBR) VCs on the ARM allows us to provide a limited form of traffic-shaping. In order to understand this limitation, one must understand how Resource Management (RM) on the ATM switch routers works. A good pointer to RM is the Configuring Resource Management configuration guide.
It is important to understand that ATM switches are primarily designed for policing, and not necessarily designed for traffic-shaping. Policing differs from traffic-shaping in that policing requires no buffering: if cells arrive at an excessive rate, they are passed, tagged or dropped but not buffered. Traffic-shaping, on the other hand, buffers cells in order to comply with a certain contract.
In order to guarantee certain amount of bandwidth, like the Sustained Cell Rate (SCR) for Variable Bit Rate (VBR) connections or the Peak Cell Rate (PCR) for Constant Bit Rate (CBR) connections, the ATM switch is equipped with a per-flow queueing feature card (FC-PFQ) that uses a Rate Scheduler (RS). CBR connections are only served by this rate scheduler and therefore are shaped. On the other hand, VBR connections, although not shaped, also use the rate scheduler to ensure the SCR. The reason for this is that traffic above SCR will be sent out using a Weighted Round Robin (WRR) strategy. Depending on how policing is configured, traffic above the contract may be dropped, but will definitely not be buffered in order to comply with the contract.
To summarize, traffic-shaping on the ARM is limited because we can only use the CBR service category. This limitation may be lifted on the Catalyst 8510 MSR and LightStream LS1010 by using the Traffic-Shaping Carrier Module (TSCAM), but it is not currently an option on the Catalyst 8540 MSR.
Sample Configuration for Soft-VCs Between ARMs with Traffic Shaping
Task
The sample configuration below illustrates how to configure shaped soft-VCs between ARMs using the following setup:
Step-by-Step Instructions
Follow the steps below to configure a shaped (PCR=128 kbps) soft-VC going from the LS1010 to the Catalyst 8540 MSR to prove that it is indeed shaped.
-
Configure a logical subinterface on each ARM. Note that this is not mandatory, as the main interface can also be used. Also, configure an IP address and a map statement on each switch. The output below shows the portion of the configurations on the Catalyst 8540 MSR and the LightStream LS1010.
Catalyst 8540 MSR:
8540# show run | begin ATM12/0/0 interface ATM12/0/0 no ip address logging event subif-link-status ! interface ATM12/0/0.1 multipoint ip address 1.1.1.1 255.255.255.252 map-group 8540_TO_1010 ! map-list 8540_TO_1010 ip 1.1.1.2 atm-vc 100 broadcast [snip]
LightStream LS1010:
1010# show run | begin ATM4/0/0 interface ATM4/0/0 no ip address ! interface ATM4/0/0.1 multipoint ip address 1.1.1.2 255.255.255.252 map-group 1010_TO_8540 ! map-list 1010_TO_8540 ip 1.1.1.1 atm-vc 100 broadcast [snip]
-
Determine the NSAP address of the Catalyst 8540 MSR. Use the show atm address command to get this information as shown below:
8540#show atm address | beg ATM12/0/0 47.0091.8100.0000.00d0.58b8.4201.4000.0c86.0000.00 ATM12/0/0 47.0091.8100.0000.00d0.58b8.4201.4000.0c86.0000.01 ATM12/0/0.1 47.0091.8100.0000.00d0.58b8.4201.4000.0c86.0010.00 ATM12/0/1 [snip]
Note that the subinterface number is reflected in the selector byte of the NSAP address.
-
Configure the soft-VC as well as a Connection Traffic Table Row (CTTR), which defines the CBR parameters:
1010#show run atm connection-traffic-table-row index 128 cbr pcr 128 interface ATM4/0/0.1 multipoint atm soft-vc 2 100 dest-address 47.0091.8100.0000.00d0.58b8.4201.4000.0c86.0000.01 2 100 pd on rx-cttr 128 tx-cttr 128 !
Note: The hyperlinks above provide a link to the explanation of the command.
Note: ARM data VCs must use VPI=2 by design.
-
Verify that the soft-VC is up and that the parameters are OK. Use the show atm vc interface atm command to get this information. The output below that appears in bold provides the relevant information.
On the Catalyst 8540 MSR:
8540#show atm vc interface atm 12/0/0.1 2 100 Interface: ATM12/0/0, Type: arm_port VPI = 2 VCI = 100 Status: UP Time-since-last-status-change: 1d22h Connection-type: SoftVC Cast-type: point-to-point Hold-priority: none Soft vc location: Destination Remote ATM address: 47.0091.8100.0000.0050.e203.0601.4000.0c82.0000.01 Remote VPI: 2 Remote VCI: 100 Soft vc call state: Active Packet-discard-option: enabled Usage-Parameter-Control (UPC): pass Wrr weight: 2 Number of OAM-configured connections: 42 OAM-configuration: disabled OAM-states: Not-applicable Cross-connect-interface: ATM1/0/3, Type: oc3suni Cross-connect-VPI = 0 Cross-connect-VCI = 37 Cross-connect-UPC: pass Cross-connect OAM-configuration: disabled Cross-connect OAM-state: Not-applicable Encapsulation: AAL5SNAP Connection Priority: Normal Threshold Group: 1, Cells queued: 0 Rx cells: 3230, Tx cells: 3230 Tx Clp0:3230, Tx Clp1: 0 Rx Clp0:3230, Rx Clp1: 0 Rx Upc Violations:2397, Rx cell drops:0 Rx pkts:110, Rx pkt drops:0 Rx connection-traffic-table-index: 2147483643 Rx service-category: CBR (Constant Bit Rate) Rx pcr-clp01: 128 Rx scr-clp01: none Rx mcr-clp01: none Rx cdvt: 1024 (from default for interface) Rx mbs: none Tx connection-traffic-table-index: 2147483643 Tx service-category: CBR (Constant Bit Rate) Tx pcr-clp01: 128 Tx scr-clp01: none Tx mcr-clp01: none Tx cdvt: none Tx mbs: none No AAL5 connection registered
On the LS1010:
1010#show atm vc interface atm 4/0/0.1 2 100 Interface: ATM4/0/0, Type: arm_port VPI = 2 VCI = 100 Status: UP Time-since-last-status-change: 1d22h Connection-type: SoftVC Cast-type: point-to-point Hold-priority: none Soft vc location: Source Remote ATM address: 47.0091.8100.0000.00d0.58b8.4201.4000.0c86.0000.01 Remote VPI: 2 Remote VCI: 100 Soft vc call state: Active Number of soft vc re-try attempts: 0 First-retry-interval: 5000 milliseconds Maximum-retry-interval: 60000 milliseconds Aggregate admin weight: 5040 TIME STAMPS: Current Slot:7 Outgoing Setup December 26 08:47:20.439 Incoming Connect December 26 08:47:20.483 Incoming Release December 26 08:51:06.267 Outgoing Setup December 26 08:51:06.271 Incoming Release December 26 08:51:06.487 Outgoing Setup December 26 08:51:11.487 Incoming Connect December 26 08:51:11.515 Packet-discard-option: enabled Usage-Parameter-Control (UPC): pass Wrr weight: 2 Number of OAM-configured connections: 0 OAM-configuration: disabled OAM-states: Not-applicable Cross-connect-interface: ATM1/0/3, Type: oc3suni Cross-connect-VPI = 0 Cross-connect-VCI = 37 Cross-connect-UPC: pass Cross-connect OAM-configuration: disabled Cross-connect OAM-state: Not-applicable Encapsulation: AAL5SNAP Connection Priority: Normal Threshold Group: 1, Cells queued: 0 Rx cells: 3230, Tx cells: 3230 Tx Clp0:3230, Tx Clp1: 0 Rx Clp0:3230, Rx Clp1: 0 Rx Upc Violations:2394, Rx cell drops:0 Rx pkts:110, Rx pkt drops:0 Rx connection-traffic-table-index: 128 Rx service-category: CBR (Constant Bit Rate) Rx pcr-clp01: 128 Rx scr-clp01: none Rx mcr-clp01: none Rx cdvt: 1024 (from default for interface) Rx mbs: none Tx connection-traffic-table-index: 128 Tx service-category: CBR (Constant Bit Rate) Tx pcr-clp01: 128 Tx scr-clp01: none Tx mcr-clp01: none Tx cdvt: none Tx mbs: none No AAL5 connection registered
-
Verify that connectivity has occurred and that the VC is well shaped. This is achieved by doing an extended ping and measuring the round-trip time as shown below:
1010#ping ip Target IP address: 1.1.1.1 Repeat count [5]: 5 Datagram size [100]: 1600 Timeout in seconds [2]: Extended commands [n]: Sweep range of sizes [n]: Type escape sequence to abort. Sending 5, 1600-byte ICMP Echos to 1.1.1.1, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 228/229/232 ms
As seen above, the round-trip time was roughly 200 ms. This makes sense since 1600 bytes is 12800 bits. Therefore, it takes about 100 ms to send the echo response and another 100 ms to get the reply. The extra ~30 ms account for processing and propagation time.
Caveat
As seen above, the actual configuration of shaped soft-VCs on the ARM is quite simple. There is, however, one important caveat that must be mentioned. Earlier, we alluded to the fact that the ATM switch was not designed for buffering. Indeed, well-behaved CBR connections should require virtually no buffers since the incoming traffic is expected to arrive at the same rate at which it was sent out of the switch. Furthermore, CBR traffic is always treated with the highest priority so that it experiences minimal delay even on congested links and queues that are not supposed to build up.
Therefore, the queues which are reserved internally for each CBR connection are quite small, as shown below:
1010#show atm resource
Resource configuration:
Over-subscription-factor 8 Sustained-cell-rate-margin-factor 1%
Abr-mode: relative-rate
Hierarchical Scheduling Mode : disabled
Service Category to Threshold Group mapping:
cbr 1 vbr-rt 2 vbr-nrt 3 abr 4 ubr 5
Threshold Groups:
Group Max Max Q Min Q Q thresholds Cell Name
cells limit limit Mark Discard count
instal instal instal
---------------------------------------------------
1 65535 63 63 25 % 87 % 0 cbr-default-tg
2 65535 127 127 25 % 87 % 0 vbrrt-default-tg
3 65535 511 31 25 % 87 % 0 vbrnrt-default-tg
4 65535 511 31 25 % 87 % 0 abr-default-tg
5 65535 511 31 25 % 87 % 0 ubr-default-tg
6 65535 1023 1023 25 % 87 % 0 well-known-vc-tg
Queues for CBR VCs are only 63 cells deep. Sixty three (63) cells represent roughly 63x48 = 3024 bytes. This means that we can only queue about two 1500 byte packets at a time on any one of these VCs. This is clearly insufficient as the traffic sent out of these VCs can be quite bursty. The desirable amount of packets that can be queued on a VC depends on the configured PCR, but it is likely to be in the order of ten packets.
Here is an example that illustrates this problem:
1010#ping ip Target IP address: 1.1.1.1 Repeat count [5]: 5 Datagram size [100]: 3000 Timeout in seconds [2]: Extended commands [n]: Sweep range of sizes [n]: Type escape sequence to abort. Sending 5, 3000-byte ICMP Echos to 1.1.1.1, timeout is 2 seconds: ..... Success rate is 0 percent (0/5) 1010#show atm vc int atm 4/0/0.1 2 100 [snip] Threshold Group: 1, Cells queued: 0 Rx cells: 3735, Tx cells: 121573 Tx Clp0:121573, Tx Clp1: 0 Rx Clp0:3735, Rx Clp1: 0 Rx Upc Violations:2735, Rx cell drops:10 Rx pkts:130, Rx pkt drops:5 [snip]
As can be seen, the five echo requests have been dropped. This is because their total size exceeded the CBR queue limit. It may not seem logical that these are counted as receive drops, but it actually makes sense. The counters are to be perceived from the switch fabric point of view, while the cells are received by the switch fabric coming from the (internal) ARM interface - the Rx counters should match the Tx counters of the physical interface (ATM1/0/3).
Ideally, we would like to store at least 5x more cells. If we should desire queues of 511 cells, use the atm threshold-group 1 max-queue-limit command to change the default value of the queue as shown below:
1010#configure threshold Enter configuration commands, one per line. End with CNTL/Z. 1010(config)#atm threshold-group 1 max-queue-limit 511
Use the show atm resource command to verify that the queue is 511 cells deep:
1010#show atm resource
Resource configuration:
[snip]
Threshold Groups:
Group Max Max Q Min Q Q thresholds Cell Name
cells limit limit Mark Discard count
instal instal instal
---------------------------------------------------
1 65535 511 63 25 % 87 % 0 cbr-default-tg
[snip]
Next, make the equivalent changes on the 8540:
8540#show atm resource
Resource configuration:
Sustained-cell-rate-margin-factor 1%
Abr-mode: EFCI
Hierarchical Scheduling Mode : disabled
Service Category to Threshold Group mapping:
cbr 1 vbr-rt 2 vbr-nrt 3 abr 4 ubr 5
Threshold Groups:
Module Group Max Max Q Min Q Q thresholds Cell Name
ID cells limit limit Mark Discard count
instal instal instal
------------------------------------------------------------
1 1 65535 63 63 25 % 87 % 0 cbr-default-tg
2 65535 127 127 25 % 87 % 0 vbrrt-default-tg
3 65535 511 31 25 % 87 % 0 vbrnrt-default-tg
4 65535 511 31 25 % 87 % 0 abr-default-tg
5 65535 511 31 25 % 87 % 0 ubr-default-tg
6 65535 1023 1023 25 % 87 % 0 well-known-vc-tg
===========================================================
[snip]
===========================================================
8 1 65535 63 63 25 % 87 % 0 cbr-default-tg
2 65535 127 127 25 % 87 % 0 vbrrt-default-tg
3 65535 511 31 25 % 87 % 0 vbrnrt-default-tg
4 65535 511 31 25 % 87 % 0 abr-default-tg
5 65535 511 31 25 % 87 % 0 ubr-default-tg
6 65535 1023 1023 25 % 87 % 0 well-known-vc-tg
===========================================================
Although the Catalyst 8540 has a different switch fabric from the LS1010, the principle is the same. The Catalyst 8540 has eight Modules (also known as Master Switch Controllers - MSC) which control the switching of cells through the fabric depending on the ingress interface. In short, the idea is to determine which module handles the traffic coming from the ARM:
8540#show mmc port | inc 12/0/0 int a12/0/0: msc#: 6 port#: 1
MSCs are counted 0 to 7 while Module IDs are counted 1 to 8. Thus, MSC#6 corresponds to Module ID. Therefore, the equivalent change on the 8540 amounts to the following:
8540#configure threshold
Enter configuration commands, one per line. End with CNTL/Z.
8540(config)#atm threshold-group module-id 7 1 max-queue-limit 511
8540#show atm resource module-id 7
Resource configuration:
Sustained-cell-rate-margin-factor 1%
Abr-mode: EFCI
Hierarchical Scheduling Mode : disabled
Service Category to Threshold Group mapping:
cbr 1 vbr-rt 2 vbr-nrt 3 abr 4 ubr 5
Threshold Groups:
Module Group Max Max Q Min Q Q thresholds Cell Name
ID cells limit limit Mark Discard count
instal instal instal
------------------------------------------------------------
7 1 65535 511 63 25 % 87 % 0 cbr-default-tg
2 65535 127 127 25 % 87 % 0 vbrrt-default-tg
3 65535 511 31 25 % 87 % 0 vbrnrt-default-tg
4 65535 511 31 25 % 87 % 0 abr-default-tg
5 65535 511 31 25 % 87 % 0 ubr-default-tg
6 65535 1023 1023 25 % 87 % 0 well-known-vc-tg
===========================================================
The purpose of these changes is to prove that more cells can be queued in these CBR VCs, so try the ping again:
1010#ping ip Target IP address: 1.1.1.1 Repeat count [5]: Datagram size [100]: 3000 Timeout in seconds [2]: Extended commands [n]: Sweep range of sizes [n]: Type escape sequence to abort. Sending 5, 3000-byte ICMP Echos to 1.1.1.1, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 432/433/436 ms
Conclusion
As of version 12.1(7)E, the ATM switch routers have the ability to provide shaped and/or soft-VCs on the ARM modules. There are certain limitations described in this document that one must be aware of, however.
If the network design requires shaping several VCs, and the switch is an LS1010 or Catalyst 8510 MSR, then it is recommended that the ATM TSCAM be used. The TSCAM was designed for this type of application.
Verify
There is currently no verification procedure available for this configuration.
Troubleshoot
There is currently no specific troubleshooting information available for this configuration.
Related Information
- Configuring Virtual Connections
- Configuring Resource Management
- Configuring the ATM Traffic-Shaping Carrier Module
- ATM Technology Support Pages
- Technical Support - Cisco Systems
| Updated: Nov 15, 2007 | Document ID: 17845 |