Multiple processors residing on a dedicated system processor module as
well as locally on interface hardware work together to ensure successful
transmission and receipt of packets over ATM virtual circuits (VCs). These
processors communicate among themselves by posting messages to perform such
functions as VC setup and teardown, physical-layer statistics collection, and
alarm generation. These messages, called love letters or love messages, are
written by one processor into a block of memory. A receiving processor then
reads the message. The output of the debug atm
events command provides a window into this messaging mechanism,
such as the following output from a PA-A3.
Jun 17 12:48:50.631 BST: atmdx_mailbox_proc(ATM5/0/0): received report type 2
Jun 17 12:48:50.631 BST: atmdx_process_love_letter(ATM5/0/0): 2 VCs core
Jun 17 12:48:55.631 BST: atmdx_mailbox_proc(ATM5/0/0): received report type 3
Jun 17 12:48:55.631 BST: atmdx_process_love_letter(ATM5/0/0): 1 VCs aux
The purpose of this document is illustrate sample debug
atm event output to help distinguish between informational
messages and messages that point to an operational problem. This document also
reviews standard ATM interface software architecture.
Caution: Before issuing debug commands, please refer to
Important Information on
Debug Commands. The debug atm events command
may print a large amount of disruptive debug output on a production router
depending on the number of VCs for which it needs to report statistics as well
as the amount of VC-related events.
There are no specific requirements for this document.
This document is not restricted to specific software and hardware
For more information on document conventions, refer to the
Cisco Technical Tips
All ATM interfaces use a software architecture that consists of
multiple blocks. Before we walk through these software blocks, we first need to
understand Cisco IOS® Software drivers and the PCI bus architecture inside your
A driver allows software engineers to implement something called
hardware abstraction. It allows engineers to create a fundamental set of
software blocks that run on any platform, and then use drivers to adapt this
platform-independent code to a specific platform such as the 7200 series or the
The PA-A3 supports a PCI host driver that allows the Segmentation and
Reassembly's (SAR's) processor to interface with the peripheral component
interconnect (PCI) buses that run length of the 7200/7400 series, as well as
the versatile interface processor (VIP) on RSP platforms. PCI buses serve as a
data path between port adapters and host memory on the VIP or on the Network
Processing Engine (NPE)/ Network Services Engine (NSE). The following diagram
illustrates the architecture of the VIP2 and the location of the PCI
This table lists the software blocks on the PA-A3:
Platform- or PA-independent software functions that all ATM
interfaces use. For example, ATM core handles OAM and ILMI
Platform-dependent software functions that "bridge" the general
ATM core software with the PCI host driver software. ATM core and the PCI host
driver exchange commands, status updates, and statistics via the bridge. The
platform ATM driver also handles receive-packet forwarding, platform-specific
initialization functions, and physical-layer statistics as shown in the
show controller atm display.
PCI host driver
Provides the PCI host interface for the SAR chip on the PA-A3.
Performs several key functions:
Downloads firmware to the SAR
Monitors framer alarms
Part of each SAR's hardware functional block. Performs several
Downloads boot code to configure the SARs and enables them to
exchange control data with the PCI host driver.
Generates interrupts when the SAR needs to write cells into
memory on the receive path and schedule cells on the transmit
Returns empty buffers to the PCI host driver.
Processes commands sent from the PCI host driver and relays
locally collected statistics to the PCI host
Start-up or boot code as well as optimized runtime images for the
ATM processor unit (APU) on the receive and transmit SARs. Downloaded from the
PCI host driver.
On the RSP/VIP platform, the platform driver resides in the RSP system
image and VIP system image, while the PCI host driver is part of the VIP system
image. On the 7200 platform, both drivers are part of the system image.
The PA-A3-specific software is bundled with the VIP software or with
the system software for other supporting platforms.
As noted above, a mailbox is part of a messaging model that Cisco IOS
uses to transport messages between two CPUs. Here is how this process generally
A driver allocates a message buffer.
A love note or letter fills the message buffer.
The receiving processor reads the message buffer.
When finished reading the command buffer, the processor generates a
"message done" interrupt.
The message buffer is returned to the free buffer
Now this document examines two sets of messages exchanged between
processors running the Cisco IOS Software components described in the
The PCI host driver collects per-VC statistics on each packet. The VIP
platform driver autonomously relays these statistics to the RSP platform driver
via a love note every second. The show atm vc
command displays the current VC data. The VIP platform driver relays framer
statistics to the RSP every 10 seconds. When the system initializes, it creates
a special background process that handles the autonomous statistics from the
VIP as a scheduled process rather than at the interrupt level to minimize
The debug atm events command prints output
on VC-related events such as setup and teardown.
Set up a VC. The platform-dependent driver delivers the request
to the PCI host driver.
Tears down an existing VC. The platform-dependent driver relays
the request to the PCI host driver.
Retrieves VC statistics on demand; supports only a single VC
Verifies QoS paramters before a VC is set up.
The SAR internally consists of hardware functional blocks. One such
block is the ATM processing unit (APU), which is a miniRISC with customized
logic for ATM-specific extensions. The PCI host driver and the APU, which runs
the ATM firmware, communicate via a messaging mailbox. At any given time, one
outstanding command for each APU is used to instruct the PA firmware to perform
a specific task, such as a VC setup. The firmware relays per-VC and per-PA
statistics to the PCI host driver every 10 seconds if the data changes.
The following output generated from debug atm
event shows the commands sent by the PCI host driver to the
firmware. The firmware returns only acknowledgments to indicate the success of
the command. These acknowledgments are not displayed in the debug
7200-1.3(config)# int atm 6/0
7200-1.3(config-if)# pvc 1/100
7200-1.3(config-if-atm-vc)# vbr-nrt 45000 45000
17:07:43: atmdx_setup_vc(ATM6/0): vc:14 vpi:1 vci:100 state:2 config_status:0
17:07:43: atmdx_pas_vc_setup(ATM6/0): vcd 14, atm hdr 0x00100640, mtu 4482
17:07:43: VBR: pcr 96000, scr 96000, mbs 94
17:07:43: vc tx_limit=1600, rx_limit=480
17:07:43: Created 64-bit VC counterss
7200-1.3(config)# int atm 6/0
7200-1.3(config-if)# no pvc 1/100
17:08:48: atmdx_teardown_vc(ATM6/0): idb state 4 vcd 14 state 4
17:08:48: atmdx_pas_teardown_vc(ATM6/0): vcd 14
Now this document applies the preceding information by walking through
the software architecture of the inverse multiplexing over ATM (IMA) network
module (NM) for the 2600 and 3600 router series.
The IMA NM has a "host" side to indicate functions or memory on the
processor module and a "local" side to indicate functions or memory on the
network module itself. The host side runs platform-independent and
platform-dependent drivers. The local side executes firmware downloaded by the
host drivers to the NM's onboard CPU. This image handles the physical-layer
functions, including control of the framer ASIC, collection of physical-layer
statistics, and generation of loopbacks and alarms. The Cisco IOS drivers and
the NM firmware communicate via mail messages.
On the local side, the NM IMA also runs an IMA driver that similarly
uses a message mailbox to communicate to the local CPU.
Messages in the direction of host side to local side are designed
mostly for configuration. These messages include:
Physical layer E1/T1 configuration data
IMA group configuration
Query for IMA group/link status
Query for RFC 1406 management information base (MIB)
Query for IMA MIB data
Messages sent in the direction of local side to host side are used to
communicate line state changes and performance statistics, including
The following sample output illustrates the love notes used to setup
and teardown a VC. We shut and no shut the physical interface to force the
teardown. Note that "rs8234" refers to the SAR on the NM.
3640-1.1(config)# int atm2/ima2
3640-1.1(config-if)# pvc 1/1
*Mar 1 00:17:20.323: Reserved bw for 1/1 Available bw = 6000
*Mar 1 00:17:20.323: rs8234_setup_vc(ATM2/IMA2): vc:4 vpi:1 vci:1
*Mar 1 00:17:20.323: rs8234_setup_vc_common() VCD=260 vp/vc=17/1 etype=0
*Mar 1 00:17:20.323: rs8234_setup_cos(ATM2/IMA2): vc:4 wred_name:- max_q:0
*Mar 1 00:17:20.327: Created 64-bit VC counters
*Mar 1 00:17:20.327: rs8234_teardown_vc(ATM2/IMA2): vc:260 vpi:1 vci:1
*Mar 1 00:17:20.327: rs8234_teardown_vc proceeds (ATM2/IMA2): vc:260 vpi:1
*Mar 1 00:17:20.327: Status and ptr is 400 Status Q is 1
*Mar 1 00:17:20.331: Resetting ATM2/IMA2
*Mar 1 00:17:20.331: rs8234_teardown_vc(ATM2/IMA2): vc:260 vpi:1 vci:1
*Mar 1 00:17:20.331: rs8234_teardown_vc proceeds (ATM2/IMA2): vc:260 vpi:1 vci:1
*Mar 1 00:17:20.331: Remove link with ports 8,links 4,channel 1
*Mar 1 00:17:22.327: %LINK-5-CHANGED: Interface ATM2/IMA2, changed state to administratively down
3640-1.1(config-if)# no shut
*Mar 1 00:17:31.287: Resetting ATM2/IMA2
*Mar 1 00:17:31.287: IMA config_interface ATM2/IMA2
*Mar 1 00:17:31.287: IMA config_restart ATM2/IMA2
*Mar 1 00:17:31.287: IMA restarting 0 VCs
*Mar 1 00:17:31.287: rs8234_setup_vc(ATM2/IMA2): vc:4 vpi:1 vci:1
*Mar 1 00:17:31.287: rs8234_setup_vc_common() VCD=260 vp/vc=17/1 etype=0
*Mar 1 00:17:31.287: rs8234_setup_cos(ATM2/IMA2): vc:4 wred_name:- max_q:0