PDF

Using Atomic Counters

Updated: April 15, 2026

Want to summarize with AI?

Log in

Describes the use of atomic counters in a system. Provides context for when and why atomic counters are used to ensure accurate counting in concurrent environments.

Atomic counters are variables that support atomic increment and decrement operations, ensuring thread-safe counting in concurrent systems.

Atomic counters

Atomic counters are traffic statistics gathering tools that

  • gather statistics about traffic between flows

  • detect drops and misrouting in the fabric for quick debugging and isolation of application connectivity issues, and

  • count packets received from bare metal endpoints without interference to the data path.

Atomic counter capabilities

In conventional settings, it is nearly impossible to monitor the amount of traffic from a bare metal NIC to a specific IP address (an endpoint) or to any IP address. Atomic counters allow an administrator to count the number of packets that are received from a bare metal endpoint without any interference to its data path. In addition, atomic counters can monitor per-protocol traffic that is sent to and from an endpoint or an application group.

Leaf-to-leaf (TEP-to-TEP) atomic counters can provide:

  • Counts of sent, received, dropped, and excess packets

    • Sent packets: The sent number reflects how many packets were sent from the source TEP (tunnel endpoint) to the destination TEP.

    • Received packets: The received number reflects how many packets the destination TEP received from the source TEP.

    • Dropped packets: The dropped number reflects how many packets were dropped during transmission. This number is the difference in the amount of packets sent and the amount of packets received.

    • Excess packets: The excess number reflects how many extra packets were received during transmission. This number is the amount of packets that were unexpectedly received due to a forwarding mismatch or a misrouting to the wrong place.

  • Short-term data collection such as the last 30 seconds, and long-term data collection such as 5 minutes, 15 minutes, or more

  • A breakdown of per-spine traffic (available when the number of TEPs, leaf or VPC, is less than 64)

  • Ongoing monitoring

Note

Leaf-to-leaf (TEP to TEP) atomic counters are cumulative and cannot be cleared. However, because 30-second atomic counters reset at 30-second intervals, they can be used to isolate intermittent or recurring problems. Atomic counters require an active fabric Network Time Protocol (NTP) policy.

Tenant atomic counters can provide:

  • Application-specific counters for traffic across the fabric, including sent, received, dropped, and excess packets

  • Modes include:

    • EPtoEP (endpoint to endpoint)

    • EPGtoEPG (endpoint group to endpoint group)

      Note

      For EPGtoEPG, the options include ipv4 only, ipv6 only, and ipv4, ipv6. Any time there is an ipv6 option, you use twice the TCAM entries, which means the scale numbers may be less than expected for pure ipv4 policies.

    • EPGtoEP (endpoint group to endpoint)

    • EPtoAny (endpoint to any)

    • AnytoEP (any to endpoint)

    • EPGtoIP (endpoint group to IP, used only for external IP address)

    • EPtoExternalIP (endpoint to external IP address)

    Beginning with the 5.2(3) release, endpoint security groups (ESGs) can be used as an alternative for EPGs in these modes.

Atomic counter usage

An administrator can enable atomic counters on all leaf switches to trace packets from endpoint 1 to endpoint 2. If any leaf switches have nonzero counters, other than the source and destination leaf switches, an administrator can drill down to those leafs.

Guidelines and restrictions for atomic counters

Follow these guidelines and restrictions when using atomic counters:

  • Use of atomic counters is not supported when the endpoints are in different tenants or in different contexts (VRFs) within the same tenant.

  • In Cisco APIC release 3.1(2m) and later, if no statistics have been generated on a path in the lifetime of the fabric, no atomic counters are generated for the path. Also, the Traffic Map in the Visualization tab (Operations > Visualization in the Cisco APIC GUI) does not show all paths, only the active paths (paths that had traffic at some point in the fabric lifetime).

  • In pure Layer 2 configurations where the IP address is not learned (the IP address is 0.0.0.0), endpoint-to-EPG and EPG-to-endpoint atomic counter policies are not supported. In these cases, endpoint-to-endpoint and EPG-to-EPG policies are supported. External policies are virtual routing and forwarding (VRF)-based, requiring learned IP addresses, and are supported.

  • When the atomic counter source or destination is an endpoint, the endpoint must be dynamic and not static. Unlike a dynamic endpoint (fv:CEp), a static endpoint (fv:StCEp) does not have a child object (fv:RsCEpToPathEp) that is required by the atomic counter.

  • In a transit topology, where leaf switches are not in full mesh with all spine switches, then leaf-to-leaf (TEP to TEP) counters do not work as expected.

  • For leaf-to-leaf (TEP to TEP) atomic counters, once the number of tunnels increases the hardware limit, the system changes the mode from trail mode to path mode and the user is no longer presented with per-spine traffic.

  • The atomic counter does not count spine proxy traffic.

  • Packets dropped before entering the fabric or before being forwarded to a leaf port are ignored by atomic counters.

  • Packets that are switched in the hypervisor (same Port Group and Host) are not counted.

  • Atomic counters require an active fabric Network Time Protocol (NTP) policy.

  • Atomic counters work for IPv6 sources and destinations, but configuring source and destination IP addresses across IPv4 and IPv6 addresses is not allowed.

  • An atomic counter policy configured with fvCEp as the source or destination counts only the traffic that is from/to the MAC and IP addresses that are present in the fvCEp managed objects. If the fvCEp managed object has an empty IP address field, then all traffic to/from that MAC address would be counted regardless of the IP address. If the Cisco APIC has learned multiple IP addresses for an fvCEp, then traffic from only the one IP address in the fvCEp managed object itself is counted as previously stated. To configure an atomic counter policy to or from a specific IP address, use the fvIp managed object as the source or destination.

  • If there is an fvIp behind an fvCEp, you must add fvIP-based policies and not fvCEp-based policies.

  • Endpoint-to-endpoint atomic counter statistics are not reported for Layer 2 bridged traffic with IPv6 headers when the endpoints belong to the same EPG.

  • For atomic counters to work for traffic flowing from an EPG or ESG to an L3Out EPG, configure the L3Out EPG with 0/1 and 128/1 to match all prefixes instead of 0/0.

  • If your Cisco APIC has the traffic map mode set to "trail" and the Cisco APIC generated the F1545 fault, the only way that you can clear this fault is by setting the traffic map mode to "path." To change the traffic map mode, go to Operations > Visualization, click Settings, choose path for Mode, then click Submit. This will give you tunnel stats per port in both ingress and egress.

    The trail mode has a greater chance of reaching the maximum scale index of tunnel logical interfaces. This mode consumes more software and hardware resources. A logical interface is the ID that is associated with the tunnel in the hardware.

    If you have a single tunnel between a tunnel endpoint (TEP) you specified the trail mode, it will consume more hardware resources as well. For example, if you have 6 fabric ports and a single tunnel, then hardware consumes a number of entries equal to the number of tunnels multiplied by the number of fabric ports.

    For software, if the number of logical interfaces allocated is greater than 2048, you will fail to have an entry in the hardware. As a result, you cannot get the stats. In the case of the atomic counter, this issue may show as drops or excesses.

    The path mode has only entries for the TEP. For a vPC, two entries will be installed. Therefore, you have a lower chance of reaching to the maximum limit.

Configure atomic counters

You can measure traffic between a combination of endpoints, endpoint groups, external interfaces, and IP addresses using atomic counter policies.

Procedure

  1. In the menu bar, click Tenants.

  2. In the submenu bar, click the desired tenant.

  3. In the Navigation pane, expand the tenant and expand Policies and then expand Troubleshoot.

  4. Under Troubleshoot, expand Atomic Counter Policy and choose a traffic topology.

    You can measure traffic between a combination of endpoints, endpoint groups, external interfaces, and IP addresses.

  5. Right-click the desired topology and choose Add topology Policy to open an Add Policy dialog box.

  6. In the Add Policy dialog box, perform the following actions:

    1. In the Name field, enter a name for the policy.
    2. Choose or enter the identifying information for the traffic source.
      The required identifying information differs depending on the type of source (endpoint, endpoint group, external interface, or IP address).
    3. Choose or enter the identifying information for the traffic destination.
    4. Optional: (Optional) In the Filters table, click the + icon to specify filtering of the traffic to be counted.
      In the resulting Create Atomic Counter Filter dialog box, you can specify filtering by the IP protocol number (TCP=6, for example) and by source and destination IP port numbers.
    5. Click Submit to save the atomic counter policy.

  7. In the Navigation pane, under the selected topology, choose the new atomic counter policy.

    The policy configuration is displayed in the Work pane.

  8. In the Work pane, click the Operational tab and click the Traffic subtab to view the atomic counter statistics.

Enable atomic counters

Enable using atomic counters to detect drops and misrouting in the fabric and enable quick debugging and isolation of application connectivity issues.

To enable using atomic counters to detect drops and misrouting in the fabric and enable quick debugging and isolation of application connectivity issues, create one or more tenant atomic counter policies, which can be one of the following types:

  • EP_to_EP—Endpoint to endpoint (dbgacEpToEp)

  • EP_to_EPG—Endpoint to endpoint group (dbgacEpToEpg)

  • EP_to_Ext—Endpoint to external IP address (dbgacEpToExt)

  • EPG_to_EP—Endpoint group to endpoint(dbgacEpgToEp)

  • EPG_to_EPG—Endpoint group to endpoing group (dbgacEpgToEpg)

  • EPG_to_IP—Endpoint group to IP address (dbgacEpgToIp)

  • Ext_to_EP—External IP address to endpoint (dbgacExtToEp)

  • IP_to_EPG—IP address to endpoint group (dbgacIpToEpg)

  • Any_to_EP—Any to endpoint (dbgacAnyToEp)

  • EP_to_Any—Endpoint to any (dbgacEpToAny)

Procedure

  1. To create an EP_to_EP policy using the REST API, use XML such as the following example:

    Example:

    <dbgacEpToEp name="EP_to_EP_Policy" ownerTag="" ownerKey="" 
dn="uni/tn-Tenant64/acEpToEp-EP_to_EP_Policy" descr="" adminSt="enabled">
<dbgacFilter name="EP_to_EP_Filter" ownerTag="" ownerKey="" descr="" 
srcPort="https" prot="tcp" dstPort="https"/>
</dbgacEpToEp>

  2. To create an EP_to_EPG policy using the REST API, use XML such as the following example:

    Example:

    <dbgacEpToEpg name="EP_to_EPG_Pol" ownerTag="" ownerKey="" 
dn="uni/tn-Tenant64/epToEpg-EP_to_EPG_Pol" descr="" adminSt="enabled">
<dbgacFilter name="EP_to_EPG_Filter" ownerTag="" ownerKey="" descr="" 
srcPort="http" prot="tcp" dstPort="http"/>
<dbgacRsToAbsEpg tDn="uni/tn-Tenant64/ap-VRF64_app_prof/epg-EPG64"/>
</dbgacEpToEpg>

The atomic counter policies are created and enabled to monitor traffic patterns and detect drops or misrouting issues in the fabric.

Troubleshoot using atomic counters with the REST API

This task enables you to retrieve and analyze atomic counter data for troubleshooting network connectivity and packet flow issues within the fabric.

Atomic counters provide detailed packet statistics and dropped packet information that help diagnose network problems. Use these REST API calls when troubleshooting endpoint-to-endpoint connectivity or external IP-to-endpoint communication issues.

Procedure

  1. To get a list of the endpoint-to-endpoint atomic counters deployed within the fabric and the associated details such as dropped packet statistics and packet counts, use the dbgEpToEpTsIt class in XML such as the following example:

    Example:

    https://apic-ip-address/api/node/class/dbgEpToEpRslt.xml

  2. To get a list of external IP-to-endpoint atomic counters and the associated details, use the dbgacExtToEp class in XML such as the following example:

    Example:

    https://apic-ip-address/api/node/class/dbgExtToEpRslt.xml

You have successfully retrieved atomic counter information that shows packet statistics and dropped packet details for troubleshooting network connectivity issues.