Because the ASAs implement stateful tracking of flows, the tracked flows go through a set of state changes. NetFlow is used to export data about the status of a flow, and is triggered by the event that caused the state change. The tracked events include flow creation, flow denial (only flows denied by ACLs), and flow teardown.
To provide periodic byte counters for flow traffic, the ASA issues a timer per flow to generate periodic NSEL events in addition to state tracking. These periodic NSEL events, called flow update events, are usually timer-driven, which makes them more in line with traditional Netflow. However, flow update events can also be triggered by some flow state changes.
The ASA also exports syslog messages that include the same information. You can disable these syslog messages to avoid performance degradation by generating both NSEL records and syslog messages that represent the same event. For a list of redundant syslog messages, see the
“Using NSEL and Syslog Messages” section
in the Cisco ASA Series General Operations CLI Configuration Guide.
Most bidirectional flows are already assembled internally and are considered a single flow. The flow records reported by NSEL on the ASAs describe both directions of the flow. The data records explicitly define the source (initiator) and destination (responder) of the connection, and you can use this information to determine the direction of flow, if required by collector applications. In addition, some NSEL records include two byte counter fields, NF_F_FWD_FLOW_DELTA_BYTES and NF_F_REV_FLOW_DELTA_BYTES, which provide direction-specific traffic data.
The RFC states that templates may be sent to the user either at regular time intervals or after a set number of data records have been exported. These update intervals must be configurable. This implementation supports template updates by time interval only. Template updates based on the number of data records are not supported.
Options Template and Data Records
No options template or data records will be exported. Some fields are supported by
commands in the CLI. Collector applications must issue
commands to obtain additional information about certain fields. In addition, collectors must have unique hostnames and IP addresses; otherwise, the inspection behavior will be unpredictable. For more information, see the “Information Model” section and the Cisco ASA Series General Operations CLI Configuration Guide.
Observation Point and Observation Domain
The ASA is an Observation Domain, with each interface also an Observation Point. Flows that are created through all interfaces are exported, and no option exists to limit or filter the exported data to a specific set of interfaces. Flow that are created by external devices that connect to the ASA are also exported.
Only records for certain flows may need to be exported, For example, the ASA can generate NSEL events for flows that match an ACE. You can use this method to restrict the number of NSEL events that are generated for NetFlow. This implementation supports the filtering of NSEL events based on traffic and event type through Modular Policy Framework, with records sent to different collectors.
For example, with two collectors, you can do the following:
Log all flow creation events to Collector 1.
Log all flow denied events matching ACL1 to Collector 1.
Log all events matching ACL1 to Collector 2.
If the Modular Policy Framework is not configured for NetFlow, no NSEL events are generated. For more information, see the Cisco ASA Series General Operations CLI Configuration Guide and the command reference.
This implementation of NetFlow only supports UDP payloads.
This section describes the data types and templates that are exported through NetFlow, and includes the following topics:
The list of required data elements was arrived at by consolidating the data exported by syslog messages that are generated for events that results in the export of NSEL records.
Table 1 lists the data elements that are exported from the ASAs through NSEL.
The columns include the following information:
ID—A unique name that represents the field type
TYPE—The value assigned for this field type
LEN—The length of the field in records exported for the selected ASA
DESC—A description of what the field type represents
Table 1 Data Records Exported Through NSEL
Connection ID Field
An identifier of a unique flow for the device
Flow ID Fields (L3 IPv4)
Source IPv4 address
Destination IPv4 address
Flow ID Fields (L3 IPv6)
Source IPv6 address
Destination IPv6 address
Flow ID Fields (L4)
ICMP type value
ICMP code value
ICMP IPv6 type value
ICMP IPv6 code value
Flow ID Fields (INTF)
Ingress IFC SNMP IF index
Egress IFC SNMP IF index
Mapped Flow ID Fields (L3 IPv4)
Post NAT Source IPv4 Address
Post NAT Destination IPv4 Address
Post NATT Source Transport Port
Post NATT Destination Transport Port
Mapped Flow ID Fields (L3 IPv6)
Post NAT Source IPv6 Address
Post NAT Destination IPv6 Address
Status or Event Fields
High-level event code. Values are as follows:
Extended event code. These values provide additional information about the event.
Timestamp and Statistics Fields
The time that the event occurred, which comes from IPFIX. Use 324 for time in microseconds, and 325 for time in nanoseconds. Time has been counted as milliseconds since 0000 UTC January 1, 1970.
The time that the flow was created, which is included in extended flow-teardown events in which the flow-create event was not sent earlier. The flow duration can be determined with the event time for the flow-teardown and flow-create times.
The delta number of bytes from source to destination.
The delta number of bytes from destination to source.
The input ACL that permitted or denied the flow
All ACL IDs are composed of the following three, four-byte values:
Hash value or ID of the ACL name
Hash value, ID, or line of an ACE within the ACL
Hash value or ID of an extended ACE configuration
The output ACL that permitted or denied a flow
AAA username of maximum permitted size
Event IDs Field
The Event ID field describes the event that resulted in the NSEL record.
Table 2 lists the values for event IDs.
Table 2 Values for Event IDs
Ignore—This value indicates that a field must be ignored. This value is not used in the current release.
Flow created—This value indicates that a new flow was created.
Flow deleted—This value indicates that a flow was deleted.
Flow denied—This value indicates that a flow was denied.
Flow updated—This value indicates that a flow timer went off or a flow was torn down.
Extended Event IDs Field
The extended event ID provides additional information about a particular event. This field includes a product-specific field ID (33002).
Table 3 lists the values for extended event IDs.
Table 3 Values for Extended Event IDs
Extended Event ID
This value indicates that the field must be ignored.
Values above 1000 represent various reasons for why a flow was denied.
A flow was denied by an ingress ACL.
A flow was denied by an egress ACL.
Possible reasons include the following:
An attempt to connect to the ASA interface was denied.
The ICMP packet to the device was denied.
The ICMPv6 packet to the device was denied.
The first packet on the TCP was not a TCP SYN packet.
Values above 2000 represent various reasons why a flow was terminated.
Event Time Field
Each NSEL data record has the event time field (NF_F_EVENT_TIME_MSEC), which is the time that the event occurred in milliseconds. The NetFlow packet may consist of multiple events; however, the time that the packet is sent does not represent the time that the event occurred, because the NetFlow service waits for multiple events to pack the NetFlow packet.
Note Different events in the life of a flow may be issued in separate NetFlow packets and may arrive out-of-order at the collector. For example, the packet containing a flow teardown event may reach the collector before the packet containing a flow creation event. As a result, it is important that collector applications use the Event Time field to correlate events.
Data Records and Templates
This section describes the templates that are supported for various events and includes the following topics:
Templates describe the format of data records that are exported through NetFlow. Each flow event has several record formats or templates associated with it, as follows:
There are different templates for different events.
There are different templates for IPv4 and IPv6 flows under each event type.
There are different templates for IPV44, IPV46, IPV64, and IPV66 flows under each event type.
The flow creation/permitted event has different templates, which are based on the size of the username field associated with the flow. Different templates are required because the size of string fields is fixed in NetFlow. Having a single template with the largest possible size for string results is a waste of bandwidth, because most strings are far shorter than the maximum value. Two types of username fields are defined, which result in two types of templates in each category.
– A common username size for usernames that are less than 20 characters
– A maximum username size for usernames that are up to a maximum of 65 characters
– Each template has the Event Type and Extended Event Type fields, which can interpret or act on the event.
The flow denied and flow deletion events have IPV46 and IPV64 templates in which the destination IP address has been translated by a NAT rule, but the source IP address has not been translated by a NAT rule; this results in different IP versions between the source and destination IP addresses. The source and destination NAT rules are not applied at the same time (the destination NAT rule is applied first), so it is possible for a NetFlow record to be generated before both NAT rules are applied or when only one NAT rule is available.
These partial NAT translation templates are not needed for flow creation and delayed flow creation events because both source and destination IP addresses need to be the same IP version for a flow to be created.
Note Template definitions are sent to all collectors, and you should use these IDs and definitions to parse data records.
Templates for Flow Creation Events
Flow creation events indicate that a flow has been created by the ASA. This event is also a log of flows that the ASA allows.
Table 4 describes the templates to use for flow creation events.
Table 4 Templates for Flow Creation Events
IPv44 flow creation event with common username size (20 chars)
For short-lived flows, NSEL collection devices would benefit from processing a single event instead of these two events—flow-create and flow-teardown. So a configurable CLI parameter is provided to delay sending of the flow-create event. If the timer fires, the flow-create event is sent. However, if the flow is torn down before the timer expires,
the flow-teardown event is sent; no flow-create event is sent.
The flow-teardown event is extended and includes all information regarding the flow; no information is lost. New templates are introduced to accommodate the extended flow-teardown events.
Templates for Extended Flow Teardown Events
Table 5 describes the templates that are used for extended flow-teardown events.
Table 5 Templates for Extended Flow Teardown Events
Extended IPv44 flow teardown with common username size (20 chars)
Flow update events indicate that a flow update timer has gone off for a flow or a flow was torn down. This event functions as a periodic byte counter for flow traffic. Flow update events also use the same templates as flow teardown events, excluding those for partial NAT translation. The NF_F_FWD_FLOW_DELTA_BYTES and NF_F_REV_FLOW_DELTA_BYTES fields contain the byte counts since the last timer interval. The NF_F_FW_EXT_EVENT field is not used and is ignored in flow update records. See
Table 7 for the templates that are used for flow teardown events.
Flow Update (at timer) and Flow Update (at teardown) Events
The ASA sets flow update timers for flows passing through it, and when the timers goes off, NSEL issues flow update (at timer) records. If there is no activity on the flow for the configured time interval, no flow update (at timer) records are sent for that interval. A flow update (at teardown) record is sent with a flow teardown record to capture the traffic in the last time interval. No flow update (at teardown) record is sent if there is no traffic on the flow for the last interval. In addition, no flow update (at teardown) record is sent for short-lived flows (that is, if teardown occurs before the first flow update (at timer) event occurs).
The flow update timer is not set nor is it ever set again if at the time of flow creation, no flow update collectors are configured or if during a flow update event, the flow update collectors are removed. Under these conditions, no flow update (at timer) event or flow update (at teardown) event is seen again.
Flow Update Records and Failover
An attempt to keep flow update records consistent before and after failover is made. After failover occurs, all flow update records are based on the last update from the previously active ASA. This update occurs every 15 seconds as long as traffic is flowing. Inaccuracies may appear in flow update records if failover pairs are brought up at different times, or if failover occurs before the active ASA has a chance to send a periodic update to the standby ASA.
Flow Update Events and Clustering
One major divergence occurs in how flow update events interact with failover and how they interact with clustering. In clustering, before ownership change, the flow director has a stub flow copy of the original flow, which would not have the active refresh timer set. Only after the original flow owner goes down will a full flow copy be generated with the active refresh timer set. This means it is highly likely that a noticeable time offset will occur between when the flow update timer goes off on the original flow owner and when the flow update timer goes off on the new flow owner.
After flow ownership changes in a cluster, all flow-update records are based on the last update that the flow director received. Flow information is updated every 15 seconds as long as there is traffic. Maintenance of up-to-date flow information uses the same methods as those provided for failover.
NetFlow and Failover
NetFlow data records and templates are only sent from the active (primary) ASA in an active-standby failover pair. The standby (secondary) ASA does not send any NetFlow-related information. However, after failover, the secondary ASA starts to send templates and NetFlow records for any replicated or new flows. The source IP address for each NetFlow collector connection is the same between the two ASAs, but the source port varies. This means that the NetFlow collectors are capable of differentiating packets sent from the primary unit and the secondary unit.
In an active-active failover pair, both ASAs may send NetFlow data records and templates simultaneously. Only the active unit per context sends the NetFlow packets, but the standby unit does not; much like in active-standby scenarios. The source IP address for each NetFlow collector connection is the same for an ASA context and its copy, but the source port varies.
Each ASA node (context) in the failover pair establishes its own connection to the NetFlow collector(s) and advertises its templates independently. The collector uses the source IP address and source port of the packet to differentiate between the NetFlow exporters.
NetFlow and Clustering
NetFlow is supported on both management and regular data interfaces; however, we recommend that you use management interfaces. When the NetFlow collector connection is configured on management-only interfaces, each ASA in the cluster uses its own per-unit source IP address and source port to send NetFlow packets. NetFlow may be used with both data interfaces in layer 2 mode and layer 3 mode. For data interfaces in layer 2 mode, each ASA in the cluster has the same source IP address, but the source port is different. Although layer 2 mode is designed to make a cluster appear as a single device, a NetFlow collector can differentiate between the different nodes in the cluster. For data interfaces in layer 3 mode, NetFlow operates the same way as management-only interfaces do.
Each ASA node in the cluster establishes its own connection to the NetFlow collector(s) and advertises its templates independently. The collector uses the source IP address and source port of the packet to differentiate between the NetFlow exporters.
For information about the commands that are used to configure the NSEL implementation on the ASA, see the Cisco ASA Series General Operations CLI Configuration Guide and the command reference. The commands are also used to display additional information about the fields in NSEL records.
External Partner Implementation Suggestions
This section provides examples of flows that generate events and includes information about how to implement collector support for the new NSEL fields for the ASA, and includes the following topics:
The following example shows an allowed flow that uses the PAT interface. The output interface IP address is 22.214.171.124. The user is authenticated as User A. No ACLs are specified; however, the flow is outbound, so it is allowed by default. According to Figure 1 and the description provided, a flow creation event would be issued.
Figure 1 Example of an Allowed Flow with a PAT Interface
The resulting NSEL record would include the following fields and values:
Example 2: Denied Flow on Egress with PAT Interface
The following example shows a denied flow through an egress ACL that uses the PAT interface. The output interface IP address is 126.96.36.199. The user is authenticated as User A. An input ACL (foo) allows the flow, but an output ACL (bar) denies the flow. The input ACL (foo) is specified with an object group, as shown in the following example:
hostname (config)# access-list bar extended deny tcp any any
hostname (config)# access-group foo in interface inside
hostname (config)# access-group bar out interface outside
According to Figure 1 and the description provided, a flow denied event would be issued.
The resulting NSEL record would include the following fields and values:
1002 (egress ACL)
Decoding Device Fields Through the CLI
To decode some of the field values that the ASA populates, direct interaction with the device may be required. We recommend that you use a dynamic mechanism such as
scripts to obtain the required information from the CLI of the device that issued the event.
The device supports console, Telnet, and SSH secure shell access; however, SSH is the recommended method because of performance and security. The following sections describe fields that you need to decode, based on interaction with the ASA, and includes the following topics:
You can also decode the Interface ID fields using SNMP GET requests from the device interface MIB. This is the only field that has MIB support.
You may use the
show interface detail
command to obtain a list of all the interfaces on the device. This output includes a line under each interface that corresponds to the Interface ID value sent in the NetFlow fields. In the following example, the interface number is 8.
hostname(config)# show interface filter-outside detail
Interface GigabitEthernet4/3 "filter-outside", is up, line protocol is up
The 12-byte raw ACL ID must be divided into its three constituent parts, as follows:
The first four bytes are the ACL Name ID.
The next four bytes are the ACL Entry ID (ACE)/Object-Group ID.
The final four bytes are the Extended ACL Entry ID.
These individual values can be looked up in the output of the
command from the ASA. The ACL Name ID is at the end of the ACL first line in this output. The ACE ID is at the end of each individual ACL entry line.
Note If you use an object-group in an access list, then the second four-byte ID is not actually the ACE ID; it is the Object-Group ID. The Extended ACE ID (the final four-byte part) refers to the actual individual ACL Entry ID. The following example shows these entries:
where 0x102154c1 are the first four bytes, 0xd0e5806e are the second four bytes, and 0x7e5ad93b are the final four bytes.
where 0x5da9bb69 are the first four bytes, 0x84434b4b are the second four bytes, and 0x00000000 are the final four bytes.
Note Each of these IDs corresponds to lines from the show access-list command example.
From these IDs, you can deduce that access-list
was applied on the input interface, and that access-list
was applied on the output interface. That information is also available through the
show run access-group
command, but the added benefit of these ACL IDs is that you can identify the individual ACE that caused the permit or deny action. Because this flow was denied on egress (determined from the extended event code), you know that the ingress ACL ID identifies the ACE line that permitted the flow and that the egress ACL ID identifies the ACE that denied the flow.
You must hard code event codes into the collector, because the ASA only issues four different high-level event types (creation, teardown, denial, and update).
Extended Event Codes
Of the four high-level event codes, only two have extended event codes: the flow denial and flow teardown event types. For the flow denied event, the list of extended event codes in
Table 3 should suffice to determine the reason why the flow was denied. However, for the flow teardown event, there are too many event codes to list in this document, and the set of reasons is quite fluid.
Obtaining Documentation and Submitting a Service Request
What’s New in Cisco Product Documentation
, which lists all new and revised Cisco technical documentation, as an RSS feed and deliver content directly to your desktop using a reader application. The RSS feeds are a free service.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks
. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)