Configure and Verify Netflow, AVC, and ETA on Catalyst 9000 Series Switches

Introduction

This document describes how to configure and validate NetFlow, Application Visibility and Control (AVC), and Encrypted Traffic Analytics (ETA).

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

• NetFlow
• AVC  
• ETA

Components Used

The information in this document is based on a Catalyst 9300 switch that runs Cisco IOS® XE software 16.12.4.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Related Products

This document can also be used with these hardware and software versions:

• 9200 (Supports NetFlow and AVC only)
• 9300 (Supports NetFlow, AVC, and ETA)
• 9400 (Supports NetFlow, AVC, and ETA)
• 9500 (Supports NetFlow and AVC only)
• 9600 (Supports NetFlow and AVC only)
• Cisco IOS XE 16.12 and later

Background Information

• Flexible NetFlow is the next-generation in flow technology that collects and measures data to allow all routers or switches in the network to become a source of telemetry.
• Flexible NetFlow allows extremely granular and accurate traffic measurements and high-level aggregated traffic collection.
• Flexible NetFlow uses flows to provide statistics for accounting, network monitoring, and network planning.
• A flow is a unidirectional stream of packets that arrives on a source interface and has the same values for the keys. A key is an identified value for a field within the packet. You create a flow via a flow record to define the unique keys for your flow.
  
  

Note: Platform (fed) commands can vary. Command can be show platform fed <active|standby>   versus show platform fed switch <active|standby> .  If the syntax noted in the examples do not parse out, please try the variant.

Network Diagram

Configure

Components

NetFlow configuration is comprised of three main components that can be used together is several variations to perform traffic analysis and data export.

Flow Record

• A record is a combination of key and nonkey fields. Flexible NetFlow records are assigned to Flexible NetFlow flow monitors to define the cache that is used for storage of flow data.
• Flexible NetFlow includes several predefined records that can be used to monitor traffic.
• Flexible NetFlow also allows custom records to be defined for a Flexible NetFlow flow monitor cache by specification of key and nonkey fields to customize the data collection to your specific requirements.
  
  

As shown in the example, flow record configuration details:

flow record TAC-RECORD-IN
match flow direction
match ipv4 source address
match interface input
match ipv4 destination address
match ipv4 protocol
collect counter packets long
collect counter bytes long
collect timestamp absolute last
collect transport tcp flags

flow record TAC-RECORD-OUT
match flow direction
match interface output
match ipv4 source address
match ipv4 destination address
match ipv4 protocol
collect counter packets long
collect counter bytes long
collect timestamp absolute last
collect transport tcp flags

Flow Exporter

• Flow exporters are used to export the data in the flow monitor cache to a remote system (server that functions as a NetFlow collector), for analysis and storage.
• Flow exporters are assigned to flow monitors to provide data export capability for the flow monitors.
  

As shown in the example, flow exporter configuration details:

flow exporter TAC-EXPORT
destination 192.168.69.2
source Vlan69

Flow Monitor

• Flow monitors are the Flexible NetFlow component that is applied to interfaces to perform network traffic monitoring.
• Flow data is collected from the network traffic and added to the flow monitor cache while the process runs. The process is based on the key and nonkey fields in the flow record.
  
  

As shown in the example, flow monitor configuration details:

flow monitor TAC-MONITOR-IN
exporter TAC-EXPORT
record TAC-RECORD-IN

flow monitor TAC-MONITOR-OUT
exporter TAC-EXPORT
record TAC-RECORD-OUT

Switch#show run int g1/0/1
Building configuration...

Current configuration : 185 bytes
!
interface GigabitEthernet1/0/1
switchport access vlan 42
switchport mode access
ip flow monitor TAC-MONITOR-IN input
ip flow monitor TAC-MONITOR-OUT output
load-interval 30
end

Flow Sampler (Optional)

• Flow samplers are created as separate components in a router’s configuration.
• Flow samplers limit the number of packets that are selected for analysis to reduce the load on the device that uses Flexible NetFlow.
• Flow samplers are used to reduce the load on the device that uses Flexible NetFlow achieved through the limit of the number of packets that are selected for analysis.
• Flow samplers exchange accuracy for router performance. If there is a reduction in the number of packets that are analyzed by the flow monitor, accuracy of the information stored in the flow moniitor cache can be impacted.

As shown in the example, example flow sampler configuration:

sampler SAMPLE-TAC
description Sample at 50%
mode random 1 out-of 2

Switch(config)#interface GigabitEthernet1/0/1
Switch(config-if)#ip flow monitor TAC-MONITOR-IN sampler SAMPLE-TAC input
Switch(config-if)#end

Restrictions

• DNA Addon license is required for full Flexible NetFlow, otherwise Sampled NetFlow is only available. 
• Flow-exporters cannot use the management port as a source.

This is not an inclusive list, consult the configuration guide for the appropriate platform and code.

Verify

Platform Independent Verification

Verify the configuration and confirm that the required NetFlow components are present:

1. Flow Record
2. Flow Exporter
3. Flow Monitor 
4. Flow Sampler (Optional)

Tip: To view the flow record, flow exporter, and flow monitor output in one command, run show running-config flow monitor <flow monitor name> expand .

As shown in the example, the flow monitor tied to the input direction and its associated components:

Switch#show running-config flow monitor TAC-MONITOR-IN expand
Current configuration:
!
flow record TAC-RECORD-IN
 match ipv4 protocol
 match ipv4 source address
 match ipv4 destination address
 match interface input
 match flow direction
 collect transport tcp flags
 collect counter bytes long
 collect counter packets long
 collect timestamp absolute last
!
flow exporter TAC-EXPORT
 destination 192.168.69.2
 source Vlan69
!
flow monitor TAC-MONITOR-IN
 exporter TAC-EXPORT
 record TAC-RECORD-IN
!

As shown in the example, the flow monitor tied to the output direction and its associated components:

Switch#show run flow monitor TAC-MONITOR-OUT expand
Current configuration:
!
flow record TAC-RECORD-OUT
 match ipv4 protocol
 match ipv4 source address
 match ipv4 destination address
 match interface output
 match flow direction
 collect transport tcp flags
 collect counter bytes long
 collect counter packets long
 collect timestamp absolute last
!
flow exporter TAC-EXPORT
 destination 192.168.69.2
 source Vlan69
!
flow monitor TAC-MONITOR-OUT
 exporter TAC-EXPORT
 record TAC-RECORD-OUT
!

Run the command show flow monitor <flow monitor name> statistics. This output is helpful to confirm that data is recorded:

Switch#show flow monitor TAC-MONITOR-IN statistics
  Cache type:                               Normal (Platform cache)
  Cache size:                                10000
  Current entries:                               1

  Flows added:                                   1
  Flows aged:                                    0

Run the command show flow monitor <flow monitor name> cacheto confirm that the NetFlow cache has output:

Switch#show flow monitor TAC-MONITOR-IN cache
  Cache type:                               Normal (Platform cache)
  Cache size:                                10000
  Current entries:                               1

  Flows added:                                   1
  Flows aged:                                    0

IPV4 SOURCE ADDRESS:       192.168.200.100
IPV4 DESTINATION ADDRESS:  192.168.100.100
INTERFACE INPUT:           Gi1/0/1
FLOW DIRECTION:            Input
IP PROTOCOL:               17
tcp flags:                 0x00
counter bytes long:        4606617470
counter packets long:      25311085
timestamp abs last:        22:44:48.579

Run the command show flow exporter <exporter name> statisticsto confirm that the exporter sent packets:

Switch#show flow exporter TAC-EXPORT statistics
Flow Exporter TAC-EXPORT:
  Packet send statistics (last cleared 00:08:38 ago):
    Successfully sent:         2                     (24 bytes)

  Client send statistics:
    Client: Flow Monitor TAC-MONITOR-IN
      Records added:           0
      Bytes added:             12
        - sent:                12

    Client: Flow Monitor TAC-MONITOR-OUT
      Records added:           0
      Bytes added:             12
        - sent:                12

Platform Dependent Verification

NetFlow Initialization - NFL Partition Table

• NetFlow partitions are initialized for different features with 16 partitions per direction (Input vs Output).
• NetFlow partition table configuration is divided into the global bank allocation, which is further subdivided into the ingress and egress flow banks.

Key Fields

• Number of partitions
• Partition enable status
• Partition limit
• Current partition usage
  
  

To view the NetFlow Partition Table, you can run the command show platform software fed switch active|standby|member| fnf sw-table-sizes asic <asic number> shadow 0 .

Note: Flows that are created are specific to the switch and asic core when they are created. The switch number (active, standby, etc) needs to specified accordingly. The ASIC number that is input is tied to the respective interface, use show platform software fed switch active|standby|member ifm mappings  to determine ASIC that corresponds to the interface. For the shadow option, always use "0".

Switch#show platform software fed switch active fnf sw-table-sizes asic 0 shadow 0

---------------------------------
     Global Bank Allocation
---------------------------------
Ingress Banks : Bank 0 Bank 1
Egress Banks  : Bank 2 Bank 3
---------------------------------
    Global flow table Info                                   <--- Provides the number of entries used per direction
INGRESS    usedBankEntry           0  usedOvfTcamEntry     0
 EGRESS    usedBankEntry           0  usedOvfTcamEntry     0
---------------------------------
    Flows Statistics
INGRESS    TotalSeen=0 MaxEntries=0 MaxOverflow=0
EGRESS     TotalSeen=0 MaxEntries=0 MaxOverflow=0

---------------------------------
       Partition Table
---------------------------------
## Dir  Limit  CurrFlowCount  OverFlowCount  MonitoringEnabled
 0 ING      0          0               0           0
 1 ING  16640          0               0           1          <-- Current flow count in hardware
 2 ING      0          0               0           0
 3 ING  16640          0               0           0
 4 ING      0          0               0           0
 5 ING   8192          0               0           1
 6 ING      0          0               0           0
 7 ING      0          0               0           0
 8 ING      0          0               0           0
 9 ING      0          0               0           0
10 ING      0          0               0           0
11 ING      0          0               0           0
12 ING      0          0               0           0
13 ING      0          0               0           0
14 ING      0          0               0           0
15 ING      0          0               0           0
 0 EGR      0          0               0           0
 1 EGR  16640          0               0           1          <-- Current flow count in hardware
 2 EGR      0          0               0           0
 3 EGR  16640          0               0           0
 4 EGR      0          0               0           0
 5 EGR   8192          0               0           1
 6 EGR      0          0               0           0
 7 EGR      0          0               0           0
 8 EGR      0          0               0           0
 9 EGR      0          0               0           0
10 EGR      0          0               0           0
11 EGR      0          0               0           0
12 EGR      0          0               0           0
13 EGR      0          0               0           0
14 EGR      0          0               0           0
15 EGR      0          0               0           0

Flow Monitor

Flow monitor configuration includes the following:

1. NetFlow ACL Configuration, which results in creation of an entry within the ACL TCAM table.

The ACL TCAM entry is comprised of:

• Lookup matching keys
• Result parameters used for NetFlow lookup, which includes the following:
  
  • Profile ID
  • NetFlow ID

2. Flow Mask Configuration, which results in creation of an entry in NflLookupTable and NflFlowMaskTable.

• Indexed by NetFlow ACL result parameters to find the flow mask for netflow lookup
  
  

NetFlow ACL

To view the NetFlow ACL configuration run the command show platform hardware fed switch active fwd-asic resource tcam table nfl_acl asic <asic number> .

Tip: If there is a Port ACL (PACL), the entry gets created on the ASIC where the interface is mapped to. In the case of a Router ACL (RACL), the entry is present on all ASIC(s).

• In this output there are NFCMD0 and NFCMD1, that are 4 bit values. In order to calculate the Profile ID, convert the values into binary.
• In this output, NFCMD0 is 1,  NFCMD1 is 2. When converted to binary: 000100010 
• In Cisco IOS XE 16.12 and onwards within the combined 8 bits, the first 4 bits is the profile ID, and the 7th bit indicates that the lookup is enabled. So in the example, 00010010, the profile ID is 1.
• In Cisco IOS XE 16.11 and older versions of code, within the combined 8 bits, the first 6 bits is the profile ID, and the 7th bit indicates that the lookup is enabled. So in this example, 00010010, the profile ID is 4.

Switch#show platform hardware fed switch active fwd-asic resource tcam table nfl_acl asic 0
Printing entries for region INGRESS_NFL_ACL_CONTROL (308) type 6 asic 0
========================================================
Printing entries for region INGRESS_NFL_ACL_GACL (309) type 6 asic 0
========================================================
Printing entries for region INGRESS_NFL_ACL_PACL (310) type 6 asic 0 
========================================================
TAQ-2 Index-32 (A:0,C:0) Valid StartF-1 StartA-1 SkipF-0 SkipA-0
Input IPv4 NFL PACL

Labels  Port   Vlan   L3If   Group
M:      00ff   0000   0000   0000
V:      0001   0000   0000   0000

   vcuResults l3Len l3Pro l3Tos SrcAddr  DstAddr  mtrid vrfid  SH
M:  00000000   0000  00    00  00000000 00000000  00   0000   0000
V:  00000000   0000  00    00  00000000 00000000  00   0000   0000

   RMAC RA MEn IPOPT MF NFF DF SO DPT TM DSEn  l3m
M:  0    0  0    0    0  0   0  0  0   0  0  0
V:  0    0  0    0    0  0   0  0  0   0  0  0

   SrcPort DstPortIITypeCode TCPFlags TTL ISBM QosLabel ReQOS S_P2P D_P2P
M:  0000    0000              00    00    0000    00      0     0     0
V:  0000    0000              00    00    0000    00      0     0     0

   SgEn SgLabel    AuthBehaviorTag  l2srcMiss  l2dstMiss ipTtl SgaclDeny
M:  0    000000    0  0   0 0     0
V:  0    000000    0  0   0 0     0

NFCMD0 NFCMD1 SMPLR LKP1 LKP2 PID QOSPRI MQLBL MPLPRO LUT0PRI CPUCOPY 
     1      2     0    1    0   0      0     0      0 0x0000f       0
Start/Skip Word: 0x00000003
Start Feature, Terminate

-----------------------------------------
Printing entries for region INGRESS_NFL_ACL_VACL (311) type 6 asic 0
========================================================
Printing entries for region INGRESS_NFL_ACL_RACL (312) type 6 asic 0
========================================================
Printing entries for region INGRESS_NFL_ACL_SSID (313) type 6 asic 0
========================================================
Printing entries for region INGRESS_NFL_CATCHALL (314) type 6 asic 0
========================================================
TAQ-2 Index-224 (A:0,C:0) Valid StartF-1 StartA-1 SkipF-0 SkipA-0
Input IPv4 NFL RACL

Labels  Port   Vlan   L3If   Group
M:      0000   0000   0000   0000
V:      0000   0000   0000   0000

   vcuResults l3Len l3Pro l3Tos SrcAddr  DstAddr  mtrid vrfid  SH
M:  00000000   0000  00    00  00000000 00000000  00   0000   0000
V:  00000000   0000  00    00  00000000 00000000  00   0000   0000

   RMAC RA MEn IPOPT MF NFF DF SO DPT TM DSEn  l3m
M:  0    0  0    0    0  0   0  0  0   0  0  0
V:  0    0  0    0    0  0   0  0  0   0  0  0

   SrcPort DstPortIITypeCode TCPFlags TTL ISBM QosLabel ReQOS S_P2P D_P2P
M:  0000    0000              00    00    0000    00      0     0     0
V:  0000    0000              00    00    0000    00      0     0     0

   SgEn SgLabel    AuthBehaviorTag  l2srcMiss  l2dstMiss ipTtl SgaclDeny
M:  0    000000    0  0   0 0     0
V:  0    000000    0  0   0 0     0

NFCMD0 NFCMD1 SMPLR LKP1 LKP2 PID QOSPRI MQLBL MPLPRO LUT0PRI CPUCOPY
     0      0     0    0    0   0      0     0      0 0x00000       0
Start/Skip Word: 0x00000003
Start Feature, Terminate

-----------------------------------------
TAQ-2 Index-225 (A:0,C:0) Valid StartF-0 StartA-0 SkipF-0 SkipA-0
Input IPv4 NFL PACL

Labels  Port   Vlan   L3If   Group
M:      0000   0000   0000   0000
V:      0000   0000   0000   0000

   vcuResults l3Len l3Pro l3Tos SrcAddr  DstAddr  mtrid vrfid  SH
M:  00000000   0000  00    00  00000000 00000000  00   0000   0000
V:  00000000   0000  00    00  00000000 00000000  00   0000   0000

   RMAC RA MEn IPOPT MF NFF DF SO DPT TM DSEn  l3m
M:  0    0  0    0    0  0   0  0  0   0  0  0
V:  0    0  0    0    0  0   0  0  0   0  0  0

   SrcPort DstPortIITypeCode TCPFlags TTL ISBM QosLabel ReQOS S_P2P D_P2P
M:  0000    0000              00    00    0000    00      0     0     0
V:  0000    0000              00    00    0000    00      0     0     0

   SgEn SgLabel    AuthBehaviorTag  l2srcMiss  l2dstMiss ipTtl SgaclDeny
M:  0    000000    0  0   0 0     0
V:  0    000000    0  0   0 0     0

NFCMD0 NFCMD1 SMPLR LKP1 LKP2 PID QOSPRI MQLBL MPLPRO LUT0PRI CPUCOPY
     0      0     0    0    0   0      0     0      0 0x00000       0
Start/Skip Word: 0x00000000
No Start, Terminate

-----------------------------------------
TAQ-2 Index-226 (A:0,C:0) Valid StartF-0 StartA-0 SkipF-0 SkipA-0
Input IPv6 NFL PACL

Labels  Port   Vlan   L3If   Group
Mask    0x0000 0x0000 0x0000 0x0000
Value   0x0000 0x0000 0x0000 0x0000

vcuResult dstAddr0 dstAddr1 dstAddr2 dstAddr3 srcAddr0
00000000  00000000 00000000 00000000 00000000 00000000
00000000  00000000 00000000 00000000 00000000 00000000

srcAddr1 srcAddr2 srcAddr3 TC HL l3Len fLabel vrfId toUs
00000000 00000000 00000000 00 00  0000  00000  000   0
00000000 00000000 00000000 00 00  0000  00000  000   0

l3Pro mtrId AE FE RE HE MF NFF SO IPOPT RA MEn RMAC DPT TMP l3m
 00    00   0  0  0  0  0   0  0    0   0   0   0    0   0    0
 00    00   0  0  0  0  0   0  0    0   0   0   0    0   0    0

DSE srcPort dstPortIITypeCode tcpFlags IIPresent cZId dstZId
 0   0000    0000      00      00     00     00
 0   0000    0000      00      00     00     00

v6RT AH ESP mREn ReQOS QosLabel PRole VRole  AuthBehaviorTag
M: 0   0   0    0    0      00         0     0 0
V: 0   0   0    0    0      00         0     0 0

   SgEn SgLabel
M:  0    000000
V:  0    000000

NFCMD0 NFCMD1 SMPLR LKP1 LKP2 PID QOSPRI MQLBL MPLPRO LUT0PRI CPUCOPY
     0      0     0    0    0   0      0     0      0 0x00000       0
Start/Skip Word: 0x00000000
No Start, Terminate

-----------------------------------------
TAQ-2 Index-228 (A:0,C:0) Valid StartF-0 StartA-0 SkipF-0 SkipA-0
conversion to string vmr l2p not supported
-----------------------------------------
TAQ-2 Index-230 (A:0,C:0) Valid StartF-0 StartA-0 SkipF-0 SkipA-0
Input MAC NFL PACL

Labels  Port   Vlan   L3If   Group
M:      0000   0000   0000   0000
V:      0000   0000   0000   0000

   arpSrcHwAddr  arpDestHwAddr  arpSrcIpAddr  arpTargetIp  arpOperation
M: 000000000000  000000000000   00000000       00000000     0000
V: 000000000000  000000000000   00000000       00000000     0000


   TRUST  SNOOP  SVALID  DVALID
M:    0     0      0      0
V:    0     0      0      0


   arpHardwareLength  arpHardwareType  arpProtocolLength  arpProtocolType
M:   00000000          00000000            00000000          00000000
V:   00000000          00000000            00000000          00000000


   VlanId l2Encap l2Protocol cosCFI   srcMAC      dstMAC    ISBM  QosLabel
M:  000    0        0000     0     000000000000 000000000000 00     00
V:  000    0        0000     0     000000000000 000000000000 00     00


  ReQOS isSnap  isLLC  AuthBehaviorTag
M:  0     0      0       0
V:  0     0      0        0

NFCMD0 NFCMD1 SMPLR LKP1 LKP2 PID QOSPRI MQLBL MPLPRO LUT0PRI CPUCOPY
     0      0     0    0    0   0      0     0      0 0x00000       0
Start/Skip Word: 0x00000000
No Start, Terminate

-----------------------------------------

Flow Mask

Run the command show platform software fed switch active|standby|member fnf fmask-entry asic <asic number> entry 1  to view that the flow mask is installed in hardware. The number of list of key fields can also be found here. 

Switch#show platform software fed switch active fnf fmask-entry asic 1 entry 1

---------------------------------
mask0_valid : 1
Mask hdl0   : 1
Profile ID  : 0
Feature 0   : 148
Fmsk0 RefCnt: 1
Mask M1     :
[511:256] => :00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[255:000] => :FFFFFFFF 00000000 FFFFFFFF 03FF0000 00000000 00FF0000 00000000 C00000FF

Mask M2     :

Key Map     :

Source   Field-Id   Size   NumPFields   Pfields
 002       090       04        01       (0 1 1 1)
 002       091       04        01       (0 1 1 0)
 002       000       01        01       (0 1 0 7)
 000       056       08        01       (0 0 2 4)
 001       011       11        04       (0 0 0 1) (0 0 0 0) (0 1 0 6) (0 0 2 0)
 000       067       32        01       (0 1 12 0)
 000       068       32        01       (0 1 12 2)

Flow Stats and Timestamp Offload Data

Run the command show platform software fed switch active fnf flow-record asic <asic number> start-index <index number> num-flows <number of flows>  to view netflow statistics as well as timestamps

Switch#show platform software fed switch active fnf flow-record asic 1 start-index 1 num-flows 1
1 flows starting at 1 for asic 1:-------------------------------------------------
Idx 996 :
{90, ALR_INGRESS_NET_FLOW_ACL_LOOKUP_TYPE1 = 0x01}
{91, ALR_INGRESS_NET_FLOW_ACL_LOOKUP_TYPE2 = 0x01}
{0, ALR_INGRESS_NFL_SPECIAL1 = 0x00}
{56, PHF_INGRESS_L3_PROTOCOL = 0x11}
{11 PAD-UNK = 0x0000}
{67, PHF_INGRESS_IPV4_DEST_ADDRESS = 0xc0a86464}
{68, PHF_INGRESS_IPV4_SRC_ADDRESS = 0xc0a8c864}
FirstSeen = 0x4b2f, LastSeen = 0x4c59, sysUptime = 0x4c9d
PKT Count = 0x000000000102d5df, L2ByteCount = 0x00000000ca371638


Switch#show platform software fed switch active fnf flow-record asic 1 start-index 1 num-flows 1
1 flows starting at 1 for asic 1:-------------------------------------------------
Idx 996 :
{90, ALR_INGRESS_NET_FLOW_ACL_LOOKUP_TYPE1 = 0x01}
{91, ALR_INGRESS_NET_FLOW_ACL_LOOKUP_TYPE2 = 0x01}
{0, ALR_INGRESS_NFL_SPECIAL1 = 0x00}
{56, PHF_INGRESS_L3_PROTOCOL = 0x11}
{11 PAD-UNK = 0x0000}
{67, PHF_INGRESS_IPV4_DEST_ADDRESS = 0xc0a86464}
{68, PHF_INGRESS_IPV4_SRC_ADDRESS = 0xc0a8c864}
FirstSeen = 0x4b2f, LastSeen = 0x4c5b, sysUptime = 0x4c9f
PKT Count = 0x0000000001050682, L2ByteCount = 0x00000000cbed1590

Application Visibility and Control (AVC)

Background Information

• Application Visibility and Control (AVC) is a solution that leverages Network-Based Recognition Version 2 (NBAR2), NetFlow V9, and various report and management tools (Cisco Prime) to help classify applications via deep packet inspection (DPI). 
• AVC can be configured on wired access ports for standalone switches or switch stacks.
• AVC can also be used on Cisco wireless controllers to identify applications based on DPI and then mark it with a specific DSCP value. It can also collect various wireless performance metrics such as bandwidth usage in terms of applications and clients. 

Performance and Scale

Performance: Each switch member is able to handle 500 connections per second (CPS) at less than 50% CPU utilization. Beyond this rate, AVC service is not guaranteed.

Scale: Ability to handle up to 5000 bi-directional flows per 24 access ports (approximately 200 flows per access port).

Wired AVC Restrictions

• AVC and Encrypted Traffic Analytics (ETA) cannot be configured together at the same time on the same interface.
• Packet classification is only supported for unicast IPv4 (TCP/UDP) traffic.
• NBAR based QoS policy configuration is only supported on wired physical ports. This includes Layer 2 access and trunk ports and Layer 3 routed ports.
• NBAR based QoS policy configuration is not supported on port-channel members, Switch Virtual Interfaces (SVIs) or sub-interfaces.
• NBAR2 based classifiers (match protocol), only support QoS actions of marking and policing. 
• "Match protocol" is limited to 255 different protocols in all policies (8 bit hardware limitation)

Note: This is not an exhaustive list of all restrictions, consult the appropriate AVC configuration guide for your platform and version of code. 

Network Diagram

Components

AVC configuration is comprised of three main componentsthat make up the solution:

Visibility: Protocol Discovery

• Protocol discovery is achieved through NBAR, which provides per interface, direction and application bytes/packets statistics.
• Protocol discovery is enabled for a specific interface through the interface configuration: ip nbar protocol-discovery.

As shown in the output, how to enable protocol discovery:

Switch(config)#interface fi4/0/5
Switch(config-if)#ip nbar protocol-discovery
Switch(config-if)#exit

Switch#show run int fi4/0/5
Building configuration...

Current configuration : 70 bytes
!
interface FiveGigabitEthernet4/0/5
ip nbar protocol-discovery
end

Control: Application Based QoS

When compared to traditional QoS which matches on IP address and UDP/TCP port, AVC achieves finer control through application based QoS, which allows you to match on application, and provides more granular control through QoS actions such as marking and policing.

• Actions are performed on aggregated traffic (not per-flow).
• Application Based QoS is achieved by creation of a class map, match of a protocol, and then creation of a policy map. 
• The Application Based QoS policy is attached to an interface.

As shown in the output, example configuration for application based QoS:

Switch(config)#class-map WEBEX
Switch(config-cmap)#match protocol webex-media
Switch(config)#end

Switch(config)#policy-map WEBEX
Switch(config-pmap)#class WEBEX
Switch(config-pmap-c)#set dscp af41
Switch(config)#end

Switch(config)#interface fi4/0/5
Switch(config-if)#service-policy input WEBEX
Switch(config)#end

Switch#show run int fi4/0/5
Building configuration...

Current configuration : 98 bytes
!
interface FiveGigabitEthernet4/0/5
service-policy input WEBEX
ip nbar protocol-discovery
end

Application-Based Flexible NetFlow

Wired AVC FNF supports two types of predefined flow records: legacy bidirectional flow records and the new directional flow records.

Bidirectional flow records keep track of client/server application statistics.

As shown in the output, example configuration of a bidirectional flow record.

Switch(config)#flow record BIDIR-1
Switch(config-flow-record)#match ipv4 version
Switch(config-flow-record)#match ipv4 protocol
Switch(config-flow-record)#match application name
Switch(config-flow-record)#match connection client ipv4 address
Switch(config-flow-record)#match connection server ipv4 address
Switch(config-flow-record)#match connection server transport port
Switch(config-flow-record)#match flow observation point
Switch(config-flow-record)#collect flow direction
Switch(config-flow-record)#collect connection initiator
Switch(config-flow-record)#collect connection new-connections
Switch(config-flow-record)#collect connection client counter packets long
Switch(config-flow-record)#connection client counter bytes network long
Switch(config-flow-record)#collect connection server counter packets long
Switch(config-flow-record)#connection server counter bytes network long
Switch(config-flow-record)#collect timestamp absolute first
Switch(config-flow-record)#collect timestamp absolute last
Switch(config-flow-record)#end

Switch#show flow record BIDIR-1
flow record BIDIR-1:
Description: User defined
No. of users: 0
Total field space: 78 bytes
Fields:
match ipv4 version
match ipv4 protocol
match application name
match connection client ipv4 address
match connection server ipv4 address
match connection server transport port
match flow observation point
collect flow direction
collect timestamp absolute first
collect timestamp absolute last
collect connection initiator
collect connection new-connections
collect connection server counter packets long
collect connection client counter packets long
collect connection server counter bytes network long
collect connection client counter bytes network long

Directional records are application-stats for input/output.

As shown in the output, configuration examples of input and output directional records:

Note: The command match interface input  specifies a match to the input interface. The command match interface output  specifies a match to the output interface. The command match application name  is mandatory for AVC support. 

Switch(config)#flow record APP-IN
Switch(config-flow-record)#match ipv4 version
Switch(config-flow-record)#match ipv4 protocol
Switch(config-flow-record)#match ipv4 source address
Switch(config-flow-record)#match ipv4 destination address
Switch(config-flow-record)#match transport source-port
Switch(config-flow-record)#match transport destination-port
Switch(config-flow-record)#match interface input          
Switch(config-flow-record)#match application name
Switch(config-flow-record)#collect interface output
Switch(config-flow-record)#collect counter bytes long
Switch(config-flow-record)#collect counter packets long
Switch(config-flow-record)#collect timestamp absolute first
Switch(config-flow-record)#collect timestamp absolute last
Switch(config-flow-record)#end

Switch#show flow record APP-IN
flow record APP-IN:
Description: User defined
No. of users: 0
Total field space: 58 bytes
Fields:
match ipv4 version
match ipv4 protocol
match ipv4 source address
match ipv4 destination address
match transport source-port
match transport destination-port
match interface input
match application name
collect interface output
collect counter bytes long
collect counter packets long
collect timestamp absolute first
collect timestamp absolute last

Switch(config)#flow record APP-OUT
Switch(config-flow-record)#match ipv4 version
Switch(config-flow-record)#match ipv4 protocol
Switch(config-flow-record)#match ipv4 source address
Switch(config-flow-record)#match ipv4 destination address
Switch(config-flow-record)#match transport source-port
Switch(config-flow-record)#match transport destination-port
Switch(config-flow-record)#match interface output 
Switch(config-flow-record)#match application name
Switch(config-flow-record)#collect interface input
Switch(config-flow-record)#collect counter bytes long
Switch(config-flow-record)#collect counter packets long
Switch(config-flow-record)#collect timestamp absolute first
Switch(config-flow-record)#collect timestamp absolute last
Switch(config-flow-record)#end

Switch#show flow record APP-OUT
flow record APP-OUT:
Description: User defined
No. of users: 0
Total field space: 58 bytes
Fields:
match ipv4 version
match ipv4 protocol
match ipv4 source address
match ipv4 destination address
match transport source-port
match transport destination-port
match interface output
match application name
collect interface input
collect counter bytes long
collect counter packets long
collect timestamp absolute first
collect timestamp absolute last

Flow Exporter

Create a flow exporter to define export parameters. 

As shown in the output, example configuration of the flow exporter:

Switch(config)#flow exporter AVC
Switch(config-flow-exporter)#destination 192.168.69.2
Switch(config-flow-exporter)#source vlan69
Switch(config-flow-exporter)#end

Switch#show run flow exporter AVC
Current configuration:
!
flow exporter AVC
destination 192.168.69.2
source Vlan69
!

Flow Monitor

Create a flow monitor to associate it to a flow record.

As shown in the output, example configuration of the flow monitor:

Switch(config)#flow monitor AVC-MONITOR
Switch(config-flow-monitor)#record APP-OUT
Switch(config-flow-monitor)#exporter AVC
Switch(config-flow-monitor)#end

Switch#show run flow monitor AVC-MONITOR
Current configuration:
!
flow monitor AVC-MONITOR
exporter AVC
record APP-OUT

Associate Flow Monitor to an Interface

You can attach up to two different AVC monitors with different predefined records to an interface at the same time.

As shown in the output, example configuration of the flow monitor:

Switch(config)#interface fi4/0/5
Switch(config-if)#ip flow monitor AVC-MONITOR out
Switch(config-if)#end

Switch#show run interface fi4/0/5
Building configuration...
Current configuration : 134 bytes
!
interface FiveGigabitEthernet4/0/5
ip flow monitor AVC-MONITOR output
service-policy input WEBEX
ip nbar protocol-discovery
end

NBAR2

NBAR2 Dynamic Hitless Protocol Pack Upgrade

Protocol packs are software packages that update the NBAR2 protocol support on a device without replacement of the Cisco software on the device. A protocol pack contains information on applications officially supported by NBAR2 which are compiled and packed together. For each application, the protocol-pack includes information on application signatures and application attributes. Each software release has a built-in protocol-pack bundled with it.

• NBAR2 provides a way to update the protocol-packet without any traffic or service interruption and without the need to modigy the software image on the device(s).
• NBAR2 protocol-packets are available for download on Cisco Software Center at: NBAR2 Protocol Pack Library .

NBAR2 Protocol Pack Upgrade

Before installation of a new protocol pack, you must copy the protocol packet to the flash on all switch(es). To load the new protocol pack, use the command ip nbar protocol-pack flash:<Pack Name>.

You do not need to reload the switch(es) to have the NBAR2 upgrade to occur.

As shown in the output, example configuration of how to load the NBAR2 Protocol Pack:

Switch(config)#ip nbar protocol-pack flash:newProtocolPack

To revert to the built-in protocol pack, use the command default ip nbar protocol-pack .

As shown in the output, example configuration of how to revert back to the built-in protocol pack:

Switch(config)#default ip nbar protocol-pack

Display NBAR2 Protocol Pack Information

To display protocol pack information use the commands listed:

• show ip nbar version
• show ip nbar protocol-pack active detail

As shown in the output, example output of those commands:

Switch#show ip nbar version
NBAR software version: 37
NBAR minimum backward compatible version: 37
NBAR change ID: 293126

Loaded Protocol Pack(s):
Name: Advanced Protocol Pack
Version: 43.0
Publisher: Cisco Systems Inc.
NBAR Engine Version: 37
State: Active

Switch#show ip nbar protocol-pack active detail
Active Protocol Pack:
Name: Advanced Protocol Pack
Version: 43.0
Publisher: Cisco Systems Inc.
NBAR Engine Version: 37
State: Active

NBAR2 Custom Applications 

NBAR2 supports the use of custom protocols to identify custom applications. Custom protocols support protocols and applications that NBAR2 does not currently support.

These can include the following:

• Specific application to an organization
• Applications specific to a geography 

NBAR2 provides a way to manually customize applications through the command ip nbar custom <myappname>. 

Note: Custom applications take precedence over built-in protocols

There are various types of application customization:

Generic protocol customization

• HTTP 
• SSL
• DNS

Composite:Customization based on multiple protocols –server-name.

Layer3/Layer4 customization

• IPv4 address

• DSCP values

• TCP/UDP ports

• Flow source or destination direction

Switch(config)#ip nbar custom MYHTTP http host *mydomain.com id 10

Example custom application called MYSSL that uses SSL unique-name mydomain.com with selector ID 11.

Switch(config)#ip nbar custom MYSSL ssl unique-name *mydomain.com id 11

DNS Customization

NBAR2 examines DNS request and response traffic, and can correlate the DNS response to an application. The IP address returned from the DNS response is cached and used for later packet flows associated with that specific application.

The command  ip nbar customapplication-namedns domain-nameidapplication-id is used for DNS customization. To extend an application, use the command  ip nbar custom  application-name dns domain-name  domain-name  extends existing-application.

Example custom application called MYDNS that uses the DNS domain name “mydomain.com” with selector ID 12.

Switch(config)#ip nbar custom MYDNS dns domain-name *mydomain.com id 12

Composite Customization 

Example custom application called MYDOMAIN that uses HTTP, SSL or DNS domain name mydomain.com with selector ID 13.

Switch(config)#ip nbar custom MYDOMAIN composite server-name *mydomain.com id 13

L3/L4 Customization

Switch(config)#ip nbar custom LAYER4CUSTOM transport tcp id 14
Switch(config-custom)#ip address 10.56.1.10 10.56.1.11
Switch(config-custom)#dscp ef
Switch(config-custom)#end

Monitor Custom Applications

To monitor custom applications utilize the show commands listed:

Switch#show ip nbar protocol-id | inc Custom
LAYER4CUSTOM             14            Custom
MYDNS                    12            Custom
MYDOMAIN                 13            Custom
MYHTTP                   10            Custom
MYSSL                    11            Custom

Switch#show ip nbar protocol-id MYSSL
Protocol Name             id            type
----------------------------------------------
MYSSL                    11            Custom

Switch#show ip nbar control-plane
NGCP Status:
============

graph sender info:
NBAR state is ACTIVATED
NBAR config send mode is ASYNC
NBAR config state is READY

NBAR update ID 3
NBAR batch ID ACK 3
NBAR last batch ID ACK clients 1 (ID: 4)
Active clients 1 (ID: 4)
NBAR max protocol ID ever 1935
NBAR Control-Plane Version: 37

<snip>

Switch#show platform software fed switch active wdavc function wdavc_stile_cp_show_info_ui

Is DP activated : TRUE
MSG ID : 3
Maximum number of flows: 262144
Current number of graphs: 1
Requests queue state : WDAVC_STILE_REQ_QUEUE_STATE_UP
Number of requests in queue : 0
Max number of requests in queue (TBD): 1
Counters:
activate_msgs_rcvd : 1
graph_download_begin_msgs_rcvd : 3
stile_config_msgs_rcvd : 1584
graph_download_end_msgs_rcvd : 3
deactivate_msgs_rcvd : 0
intf_proto_disc_msgs_rcvd : 1
intf_attach_msgs_rcvd : 2
cfg_response_msgs_sent : 1593
num_of_handle_msg_from_fmanfp_events : 1594
num_of_handle_request_from_queue : 1594
num_of_handle_process_requests_events : 1594

Switch#show platform software fed switch active wdavc flows

CurrFlows=1, Watermark=1

IX |IP1           |IP2           |PORT1|PORT2|L3   |L4   |VRF |TIMEOUT|APP     |TUPLE|FLOW     |IS FIF  |BYPASS|FINAL |#PKTS |BYPASS
   |              |              |     |     |PROTO|PROTO|VLAN|SEC    |NAME    |TYPE |TYPE     |SWAPPED |      |      |      |PKT
--------------------------------------------------------------------------------------
1  |192.168.100.2 |192.168.200.2 |68   |67   |1    |17   |0   |360    |unknown |Full |Real Flow|Yes     |True |True   |40    |40

Key Fields:

CurrFlows: Demonstrates how many active flows that are tracked by AVC

Watermark: Demonstrates the largest number of flows historically tracked by AVC

TIMEOUT SEC: Inactivity timeout based on the identified application

APP NAME: Identified application 

FLOW TYPE: Real Flow indicates this was created as a result of inbound data. Pre Flow indicates this flow is created as a result of inbound data. Pre-flows are used for anticipated media flows

TUPLE TYPE: Real flows are always full tuple, Pre-flows are either full tuple or half tuple

BYPASS: If set to TRUE, indicates that no more packets are required by software in order to identify this flow

FINAL: If set to TRUE, indicates that the application does not change anymore for this flow

BYPASS PKT: How many packets were needed in order to get to final classification

#PKTS: How many packets were actually punted to software for this flow

View  additional details about current flows, you can utilize the command show platform software fed switch active wdavc function wdavc_ft_show_all_flows_seg_ui.

Switch#show platform software fed switch active wdavc function wdavc_ft_show_all_flows_seg_ui
CurrFlows=1, Watermark=1

IX |IP1           |IP2          |PORT1|PORT2|L3   |L4   |VRF |TIMEOUT|APP     |TUPLE |FLOW     |IS FIF  |BYPASS|FINAL |#PKTS |BYPASS
   |              |             |     |     |PROTO|PROTO|VLAN|SEC    |NAME    |TYPE  |TYPE     |SWAPPED |      |      |      |PKT
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
1  |192.168.100.2 |192.168.200.2|68   |67   |1    |17   |0   |360    |unknown |Full  |Real Flow|Yes     |True  |True  |40    |40

   SEG IDX |I/F ID |OPST I/F |SEG DIR |FIF DIR |Is SET |DOP ID |NFL HDL  |BPS PND |APP PND |FRST TS  |LAST TS  |BYTES   |PKTS  |TCP FLGS
   ------------------------------------------------------------------------------------------------------------------------------------
   0       |9      |----     |Ingress |True    |True   |0      |50331823 |0       |0       |177403000|191422000|24252524|70094 |0

Key Fields

I/F ID: Specifies the Interface ID

SEG DIR: Specifies ingress of egress direction

FIF DIR: Determines whether or not this is the flow initiator direction

NFL HDL: Flow ID in hardware

To view the entry in hardware run the command show platform software fed switch active fnf flow-record asic <number> start-index <number> num-flows <number of flows>.

Note: To choose the ASIC, it is the ASIC instance which the port is mapped to. To identify the ASIC, utilize the command  show platform software fed switch active|standby|member ifm  mappings. The start-index can be set to 0 if you are not interested in a specific flow. Otherwise, the start-index needs to be specified. For num-flows, that specifies the number of flows that can be viewed, maximum 10. 

Switch#show platform software fed switch active fnf flow-record asic 3 start-index 0 num-flows 1
1 flows starting at 0 for asic 3:-------------------------------------------------
Idx 175 :
{90, ALR_INGRESS_NET_FLOW_ACL_LOOKUP_TYPE1 = 0x01}
{91, ALR_INGRESS_NET_FLOW_ACL_LOOKUP_TYPE2 = 0x01}
{0, ALR_INGRESS_NFL_SPECIAL1 = 0x00}
{11 PAD-UNK = 0x0000}
{94, PHF_INGRESS_DEST_PORT_OR_ICMP_OR_IGMP_OR_PIM_FIRST16B = 0x0043}
{93, PHF_INGRESS_SRC_PORT = 0x0044}
{67, PHF_INGRESS_IPV4_DEST_ADDRESS = 0xc0a8c802}
{68, PHF_INGRESS_IPV4_SRC_ADDRESS = 0xc0a86402}
{56, PHF_INGRESS_L3_PROTOCOL = 0x11}
FirstSeen = 0x2b4fb, LastSeen = 0x2eede, sysUptime = 0x2ef1c
PKT Count = 0x000000000001216f, L2ByteCount = 0x0000000001873006

Look for Various Errors and Warnings in the Data Path

Utilize  the command  show platform software fed switch active|standby|member wdavc function wdavc_ft_show_stats_ui | inc err|warn|fail   to view potential flow table errors:

Switch#show platform software fed switch active wdavc function wdavc_ft_show_stats_ui | inc err|warn|fail
Bucket linked exceed max error : 0
extract_tuple_non_first_fragment_warn : 0
ft_client_err_alloc_fail : 0
ft_client_err_detach_fail : 0
ft_client_err_detach_fail_intf_attach : 0
ft_inst_nfl_clock_sync_err : 0
ft_ager_err_invalid_timeout : 0
ft_intf_err_alloc_fail : 0
ft_intf_err_detach_fail : 0
ft_inst_err_unreg_client_all : 0
ft_inst_err_inst_del_fail : 0
ft_flow_seg_sync_nfl_resp_pend_del_warn : 0
ager_sm_cb_bad_status_err : 0
ager_sm_cb_received_err : 0
ft_ager_to_time_no_mask_err : 0
ft_ager_to_time_latest_zero_ts_warn : 0
ft_ager_to_time_seg_zero_ts_warn : 0
ft_ager_to_time_ts_bigger_curr_warn : 0
ft_ager_to_ad_nfl_resp_error : 0
ft_ager_to_ad_req_all_recv_error : 0
ft_ager_to_ad_req_error : 0
ft_ager_to_ad_resp_error : 0
ft_ager_to_ad_req_restart_timer_due_err : 0
ft_ager_to_flow_del_nfl_resp_error : 0
ft_ager_to_flow_del_all_recv_error : 0
ft_ager_to_flow_del_req_error : 0
ft_ager_to_flow_del_resp_error : 0
ft_consumer_timer_start_error : 0
ft_consumer_tw_stop_error : 0
ft_consumer_memory_error : 0
ft_consumer_ad_resp_error : 0
ft_consumer_ad_resp_fc_error : 0
ft_consumer_cb_err : 0
ft_consumer_ad_resp_zero_ts_warn : 0
ft_consumer_ad_resp_zero_pkts_bytes_warn : 0
ft_consumer_remove_on_count_zero_err : 0
ft_ext_field_ref_cnt_zero_warn : 0
ft_ext_gen_ref_cnt_zero_warn : 0

Utilize the command  show platform software fed switch active wdavc function wdavc_stile_stats_show_ui | inc err   to view any potential NBAR errors:

Switch#show platform software fed switch active wdavc function wdavc_stile_stats_show_ui | inc err
find_flow_error : 0
add_flow_error : 0
remove_flow_error : 0
detach_fo_error : 0
is_forward_direction_error : 0
set_flow_aging_error : 0
ft_process_packet_error : 0
sys_meminfo_get_error : 0

Verify that Packets are Cloned to CPU

Utilize the command  show platform software fed switch active punt cpuq 21 | inc received   to verify that packets are cloned to the CPU for NBAR processing:

Note: In the lab this number did not increment. 

Switch#show platform software fed switch active punt cpuq 21 | inc received
Packets received from ASIC : 63

Identify CPU Congestion

In times of congestion, packets can be dropped before sent to WDAVC process. Utilize the command  show platform software fed switch active wdavc function fed_wdavc_show_ots_stats_uito validate:

Switch#show platform software fed switch active wdavc function fed_wdavc_show_ots_stats_ui
OTS Limits
----------------------------------------------
ots_queue_max : 20000
emer_bypass_ots_queue_stress : 4000
emer_bypass_ots_queue_normal : 200
OTS Statistics
----------------------------------------------
total_requests : 40
total_non_wdavc_requests : 0
request_empty_field_data_error : 0
request_invalid_di_error : 0
request_buf_coalesce_error : 0
request_invalid_format_error : 0
request_ip_version_error : 0
request_empty_packet_error : 0
memory_allocation_error : 0
emergency_bypass_requests_warn : 0
dropped_requests : 0
enqueued_requests : 40
max_ots_queue : 0

Tip: To clear the punt drop counter utilize the command show platform software fed switch active wdavc function fed_wdavc_clear_ots_stats_ui.

Identify Scale Issues

If there are no free FNF entries in hardware, traffic is not subject to NBAR2 classification. Utilize the command  show platform software fed switch active fnf sw-table-sizes asic <number> shadow 0   to confirm:

Note: Flows that are created are specific to the switch and asic core when they are created. The switch number (active, standby, etc) needs to specified accordingly. The ASIC number that is input is tied to the respective interface, use  show platform software fed switch active|standby|member ifm mappings   to determine ASIC that corresponds to the interface. For the shadow option, always use 0.

Switch#show platform software fed switch active fnf sw-table-sizes asic 3 shadow 0

---------------------------------
Global Bank Allocation
---------------------------------
Ingress Banks : Bank 0
Egress Banks : Bank 1
---------------------------------
Global flow table Info
INGRESS usedBankEntry 1 usedOvfTcamEntry 0
EGRESS usedBankEntry 0 usedOvfTcamEntry 0 <-- 256 means TCAM entries are full
---------------------------------
Flows Statistics
INGRESS TotalSeen=1 MaxEntries=1 MaxOverflow=0
EGRESS TotalSeen=0 MaxEntries=0 MaxOverflow=0

---------------------------------
Partition Table
---------------------------------
## Dir  Limit CurrFlowCount OverFlowCount MonitoringEnabled
 0 ING      0         0             0           0
 1 ING  16640         1             0           1
 2 ING      0         0             0           0
 3 ING  16640         0             0           0
 4 ING      0         0             0           0
 5 ING   8192         0             0           1
 6 ING      0         0             0           0
 7 ING      0         0             0           0
 8 ING      0         0             0           0
 9 ING      0         0             0           0
10 ING      0         0             0           0
11 ING      0         0             0           0
12 ING      0         0             0           0
13 ING      0         0             0           0
14 ING      0         0             0           0
15 ING      0         0             0           0
 0 EGR      0         0             0           0
 1 EGR  16640         0             0           1
 2 EGR      0         0             0           0
 3 EGR  16640         0             0           0
 4 EGR      0         0             0           0
 5 EGR   8192         0             0           1
 6 EGR      0         0             0           0
 7 EGR      0         0             0           0
 8 EGR      0         0             0           0
 9 EGR      0         0             0           0
10 EGR      0         0             0           0
11 EGR      0         0             0           0
12 EGR      0         0             0           0
13 EGR      0         0             0           0
14 EGR      0         0             0           0
15 EGR      0         0             0           0

Encrypted Traffic Analytics (ETA)

Background Information

• ETA focuses on identification of malware communication in encrypted traffic through passive monitoring, extraction of relevant data elements, and a combination of behavioral modeling and machine learning with cloud-based global security.
• ETA leverages telemetry from NetFlow as well as encrypted malware detection and cryptographic compliance and sends this data to Cisco Stealthwatch. 
• ETA extracts two main data elements: the Initial Data Packet (IDP) and the Sequence of Packet Length and Time (SPLT).

Network Diagram

Components

ETA is comprised of several different components that are used in conjunction to create the ETA solution:

• NetFlow - Standard that defines data elements exported by network devices that describe the flows on the network.
• Cisco Stealthwatch - Harnesses the power of network telemetry that includes NetFlow, IPFIX, proxy logs, and deep packet inspection of raw packets - to provide advanced network visibility, security intelligence, and analytics. 
• Cisco Cognitive Intelligence - Finds malicious activity that has bypassed security controls or entered through unmonitored channels and  inside an organization environment. 
• Encrypted Traffic Analytics - Cisco IOS XE feature that uses advanced behavioral algorithms to identify malicious traffic patterns through analysis of inftraflow metadata of encrypted traffic, detects potential threats hidden in encrypted traffic. 

Note: This part of the document only focuses on configuration and verification of ETA and NetFlow on the Catalyst 9000 series switch and does not cover Stealthwatch Management Console (SMC) and Flow Collector (FC) deployment to the Cognitive Intelligence Cloud. 

Restrictions

• Deployment of ETA requires DNA Advantage to function
• ETA and a transmit (TX) Switched Port Analyzer (SPAN) is not supported on the same interface.

This is not an inclusive list, consult the appropriate configuration guide for the switch and version of code for all restrictions. 

Configuration

As show in the output, enable ETA on the switch globally and define the flow export destination:

C9300(config)#et-analytics
C9300(config-et-analytics)#ip flow-export destination 172.16.18.1 2055

Tip: You MUST use port 2055, do not use another port number.

Next, configure Flexible NetFlow as show in the output:

Configure Flow Record

C9300(config)#flow record FNF-RECORD
C9300(config-flow-record)#match ipv4 protocol
C9300(config-flow-record)#match ipv4 source address
C9300(config-flow-record)#match ipv4 destination address
C9300(config-flow-record)#match transport source-port
C9300(config-flow-record)#match transport destination-port
C9300(config-flow-record)#collect counter bytes long
C9300(config-flow-record)#collect counter packets long
C9300(config-flow-record)#collect timestamp absolute first
C9300(config-flow-record)#collect timestamp absolute last

Configure Flow Monitor

C9300(config)#flow exporter FNF-EXPORTER
C9300(config-flow-exporter)#destination 172.16.18.1
C9300(config-flow-exporter)#transport udp 2055
C9300(config-flow-exporter)#template data timeout 30
C9300(config-flow-exporter)#option interface-table
C9300(config-flow-exporter)#option application-table timeout 10
C9300(config-flow-exporter)#exit

Configure Flow Record

C9300(config)#flow monitor FNF-MONITOR
C9300(config-flow-monitor)#exporter FNF-EXPORTER
C9300(config-flow-monitor)#record FNF-RECORD
C9300(config-flow-monitor)#end

Apply Flow Monitor

C9300(config)#int range g1/0/3-4
C9300(config-if-range)#ip flow mon FNF-MONITOR in
C9300(config-if-range)#ip flow mon FNF-MONITOR out
C9300(config-if-range)#end

Enable ETA on Switch Interface(s)

C9300(config)#interface range g1/0/3-4
C9300(config-if-range)#et-analytics enable

Verify

Verify that the ETA monitor, eta-mon, is active. Confirm that the status is allocated through the command  show flow monitor eta-mon .

C9300#show flow monitor eta-mon
Flow Monitor eta-mon:
Description: User defined
Flow Record: eta-rec
Flow Exporter: eta-exp
Cache:
Type: normal (Platform cache)
Status: allocated
Size: 10000 entries
Inactive Timeout: 15 secs
Active Timeout: 1800 secs

Verify that the ETA cache is populated. When NetFlow and ETA are configured on the same interface, utilize show flow monitor <monitor name> cache instead of show flow monitor eta-mon cache  as the output from show flow monitor eta-mon cache  is empty:

C9300#show flow monitor FNF-MONITOR cache
Cache type: Normal (Platform cache)
Cache size: 10000
Current entries: 4

Flows added: 8
Flows aged: 4
- Inactive timeout ( 15 secs) 4

IPV4 SOURCE ADDRESS: 192.168.10.2
IPV4 DESTINATION ADDRESS: 192.168.20.2
TRNS SOURCE PORT: 0
TRNS DESTINATION PORT: 0
IP PROTOCOL: 1
counter bytes long: 500
counter packets long: 5
timestamp abs first: 21:53:23.390
timestamp abs last: 21:53:23.390

IPV4 SOURCE ADDRESS: 192.168.20.2
IPV4 DESTINATION ADDRESS: 192.168.10.2
TRNS SOURCE PORT: 0
TRNS DESTINATION PORT: 0
IP PROTOCOL: 1
counter bytes long: 500
counter packets long: 5
timestamp abs first: 21:53:23.390
timestamp abs last: 21:53:23.390

IPV4 SOURCE ADDRESS: 192.168.20.2
IPV4 DESTINATION ADDRESS: 192.168.10.2
TRNS SOURCE PORT: 0
TRNS DESTINATION PORT: 0
IP PROTOCOL: 1
counter bytes long: 500
counter packets long: 5
timestamp abs first: 21:53:23.390
timestamp abs last: 21:53:23.390

IPV4 SOURCE ADDRESS: 192.168.10.2
IPV4 DESTINATION ADDRESS: 192.168.20.2
TRNS SOURCE PORT: 0
TRNS DESTINATION PORT: 0
IP PROTOCOL: 1
counter bytes long: 500
counter packets long: 5
timestamp abs first: 21:53:23.390
timestamp abs last: 21:53:23.390

Validate that flows are exported towards the SMC and FC with the command show flow exporter eta-exp statistics .

C9300#show flow exporter eta-exp statistics
Flow Exporter eta-exp:
Packet send statistics (last cleared 03:05:32 ago):
Successfully sent: 3 (3266 bytes)

Client send statistics:
Client: Flow Monitor eta-mon
Records added: 4
- sent: 4
Bytes added: 3266
- sent: 3266

Confirm that SPLT and IDP are exported to the FC with the command  show platform software fed switch active fnf et-analytics-flows.

C9300#show platform software fed switch active fnf et-analytics-flows

ET Analytics Flow dump

=================
Total packets received : 20
Excess packets received : 0
Excess syn received : 0
Total eta records added : 4
Current eta records : 0
Total eta splt exported : 2
Total eta IDP exported : 2

Validate which interfaces are configured for et-analytics with the command  show platform software et-analytics interfaces .

C9300#show platform software et-analytics interfaces
ET-Analytics interfaces
GigabitEthernet1/0/3
GigabitEthernet1/0/4

ET-Analytics VLANs

Use the command  show platform software et-analytics global  to view a global state of ETA:

C9300#show plat soft et-analytics global
ET-Analytics Global state
=========================
All Interfaces : Off
IP Flow-record Destination : 10.31.126.233 : 2055
Inactive timer : 15

ET-Analytics interfaces
GigabitEthernet1/0/3
GigabitEthernet1/0/4

ET-Analytics VLANs