White Paper
Last updated: June 2006
The next-generation in flow technology allowing optimization of the network infrastructure, reducing operation costs, improving capacity planning and security incident detection with increased flexibility and scalability.
INTRODUCTION
NetFlow invented by Cisco has become the standard for acquiring IP operational data for many customers. Visibility into the network is an indispensable tool. In response to new requirements and pressures, network operators are finding it critical to understand how the network is behaving including:
• Application and network usage
• Understand who, what, when, where, and how network traffic is flowing
• Network efficiency and utilization of network resources
• The impact of changes to the network
• Network anomaly and security vulnerabilities
• Long term compliance, business process and audit trail
Applications for NetFlow data are constantly being invented but the key usages include:
• Real-time network monitoring
• Application and user profiling
• Network planning and capacity planning
• Security incident detection and classification
• Accounting and billing
• Network data warehousing, forensics and data mining
• Troubleshooting
NetFlow Based Network Awareness
The ability to characterize IP traffic and understand who sent it, the traffic destination, the time of day, the application information, is critical for network availability, performance and troubleshooting. Monitoring IP traffic flows facilitates more accurate capacity planning and ensures that resources are used appropriately in support of organizational goals. It helps Cisco customers determine how to optimize resource usage, plan network capacity, where to apply Quality of Service (QoS) and it plays a vital role in network security to detect Denial-of-Service (DoS) attacks and network-propagated worms.
NetFlow facilitates solutions to many common problems encountered by network professionals as shown in Table 1.
Table 1. Common Solutions Facilitated by NetFlow
For a primer in the basics of NetFlow please read the "Introduction to NetFlow-A Technical Overview" document first.
THE NEXT GENERATION IN FLOW TECHNOLOGY-FLEXIBLE NETFLOW
Cisco is now innovating flow technology to a new level beyond what has been traditionally available. Cisco IOS Flexible NetFlow is Cisco's next-generation flow technology. Flexible NetFlow provides enhanced optimization of the network infrastructure, reduces costs, and improves capacity planning and security detection beyond other flow based technologies available today.
Key Advantages to using Flexible NetFlow:
• Flexibility, scalability, aggregation of flow data beyond traditional NetFlow
• The ability to monitor a wider range of packet information producing new information about network behavior
• Enhanced network anomaly and security detection
• User configurable flow information to perform customized traffic identification and the ability to focus and monitor specific network behavior
• Convergence of multiple accounting technologies into one accounting mechanism
It provides a NetFlow architecture that can track multiple NetFlow applications simultaneously. For example, the user can create simultaneous and separate Flow Monitors for security analysis and traffic analysis. Cisco IOS Flexible NetFlow provides enhanced security detection and or network troubleshooting by allowing customization of flow information. For example, the user can create a specific Flow Monitor to focus and analyze a particular network issue or incident. It provides real-time monitoring with immediate flow cache capabilities and long term or permanent tracking of flow data. Cisco IOS Flexible NetFlow will enhance NetFlow's already rich feature capabilities allowing the tracking of information at layer 2 for switching environments, layer 3 and 4 for IP information and up to layer 7 with deep packet inspection for application monitoring. Figure 1 shows various types of Flow Monitors to view and understand network behavior.
Figure 1. Example of Flexible NetFlow Customizable Flow Monitors
An Example of Application Tracking with Flexible NetFlow
Flexible NetFlow unlike traditional NetFlow allows the user to customize and focus on specific network information. Scalability of the NetFlow analysis is optimized and Flexible NetFlow gives the opportunity to track the important information for the organization. By targeting specific information the amount of information will be reduced and the number of flows being exported reduced, allowing enhanced scalability and aggregation. If for instance the user was interested in TCP application analysis, in Flexible NetFlow the user would configure the tracking of the NetFlow field's source and destination IP addresses, TCP source and destination ports and NetFlow will examine the packets for this information. This information will effectively show who is sending and receiving the traffic per application port. In traditional NetFlow, packet information is used to create flows but this information is fixed and not configurable by the user. In traditional NetFlow aggregation comes with the expense of lost information but in Flexible NetFlow the user can actually track multiple sets of information to make sure all flow information in the network is captured efficiently. In the above example the user only needs four NetFlow fields to track application usage and this is contrasted to 7 fields in traditional NetFlow. In traditional NetFlow, the user must track the 7 key fields and each field tracked leads to a greater number of flows.
An Example of Security Detection with Flexible Netflow
Flexible NetFlow is an excellent attack detection tool with capabilities to track all parts of the IPv4 header and even packet sections, and characterize this information into flows. It is expected that security detection systems will listen to NetFlow data and upon finding an issue in the network, create a virtual bucket or virtual cache that will be configured to track specific information and pinpoint details about the attack pattern or worm propagation. The capability to create caches on the fly with specific information combined with input filtering (ie: filtering all flows to a specific destination) allows Flexible NetFlow to be a better security detection tool than current flow technologies. It is expected common attacks such as port scans for worm target discovery and worm propagation will be tracked in Flexible NetFlow. Let's discuss a common simpler attack, in which TCP flags are used to flood open TCP requests to a destination server (ie: SYN flood attack). The attacking device will send a stream TCP SYN's to a given destination address but never send the ACK in response to the servers SYN-ACK as part of the TCP 3-way handshake. The flow information needed for security monitor requires the tracking of three key fields: destination address or subnet, TCP flags and packet count. The security device may be monitoring general NetFlow information and this data may trigger a detailed view of this particular attack. The detailed Flow Monitor might include input filtering to limit what traffic is visible in the NetFlow cache along with the tracking of the specific information to diagnose the TCP based attack. In this case, the user may want to filter all flow information to the server destination address or subnet to limit the amount of information the security server needs to evaluate. If the security detection server decided it understood this attack, it might then program another virtual cache or bucket to export payload information or sections of packets to take a deeper look at a signature within the packet. The above is just one of many possible examples of how Flexible NetFlow can be used to detect security incidents.
What are the Key Components Within Flexible Netflow?
NetFlow has a number of key components:
• NetFlow cache
• NetFlow Flow Record
• Flow export timers
• NetFlow export format
• NetFlow server for collection and reporting
The NetFlow cache stores flow information. The Flow Record is created by inspecting a packet and a description of packet information is added to the NetFlow cache as the Flow Record. NetFlow exports or pushes flow information to the NetFlow reporting server. This is in contrast to SNMP pull or the polling model to retrieve data. The Flow Records in the cache will expire or terminate and be exported to a NetFlow collector and be used to create management reports. The flows will expire based on timers and if the flow is inactive (no new packets and bytes for the flow) it will be exported. Flow timers are discussed later in the document.
What is a Flexible NetFlow Key Field?
Each packet that is forwarded within a router or switch is examined for a set of IP packet attributes. These attributes are the IP packet identity or key fields for the flow and determine if the packet information is unique or similar to other packets.
Traditionally, an IP flow is based on a set of seven IP packet attributes. This set of key fields is tracked and if the set of values for these fields are unique, a new Flow Record is created in the NetFlow cache.
All packets with the same source/destination IP address, source/destination ports, protocol, interface and class of service are grouped into a flow and then packets and bytes tallied. This methodology of flow characterization or determining a flow is scalable because a large amount of network information is condensed into a database of NetFlow information called the NetFlow cache. The NetFlow cache is associated with a flexible Flow Monitor capability that will be discussed in more detail within this document. Figure 2 shows the basic components of NetFlow including the flow key fields, NetFlow cache and reporting server.
Figure 2. Creating a Flow in the NetFlow Cache
What is a Flexible NetFlow Non-Key Field?
Additional information can be added to the Flow Record and this information is named non-key fields. Non-key fields are added to the flow entry in the NetFlow cache and exported. The non-key fields are not used to create or characterize the flows but are exported and just added to the flow. In Flexible NetFlow, non-key fields are also configurable by the user. If a field is non-key, normally only the first packet of the flow is used for the value in this field.
Typical non-key NetFlow fields include:
• Flow timestamps to understand the life of a flow; timestamps are useful for calculating packets and bytes per second
• Next hop IP addresses including BGP routing Autonomous Systems (AS)
• Subnet mask for the source and destination addresses to calculate prefixes
• TCP flags to examine TCP handshakes
Creating a Flow with Key and Non-key Fields
Figure 3 is an example of flow creation based on NetFlow key and non-key fields. The first packet is inspected and the key field values are found and the set of key field values is unique within the flow cache and a flow is created.
Figure 3. Packet 1 has Key and Non-Key Field Values in the Netflow Cache
Packet 2 is then inspected and the values of the key fields are the same as packet 1 and not unique, so packets and bytes are incremented for the existing flow. Figure 4 shows how the packets and bytes are incremented for the flow.
Figure 4. Packet 2 Key and Non-key Field Values and the NetFlow Cache
Packet 3 is then inspected and the value of source address key field changes, and when the flow cache is inspected, the set of key field values is unique and therefore a new flow is created in the NetFlow cache. Figure 5 shows information for packet 3.
Figure 5. Packet 3 Key and Non-key Field Values and the NetFlow Cache
The set of key field values are used to determine if a flow is unique and should be tracked as a new flow or if packets and bytes should be tallied for an existing flow.
Flexible NetFlow will allow the user to select what key and non-key fields to define flows. This capability allows the user flexibility, aggregation and scalability beyond traditional NetFlow.
What is a Flexible NetFlow Flow Monitor?
A Flexible NetFlow Flow Monitor describes the NetFlow cache or information stored in the cache. The Flow Monitor contains the Flow Records or key and non-key fields within the cache. Also, part of the Flow Monitor is the Flow Exporter which contains information about the export of NetFlow information including the destination address of the NetFlow collector. The Flow Monitor includes various cache characteristics including the timers for exporting, the size of the cache and if required, the packet sampling rate. Figure 6 depicts the Flow Monitor and its components.
Figure 6. Flow Monitor Components
What is a Flexible NetFlow Flow Record?
A Flow Record is a set of key and non-key NetFlow field values used to characterize flows in the NetFlow cache. Flow Records may be pre-defined for ease of use or customized and user defined. A typical pre-defined record will aggregate flow data and allow users to target common applications for NetFlow. User defined records will allow selection of specific key or non-key fields in the Flow Record. The user defined field is the key to Flexible NetFlow allowing a wide range of information to be characterized and exported by NetFlow. It is expected that different network management applications will support specific user defined and pre-defined Flow Records based on what they are monitoring (ie: security detection, traffic analysis, capacity planning).
What is a Flexible NetFlow Exporter?
There are two primary methods to access NetFlow data: the Command Line Interface (CLI) with show commands or utilizing an application reporting tool receiving export. NetFlow export, unlike SNMP, polling pushes information periodically to the NetFlow reporting collector. In general, the NetFlow cache is constantly filling with flows, and software in the router or switch is searching the cache for flows that have terminated or expired, and these flows are exported to the NetFlow collector server. Flow export is optional but it's the only method to get a complete view of all NetFlow data in the Cisco device. The Flexible NetFlow Exporter allows the user to define where the export can be sent, the type of transport for the export and properties for the export. Multiple exporters can be configured per Flow Monitor or the same exporter can be used by multiple monitors. For example, the user may want to send the same data to a billing and also a traffic analysis server. The number of exporters is only limited by the resources on the network device. The Flexible NetFlow Exporter also has the capability to send optional data such as tables of information to a NetFlow collector. An example of table export would be the interface If-index to interface name mapping. The exporter can also allow the class of service to be marked for the export stream. The exporter will support various export formats including v5, v9 and the IETF IP Flow Information Export standard (IPFIX). The exporter can support various transport protocols including UDP, Stream Control Transmission Protocol (SCTP).
Figure 7 shows the flexibility of Flexible NetFlow including the ability to create different Flow Monitors per device interface and the ability to have different Flow Records and exporters per Flow Monitor.
Figure 7. Flexible NetFlow Flow Monitor, Export and Record Definition
What are the Netflow Cache Characteristics and Flow Timers?
Flows are continuously being created, are tracked and then expire and are exported from the NetFlow cache to a reporting server. A flow is ready for export when it is inactive for a certain time (ie: no new packets received for the flow); or if the flow is long lived (active) and lasts greater than the active timer (ie: long FTP download). Their are timers to determine if a flow is inactive or if a flow is long lived, and the default for the inactive flow timer is 15 seconds and the active flow timer is 30 minutes. All the timers for export are configurable.
Each cache can also be configured for packet sampling and therefore a subset of packets can be randomly sampled from the traffic stream and used to characterize the network traffic. Packet sampling is effective for capacity planning, where an approximate view of network traffic may be enough to give a good indication of traffic capacity. Packet sampling can be used for high speed interfaces (ie: OC-192) where creating flows for every packet is not possible. Packet sampling can be an effective method to decrease the volume of NetFlow export and decrease CPU utilization. Many Cisco platforms can support full NetFlow (non-sampled) but in some environments sampling is a necessity.
What is the NetFlow Version 9 Export Format?
There are various formats for the export packet and these are commonly called the export version. The export versions are well documented formats including version 5, 7, and 9. The most common format used is NetFlow export version 5, but version 9 is the latest Cisco invented format and has some advantages for key technologies such as security, traffic analysis and multicast. Without version 9 export format, Flexible NetFlow would not be possible. NetFlow version 5 is a fixed export format which means the data in export version 5 cannot be expanded beyond what is available today. The key to Flexible NetFlow is the ability to allow flexibility in the configuration of the data gathered and monitored by NetFlow. The best method to export a wide range of information from the packet is to use NetFlow version 9, which is a generic format to export any information from a network device. NetFlow version 9 uses a template to describe the data it will export and the data is specifically associated with the template. The NetFlow Version 9 record format consists of a packet header followed by at least one or more template or data FlowSets. A template FlowSet provides a description of the fields that will be present in future data FlowSets . These data FlowSets may occur later within the same export packet or in subsequent export packets. Template and data FlowSets can be intermingled within a single export packet, as illustrated in Figure 8.
Figure 8. NetFlow Version 9 Export Packet
NetFlow version 9 will periodically export the template data, so the collector will understand what data is to be sent and also export the data flow set for the template. The key advantage to Flexible NetFlow is the user configures a Flow Record which is effectively converted to a version 9 template and then forwarded to the collector.
Figure 9 is a detailed example of the NetFlow export format including the header, template and data FlowSets.
Figure 9. Detailed View of a NetFlow Version 9 Export Format
How to Configure Flexible NetFlow Exporter, Record and Flow Monitor?
Configuring Flexible NetFlow can be quite easy and the user can use pre-defined or user defined Flow Records. The first example will demonstrate pre-defined Flow Records and the configuration of traditional NetFlow export using NetFlow version 9. By default, the export version used in Flexible NetFlow is NetFlow version 9.
The key components that need to be configured:
1. Configure the exporter if it is to export to a collector.
2. Configure the Flow Monitor with the pre-defined Flow Record and Flow Exporter attached to the monitor.
3. Add the Flow Monitor to the interface to monitor either ingress (input) or egress (output traffic).
In example one, the Flow Exporter is named "export-to-server" and is attached under the Flow Monitor "my-flow-monitor". Also notice the pre-defined Flow Record chosen is "original-netflow". This record effectively configures traditional NetFlow. Keep in mind many
pre-defined Flow Records are available for the user to configure.
pre-defined Flow Records are available for the user to configure.
Example 1: Predefined Flow Record Configuration
flow exporter export-to-server
destination 172.16.1.1
flow monitor my-flow-monitor
record netflow-orginal
exporter export-to-server
interface Ethernet 1/0
ip flow monitor my-flow-monitor input
In the next example, a user defined Flow Record will be created and customized with the information tracked by Flexible NetFlow. A good example would be the idea outlined earlier in the white paper to monitor how much traffic that will be used per TCP application. In contrast to traditional NetFlow, in this case Flexible NetFlow will monitor just the key fields or information desired, which increases scalability by reducing the number key fields that are necessary for flow creation. In this example, the user also wants to track packets and bytes. The "match" keyword is used to denote the field as a key field and as discussed above, is used to characterize and create flows. The "collect" keyword is used to denote the non-key field and will be used for information that is to be added to the flow, but this information is not used when creating the flow, but exported along with the flow.
The key components that need to be configured:
1. Configure the user defined Flow Record with key and non-key fields
2. Configure the exporter if it is to export to a collector
3. Configure the Flow Monitor with the user defined Flow Record and Flow Exporter attached to the monitor
4. Add the Flow Monitor to the interface to monitor either ingress (input) or egress (output traffic)
Example 2 shows how to configure a user defined Flow Record with the Flow Record created named "app-traffic-analysis" with the Flow Exporter and Flow Monitor named the same as in example 1.
Example 2: User Defined Flow Record Configuration
flow record app-traffic-analysis
description This flow record tracks application usage
match transport tcp destination-port
match transport tcp source-port
match ipv4 destination address
match ipv4 source address
collect counter bytes
collect counter packets
flow exporter export-to-server
destination 172.16.1.1
flow monitor my-flow-monitor
record app-traffic-analysis
exporter export-to-server
interface Ethernet 1/0
ip flow monitor my-flow-monitor input
Another nice feature of Flexible NetFlow is once a user defined Flow Record is created, this record will become available as a pre-defined record that can be easily selected for future use.
What Packet Fields can be Tracked in Flexible NetFlow?
Flexible NetFlow is designed to track information from layer 2 to layer 7 and extract this information from the IP packet and create flows. The following tables outline the key and non-key information that is available in the first release of Flexible NetFlow. There are a large number of fields that can be tracked and different types of applications will utilize a portion of the information available. This information includes routing, transport, and IPv4 packet information. Tables 2 though 6 provide information that can be tracked within Flexible NetFlow.
Table 2. IPv4 Information that is Tracked with Flexible NetFlow
Table 3. IPv4 Routing Fields that are Available in Flexible NetFlow
Table 4. IPv4 Transport Fields that are Available in Flexible NetFlow
Table 5. Other Fields that Can be Tracked in Flexible NetFlow
What Show Commands Can be Used to Track Flexible NetFlow?
The following table 6 is a list of show commands available within Flexible NetFlow. The user can view information including records, interface, exporters and Flow Monitors and flows in the cache.
Table 6. Show Commands Available within Flexible NetFlow
The following is an example show command to view the configured Flow Monitor, record or exporter by parsing the running configuration. The advantage of this command is the output can be cut and pasted directly into the device in configuration mode.
R3#show run flow ?
exporter Show Flow Exporter configuration
monitor Show Flow Monitor configuration
record Show Flow Record configuration
This command will parse the running config and show the details of a Flow Monitor:
R3#show run flow monitor My-flow-monitor
Building configuration...