Configuring MPLS-aware NetFlow

MPLS-aware NetFlow is an extension of the NetFlow accounting feature that provides highly granular traffic statistics for Cisco routers. MPLS-aware NetFlow collects statistics on a per-flow basis just as NetFlow does.

A flow is a unidirectional set of packets (IP or MPLS) that arrive at the router on the same subinterface, have the same source and destination IP addresses, the same Layer 4 protocol, the same TCP/UDP source and destination ports, and the same type of service byte in the IP header.

An MPLS flow contains up to three of the same incoming MPLS labels of interest with experimental bits and end-of-stack bits in the same positions in the packet label stack. MPLS-aware NetFlow captures MPLS traffic that contains both IP and non-IP packets. It reports non-IP packets, but sets the IP NetFlow fields to 0. It can also be configured to capture and report IP packets, setting to 0 the IP NetFlow fields. MPLS-aware NetFlow uses the NetFlow Version 9 export format. MPLS-aware NetFlow exports up to three labels of interest from the incoming label stack, the IP address associated with the top label, and traditional NetFlow data.

MPLS-aware NetFlow statistics can be used for detailed MPLS traffic studies and analysis that can provide information for a variety of purposes such as MPLS network management, network planning, and enterprise accounting.

A network administrator can turn on MPLS-aware NetFlow inside an MPLS cloud on a subset of provider backbone (P) routers. These routers can export MPLS-aware NetFlow data to an external NetFlow collection device for further processing and analysis or you can display NetFlow cache data on a router terminal.

As packets move through an MPLS network, LSRs can add labels to the MPLS label stack. LSRs in an MPLS cloud can add up to six labels to the MPLS label stack. An LSR adds the MPLS labels to the top of the IP packet. The figure below shows an example of an incoming MPLS label stack that LSRs added to an IP packet as it traversed an MPLS cloud.

Figure 1

Example of an MPLS Label Stack Added to an IP Packet in an MPLS Cloud

In the example of an MPLS label stack in the figure above:

• The 33 represents the top label of this packet.

This label was the last label added to the MPLS label stack and the label that MPLS-aware NetFlow captures if you indicate the label of interest as 1.

• The 42 represents the second label in the MPLS stack.

MPLS-aware NetFlow captures this label if you indicate 2 (second from the top) as a label of interest.

• The 16 represents the third label in the MPLS label stack.

MPLS-aware NetFlow captures this label if you indicate 3 (third from the top) as a label of interest.

• Lb4-Lb6 represents the fourth to sixth labels in the MPLS stack. LSRs in an MPLS cloud add up to six labels to the MPLS label stack.

MPLS-aware NetFlow captures these labels if you indicate 4, 5, or 6 as labels of interest.

• The B represents miscellaneous bits. These include the following:
  • Exp--Three bits reserved for experimental use
  • S--End-of-stack bits, set to 1 for the last entry in the stack and to 0 for every other entry
  • Time to Live (TTL)--Eight bits used to encode a hop count (or time to live) value

The figure below shows a sample Carrier Supporting Carrier (CSC) topology and the incoming MPLS label stack on multiple LSRs as the packet travels through the network. The figure shows what the stack might look like at a provider core LSR.

Figure 2

Provider and Customer Networks and MPLS Label Imposition

In the example in the figure above, a hierarchical VPN is set up between two customer edge (CE) routers.

• Traffic flows from the CE router to a provider edge (PE) router, possibly one belonging to an Internet service provider (ISP). Here, a VPN label (16) is imposed on the inbound IP packet.
• The ISP network eventually connects to an Internet backbone provider where a CSC label (42) is imposed on the label stack.
• As packets traverse the backbone network, a Label Distribution Protocol (LDP) label (33) is imposed on the label stack.

At the inbound interface shown in the figure above, MPLS-aware NetFlow captures the MPLS label stack and reports that the top label (33) is an LDP label, the second label (42) is a CSC label, and the third label (16) is a VPN label.

With NetFlow and MPLS-aware NetFlow enabled on the P router, you can determine the label type for the specified labels, and the IP address associated with the top label on the incoming interface (see the MPLS-aware NetFlow Capture of MPLS Labels). Thus, you can track specific types of MPLS traffic, such as TE, LDP, or VPNs.

When you configure the MPLS-aware NetFlow feature, you select the MPLS label positions in the incoming label stack that you are interested in monitoring. You can capture up to three labels from positions 1 to 6 in the MPLS label stack. Label positions are counted from the top of the stack. For example, the position of the top label is 1, the position of the next label is 2, and so on. You enter the stack location value as an argument to the following command:

ip flow-cache mpls label-positions
 
[label-position-1 [label-position-2 [label-position-3]]]
 

The label-position-n argument represents the position of the label on the incoming label stack. For example, the ip flow-cache mpls label-positions 1 3 4 command configures MPLS-aware NetFlow to capture and export the first (top), third, and fourth labels. If you enter this command and the label stack consists of two MPLS labels, MPLS-aware NetFlow captures only the first (top) label. If some of the labels you requested are not available, they are not captured or reported.

In addition to capturing MPLS labels from the label stack, MPLS-aware NetFlow records the following MPLS label information:

• Type of top label--The type can be any of the following: unknown, TE tunnel midpoint, AToM, VPN, BGP, or LDP.
• The IP address associated with the top label--The route prefix to which the label maps.

Note	The IP address for any TE tunnel midpoint or AToM top label is reported as 0.0.0.0.

MPLS-aware NetFlow is enabled globally on the router. However, NetFlow is enabled per interface and must be enabled in either full or sampled mode on the interfaces where you choose to capture and export MPLS and IP NetFlow data.

Note	See the table below for information about Cisco 12000 series Internet router line card support for NetFlow (full and sampled modes).

The MPLS-aware NetFlow feature allows the display of a snapshot of the NetFlow cache, including MPLS flows, on a terminal through the use of the show ip cache verbose flow command. For example, the following output from a provider core router (P router) shows position, value, experimental bits, and end-of-stack bit for each MPLS label of interest. It also shows the type of the top label and the IP address associated with the top label.

SrcIf          SrcIPaddress    DstIf          DstIPaddress    Pr TOS Flgs  Pkts
Port Msk AS                    Port Msk AS    NextHop              B/Pk  Active
PO3/0          10.1.1.1        PO5/1          10.2.1.1        01 00  10       9
0100 /0  0                     0200 /0  0     0.0.0.0               100     0.0
Pos:Lbl-Exp-S 1:12305-6-0 (LDP/10.10.10.10) 2:12312-6-1

In this example from a P router:

• The value of the top label is 12305.
• The experimental bits value is 6 and the end-of-stack bit is 0.
• The label type is LDP and the IP address associated with the label is 10.10.10.10.
• The value of the second label is 12312, the experimental bits value is 6, and the end-of-stack bit is 1.

To fully understand and use the information gathered on the P router, you need information from the Label Forwarding Information Base (LFIB) on the PE router.

Note

The MPLS application owner for a label is not reported by MPLS-aware NetFlow for any MPLS label except for the top label. IP information, the label number, and the MPLS application are reported for the top label. Only IP information and the label number are reported for labels other than the top label. Therefore, you need to understand your network if you are interested in identifying the MPLS application owner for labels other than the top MPLS label.

Using MPLS-aware NetFlow, you can monitor various labels in the MPLS label stack. You can also export this information to a NetFlow collector for further processing with a data analyzer and look at MPLS traffic patterns in your network.

MPLS-aware NetFlow captures and reports on other information in addition to MPLS labels. It provides per-flow statistics for both incoming IP and MPLS traffic.

• For MPLS traffic, MPLS-aware NetFlow captures and reports up to three labels of interest and the label type and associated IP address of the top label, along with a subset of NetFlow data.
• For IP traffic, MPLS-aware NetFlow provides the regular NetFlow data.
• MPLS-aware NetFlow uses the Version 9 format to export both IP and MPLS NetFlow data.

MPLS-aware NetFlow provides the following traditional NetFlow per-flow statistics:

• Number of packets
• Number of bytes, counting either MPLS payload size only or MPLS payload size plus MPLS label stack size
• Time stamp of the first packet
• Time stamp of the last packet

In addition to these statistics, MPLS-aware NetFlow exports values for the following fields for each flow, using the Version 9 NetFlow export format:

• Regular NetFlow fields:
  • Source IP address
  • Destination IP address
  • Transport layer protocol
  • Source application port number
  • Destination application port number
  • IP ToS
  • TCP flags
  • Input interface
  • Output interface

Note	With the exception of the input interface and output interface fields, these regular NetFlow fields are not included in a flow if the no-ip-fields keyword is specified in the ip flow-cache mpls label-positions command.

• Additional fields:
  • Up to three incoming MPLS labels with experimental bits and an end-of-stack bit
  • Positions of the MPLS labels in the label stack
  • Type of the top label
  • An address prefix associated with the top label specific to the label type: TE--This is always set to "0.0.0.0" because tunnel label addresses are not supported. LDP--The address prefix is the IP address of the next-hop. VPN--If the VRFs do not have overlapping IP addresses, the address prefix is the destination prefix. If the VRFs have overlapping IP addresses the destination prefix given may be ambiguous.

Note	Unlike NetFlow, MPLS-aware NetFlow reports a 0 value for IP next-hop, source, and destination BGP autonomous system numbers, or source and destination prefix masks for MPLS packets.

Note	If you are exporting MPLS data to a NetFlow collector or a data analyzer, the collector must support the NetFlow Version 9 flow export format, and you must configure NetFlow export in Version 9 format on the router.

The table below shows full and sampled MPLS-aware NetFlow support. Information in the table is based on the Cisco IOS release and includes the commands to implement the functionality on a supported platform.

1.    enable

2.    configure terminal

3.    interface type /number

4.    ip flow {ingress}

5.    exit

6.    Repeat Steps 3 through 5 for each interface you want to configure NetFlow on.

7.    ip flow-export version 9 [origin-as | peer-as][bgp-nexthop]

8.    ip flow-cache mpls label-positions [label-position-1 [label-position-2 [label-position-3]]] [no-ip-fields] [mpls-length]

9.    exit

1.    enable

2.    configure terminal

3.    flow-sampler-map sampler-map-name

4.    mode random one-out-of packet-interval

5.    exit

6.    interface type / number

7.    flow-sampler sampler-map-name

8.    end

• Troubleshooting Tips
  

1.    show flow-sampler [sampler-map-name]

Router# show flow-sampler mysampler
Sampler : mysampler, id : 1, packets matched : 10, mode : random sampling mode
  sampling interval is : 100

Router# show flow-sampler
Sampler : mysampler, id : 1, packets matched : 10, mode : random sampling mode
  sampling interval is : 100
 Sampler : mysampler1, id : 2, packets matched : 5, mode : random sampling mode 
  sampling interval is : 200

1.    enable

2.   

3.    show ip cache verbose flow

4.    show ip cache flow

5.    exit (Cisco 12000 series routers only)

Router> enable

Router# attach 3

Router# if-con 3

Router# show ip cache verbose flow
SrcIf          SrcIPaddress    DstIf          DstIPaddress    Pr TOS Flgs  Pkts
Port Msk AS                    Port Msk AS    NextHop              B/Pk  Active
PO3/0          10.1.1.1        PO5/1          10.2.1.1        01 00  10       9
0100 /0  0                     0200 /0  0     0.0.0.0               100     0.0
Pos:Lbl-Exp-S 1:12305-6-0 (LDP/10.10.10.10) 2:12312-6-1

Router# show ip cache verbose flow
...             
SrcIf          SrcIPaddress    DstIf          DstIPaddress    Pr TOS Flgs  Pkts
Port Msk AS                    Port Msk AS    NextHop              B/Pk  Active
PO3/0          10.1.1.1        PO5/1          10.2.1.1        01 00  10       9
0100 /0  0                     0200 /0  0     0.0.0.0               100     0.0
Pos:Lbl-Exp-S 1:12305-6-0 (LDP/10.10.10.10) 2:12312-6-1

Router# show ip cache flow
...
SrcIf         SrcIPaddress    DstIf         DstIPaddress    Pr SrcP DstP  Pkts
PO3/0         10.1.1.1        PO5/1         10.2.1.1        01 0100 0200     9
...

Router# show ip cache flow
...
SrcIf         SrcIPaddress    DstIf         DstIPaddress    Pr SrcP DstP  Pkts
PO3/0         10.1.1.1        PO5/1         10.2.1.1        01 0100 0200     9
...

Router# exit

Router# if-quit

• Defining the NetFlow Sampler
  
• Applying the NetFlow Sampler to an Interface