Every attack leaves a trail that can be used to identify subsequent attempts to perform the same or a similar attack. The trail is left across different log files. These files are produced by the attacked device and various network devices that the malicious traffic traverses. To investigate the malicious activity, administrators need only know where to look and what to look for within the corresponding log files. Using these traces, an administrator can increase security from already deployed routers and find traces of compromises even in cases where a Cisco Intrusion Prevention System (IPS) does not have corresponding signatures.
For purposes of this paper, Cisco Security Intelligence Operations analysts observed an exploit of a vulnerability in Microsoft Internet Explorer using two different devices—a Cisco IPS and a router. By viewing the log entries of the IPS along with the NetFlow records generated by the router, administrators can investigate network traffic anomalies using the IPS, and improve network security using existing capabilities of the router.
The vulnerability that was observed is described in Microsoft Security Bulletin MS08-078 and Cisco IntelliShield alert 17236. The exploit is based on a vulnerability in Internet Explorer that is triggered while processing XML tags in a web page. The following briefly describes the phases of the behavior of the exploit:
- After the user visits a malicious web page, malicious XML code is executed, causing a compromise of Internet Explorer.
- Additional malicious code is downloaded and executed from several different IP addresses. Overall, 29 files from four different hosts were downloaded.
- Malicious code opens a port that enables the attacker to gain remote access to the compromised computer.
Note: For purposes of this paper, host names will be referenced as hostA, hostB, hostC, and hostD.
Cisco IPS is a device that inspects each network packet to identify a unique pattern within the packet. The pattern is called a signature, which corresponds to a particular exploit or malware. Cisco IPS devices can be monitored and controlled by Cisco IPS Manager Express, which is the software used for the basis of this paper.
A flow is identified as a unidirectional stream of packets between a given source and destination—both are defined by a network-layer IP address and transport-layer port numbers. Specifically, a flow is identified as the combination of the following seven key fields:
- Source IP address
- Destination IP address
- Source port number
- Destination port number
- Layer 3 protocol type
- Type of Service byte
- Input logical interface (ifIndex)
These fields define a unique flow. If a flow contains a field that is different from another flow, it is considered a new flow. Flows are processed in a NetFlow cache.
For TCP flows, a router can distinguish when a flow starts and ends by way of TCP session flags. A UDP flow includes all packets that satisfy the criteria for a flow and where the time interval between two successive packets is less than 15 seconds. A new flow is created if more than 15 seconds lapses between two UDP packets even if they satisfy all flow criteria.
If the Cisco IPS has a signature for a particular exploit, the IPS log will show exactly which threat was encountered, as seen in the following example.
Note: The 10.1.1.x are the targeted IP addresses, and the 10.2.2.x are the IP addresses of the attacker.
"high","01/28/2009","01:09:30","IPS_device","Windows RPC DCOM Overflow","3327/12","10.1.1.1","10.2.2.2","deniedFlow","135","60","95"
For purposes of this document, it is assumed that Cisco IPS does not have the signature for this exploit installed. Under this assumption, the following phases of the exploit will not be logged by Cisco IPS.
- A user visits a malicious website and Microsoft Internet Explorer executes malicious code.
- The attacker leverages malicious code to download additional malware from several IP addresses. These actions are not logged because they look like any other HTTP session.
- The compromised computer opens a port that allows the attacker to log in to the computer and control it. Logging in to the compromised computer will not be recorded by Cisco IPS because it is indistinguishable from legitimate access.
Typically, after a compromise occurs, the attacker will scan the network and attempt to compromise more computers either within or outside of the organization. The Cisco IPS, however, will recognize scanning as a malicious activity and generate the following log trail:
"low","01/28/2009","01:09:30","IPS_device","ICMP Network Sweep w/Echo","2100","10.1.1.1","10.2.2.2"
Depending on the type of scan that the attacker performs, additional log entries may appear, including the following:
- TCP SYN Host Sweep
- AD - External TCP Scanner Single Scanner
- AD - External Other Scanner Single Scanner
Until Cisco IPS has the correct signature in place, it will not recognize the exploit. However, the IPS will flag some of the deviant behavior of the compromised computer, enabling a more detailed investigation into the attack.
NetFlow can be configured in two ways: to export all or selected flows. When NetFlow exports selected flows, it is called sampled NetFlow. Sampled NetFlow can export every nth flow that it records, for example, every 100th or 10,000th flow. Sampled NetFlow is useful when addressing very high bandwidths. In the following example, sampled NetFlow is not used. Instead, each flow is recorded and exported in order to examine traces for each phase of the compromise.
To produce NetFlow outputs, the NFDUMP tool was used. This tool is free and available at http://nfdump.sourceforge.net/. Documentation about how to use this tool is available at the same website. Note that not all flows will be shown, only representative samples. The three phases of the exploit are recorded by the router as seen in the following sets of NetFlow records.
- A user visits a malicious website. The visit is recorded as two flows—one flow originating from the internal host to an external host and one flow originating from the external host back to the internal host. The following traces resulted:
Date flow start Duration Proto Src IP Addr:Port Dst IP Addr:Port Packets Bytes Flows 08-12-09 13:06:46.343 7.620 TCP 10.1.1.1:1153->10.2.2.3:80 9 863 1 08-12-09 13:06:46.551 7.412 TCP 10.2.2.3:80->10.1.1.1:1153 8 8407 1
2. The attacker-supplied malicious code downloads additional malware from several IP addresses. NetFlow logs capture this activity as the following:
Date flow start Duration Proto Src IP Addr:Port Dst IP Addr:Port Packets Bytes Flows [contacting the hostB] 2008-12-09 13:39:35.606 0.448 TCP 10.1.1.1:1353->10.2.2.1:80 5 510 1 2008-12-09 13:39:35.826 0.448 TCP 10.2.2.1:80->10.1.1.1:1353 5 360 1 [contacting the hostC] 2008-12-09 13:45:35.495 17.492 TCP 10.2.2.4:80->10.1.1.1:1496 5 1107 1 2008-12-09 13:45:35.255 17.888 TCP 10.1.1.1:1496->10.2.2.4:80 10 1740 1 2008-12-09 13:46:57.983 0.000 TCP 10.1.1.1:1496->10.2.2.4:80 1 40 1 2008-12-09 16:45:37.764 0.252 TCP 10.2.2.4:80->10.1.1.1:4292 2 392 1 2008-12-09 16:45:37.516 0.676 TCP 10.1.1.1:4292->10.2.2.4:80 4 462 1 2008-12-09 16:46:38.028 0.000 TCP 10.1.1.1:4292->10.2.2.4:80 1 40 1 [contacting the hostD] 2008-12-09 13:45:53.811 0.444 TCP 10.1.1.1:1515->10.2.2.5:80 4 263 1 2008-12-09 13:45:48.879 1.648 TCP 10.1.1.1:1505->10.2.2.5:80 18 932 1 2008-12-09 13:45:49.087 1.288 TCP 10.2.2.5:80->10.1.1.1:1505 27 38125 1 2008-12-09 13:45:54.027 0.224 TCP 10.2.2.5:80->10.1.1.1:1515 2 1256 1 2008-12-09 13:45:57.551 1.128 TCP 10.1.1.1:1525->10.2.2.5:80 10 648 1 2008-12-09 13:45:57.759 0.920 TCP 10.2.2.5:80->10.1.1.1:1525 12 15792 1 2008-12-09 13:46:32.227 17.796 TCP 10.1.1.1:1525->10.2.2.5:80 31 1700 1 2008-12-09 13:46:52.983 0.000 TCP 10.1.1.1:1505->10.2.2.5:80 1 40 1 [additional flows removed]
In the preceding example, flows are grouped as going to and from a particular host to increase clarity. In practice, the logs are sorted in chronological order; therefore, conversations between different hosts will be mixed.
3. The compromised system opens a port that allows the attacker to control it. This step will not leave any traces in the NetFlow logs if there is no network activity.
After the compromise, the compromised system scans the network, accepts incoming connections from the attacker, and sends data to the attacker. These actions are visible in the NetFlow logs as seen in the following example:
ICMP “Echo request” ('ping') Date flow start Duration Proto Src IP Addr:Port Dst IP Addr:Port Packets Bytes Flows 2009-02-05 04:58:59.518 10.000 ICMP 10.1.1.1:0->10.2.2.6:8.0 11 924 1 2009-02-05 04:58:59.518 10.000 ICMP 10.2.2.6:0->10.1.1.1:0.0 11 924 1
The following logs show unsuccessful attempts to establish Telnet, SSH, and Web connections respectively:
Date flow start Duration Proto Src IP Addr:Port Dst IP Addr:Port Packets Bytes Flows 2009-02-05 04:59:32.934 0.000 TCP 10.1.1.1:34543->10.2.2.6:23 1 60 1 2009-02-05 04:59:39.730 0.000 TCP 10.1.1.1:34545->10.2.2.6:22 1 60 1 2009-02-05 04:59:46.166 0.000 TCP 10.1.1.1:34547->10.2.2.6:80 1 60 1
Cisco IPS cannot directly recognize a compromise until it has an appropriate signature loaded, but the IPS can recognize certain types of suspicious activities that are independent of a specific signature, such as scanning. Administrators can use these traces as a signal for closer inspection of a host to determine if it has been compromised.
NetFlow records every network activity, but NetFlow alone does not offer any interpretation of the meaning of an event. Administrators can monitor for unusual activity and use that as a trigger to examine a host. Knowing the expected behavior of the host is the goal. After administrators establish that a host has been compromised, they can review the origin of the initial compromise, which allows them to screen other activity that is related to the malicious host.
The Cisco IPS and router are just two examples that demonstrate how additional value can be gained from existing devices. Instead of a router, the Cisco Catalyst switch could have been used, or any other device that supports NetFlow as the source of NetFlow information. Without additional investment, administrators can creatively use existing infrastructure to improve the security of their network.
Damir Rajnovic (firstname.lastname@example.org)
Incident Manager, Cisco PSIRT
Damir joined Cisco in 1999; his current focus is security industry collaboration and standardization.
Cisco IPS Manager Express
This document is part of Cisco Security Research & Operations.
This document is provided on an "as is" basis and does not imply any kind of guarantee or warranty, including the warranties of merchantability or fitness for a particular use. Your use of the information on the document or materials linked from the document is at your own risk. Cisco reserves the right to change or update this document at any time.