Contents
Introduction
Log Messages of Interest
Cisco ASA Firewall
Cisco Router
Command-Line Syntax
Cisco ASA
Cisco Router
Quick Identification and Impetus
Methodology
Trends
Summary
References
Syslog messages from transit network devices can provide insight into and context for security events that may not be available from other sources. This insight aids in determining the validity and extent of an incident. Within the context of a security incident, administrators can use syslog messages to understand communication relationships, timing, and, in some cases, the attacker's motives and/or tools. These events should be considered complementary and should be used in conjunction with other forms of network monitoring that may already be in place.
The focus of this paper is the analysis of data contained in the memory buffer of the device, using tools native to the device itself, after a trigger event has occurred. This kind of analysis may require an administrator to obtain additional information about the event. This paper does not provide recommendations for logging configurations or tools that provide automated analysis. In most cases, historical analysis will be performed on logging-specific servers, where tools and syntax may be similar. It should be noted that if logging servers exist, using these techniques on such servers would be preferable. However, because of the varied nature of implementations, these examples will focus on using native Cisco IOS Software capabilities and locally available syslog information. References are also provided to aid in automation or syntactical subtleties.
For the purpose of this guide, Cisco Adaptive Security Appliance (ASA) software version 7.2 will be used for firewall examples and Cisco IOS Software version 12.3 will be the primary IOS version used for router examples, although the ACL Syslog Correlation feature requires Cisco IOS Software 12.4(22)T or later. Syslog messages can be one of eight predefined severity levels. The following table briefly summarizes the different severity logging levels:
| Level | System | Description |
|---|---|---|
| Emergency | 0 | System unusable messages |
| Alert | 1 | Immediate action required messages |
| Critical | 2 | Critical condition messages |
| Error | 3 | Error condition messages |
| Warning | 4 | Warning condition messages |
| Notification | 5 | Normal but significant messages |
| Information | 6 | Informational messages |
| Debugging | 7 | Debugging messages |
Although all log messages can be of use in certain circumstances, in most cases a small subset of log messages will initially provide the most benefit. After these events have been examined, administrators can expand the scope of their analysis by searching for additional details. The following table summarizes the most common messages and the associated severity level. Fortunately, most of these messages come from fairly contiguous mnemonic identifiers, which aid in identification when using command-line tools.
| Mnemonic | Severity | Description |
|---|---|---|
| 4000nn ("nn" indicates multiple messages currently 400000 - 400050) | 4 | IPS:number string from IP_address to IP_address on interface interface_name |
| 106001 | 2 | Inbound TCP connection denied from IP_address/port to IP_address/port flags tcp_flags on interface interface_name |
| 106002 | 2 | protocol Connection denied by outbound list acl_ID src inside_address dest outside_address |
| 106006 | 2 | Deny inbound UDP from outside_address/outside_port to inside_address/inside_port on interface interface_name |
| 106007 | 2 | Deny inbound UDP from outside_address/outside_port to inside_address/inside_port due to DNS {Response|Query} |
| 106010 | 3 | Deny inbound protocol src interface_name:dest_address/dest_port dst |
| 106012 | 3 | Deny IP from IP_address to IP_address, IP options hex |
| 106013 | 3 | Dropping echo request from IP_address to PAT address IP_address |
| 106014 | 3 | Deny inbound icmp src interface_name: IP_address dst interface_name: IP_address (type dec, code dec) |
| 106015 | 6 | Deny TCP (no connection) from IP_address/port to IP_address/port flags tcp_flags on interface interface_name |
| 106016 | 2 | Deny IP spoof from (IP_address) to IP_address on interface interface_name. |
| 106017 | 2 | Deny IP due to Land Attack from IP_address to IP_address |
| 106018 | 2 | ICMP packet type ICMP_type denied by outbound list acl_ID src inside_address dest outside_address |
| 106020 | 2 | Deny IP teardrop fragment (size = number, offset = number) from IP_address to IP_address |
| 106021 | 1 | Deny protocol reverse path check from source_address to dest_address on interface interface_name |
| 106022 | 1 | Deny protocol connection spoof from source_address to dest_address on interface interface_name |
| 106023 | 4 | Deny protocol src [interface_name:source_address/source_port] dst interface_name:dest_address/dest_port [type {string}, code {code}] by access_group acl_ID |
| 106100 | 4 | access-list acl_ID {permitted | denied | est-allowed} protocol interface_name/source_address(source_port) -> interface_name/dest_address(dest_port) hit-cnt number ({first hit | number-second interval}) |
| 710003 | 3 | {TCP|UDP} access denied by ACL from source_IP/source_port to interface_name:dest_IP/service |
Note: Basic IPS support needs to be enabled using the ip audit feature in order for log messages 400000 - 400050 (4000nn) to be generated. These messages do not require the use of the AIP SSM IPS features. Information on configuring the ip audit feature is available in the Cisco Security Appliance Command Reference, Version 7.2.
In addition to the messages in the preceding table, several other connection-related messages of severity levels 6 (informational) and 7 (debug) are commonly used during analysis. Logging at severity levels 6 and 7 will have a performance impact.
As with the Cisco ASA, a large number of log messages may be useful on Cisco IOS Software. However, in most cases a small subset that will need to be examined more often. Router log messages do not contain numerical identifiers that assist in identifying the messages. The following is a list of router log messages that are most likely to be useful when analyzing security-related incidents. Because many organizations do not make extensive use of logging on routers and because router logging is somewhat limited, NetFlow is often a more effective means of analysis.
| Mnemonic | Severity | Description |
|---|---|---|
| %SEC-6-IPACCESSLOGDP | 6 | A packet matching the log criteria for the given access list has been detected. |
| %SEC-6-IPACCESSLOGNP | 6 | A packet matching the log criteria for the given access list has been detected. |
| %SEC-6-IPACCESSLOGP | 6 | A packet matching the log criteria for the given access list has been detected (TCP or UDP) |
| %SEC-6-IPACCESSLOGRL | 6 | Some packet-matching logs were missed because the access list log messages were rate limited, or no access list log buffers were available. |
| %SEC-6-IPACCESSLOGRP | 6 | A packet matching the log criteria for the given access list has been detected. |
| %SEC-6-IPACCESSLOGS | 6 | A packet matching the log criteria for the given access list was detected. |
| %SEC-4-TOOMANY | 4 | The system was not able to process the packet because there was not enough room for all of the desired IP header options. The packet has been discarded. |
| %IPV6-6-ACCESSLOGP | 6 | A packet matching the log criteria for the given access list was detected. |
| %IPV6-6-ACCESSLOGDP | 6 | A packet matching the log criteria for the given access list was detected. |
| %IPV6-6-ACCESSLOGNP | 6 | A packet matching the log criteria for the given access list was detected. |
Administrators can use the show logging command with several different keywords to find relevant log messages. In most cases, the grep command followed by a regular expression will yield the most flexibility and best results. It should be noted that often several different regular expressions may be used to accomplish the same task. These examples are for messages stored in the logging buffer locally on the device itself. The size of the buffer can be modified using the logging buffer size size in bytescommand.
More information on the specific command syntax and regular expressions is available in Using the Command Line Interface. To find messages of a specific severity (e.g. ASA-4), administrators can use grep as follows:
Firewall# show logging | grep ASA-4 Aug 24 2007 08:54:31: %ASA-4-500004: Invalid transport field for protocol=TCP, from 192.168.208.63/46855 to
192.168.150.77/0 Aug 24 2007 08:54:31: %ASA-4-500004: Invalid transport field for protocol=TCP, from 192.168.208.63/46856 to
192.168.150.77/0 Aug 24 2007 08:54:48: %ASA-4-106023: Deny tcp src outside:192.168.208.63/46857 dst inside:192.168.150.77/443
by access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 24 2007 08:54:48: %ASA-4-106023: Deny tcp src outside:192.168.208.63/46863 dst inside:192.168.150.77/256
by access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 24 2007 08:54:48: %ASA-4-106023: Deny tcp src outside:192.168.208.63/46867 dst inside:192.168.150.77/389
by access-group "OUTSIDE" [0x5063b82f, 0x0]
To find a specific logging message (for example, 106023), grep can be used as follows:
Firewall# show logging | grep 106023 Aug 24 2007 08:54:49: %ASA-4-106023: Deny tcp src outside:192.168.208.63/47511 dst inside:192.168.150.77/352 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 24 2007 08:54:49: %ASA-4-106023: Deny tcp src outside:192.168.208.63/47513 dst inside:192.168.150.77/167 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 24 2007 08:54:49: %ASA-4-106023: Deny tcp src outside:192.168.208.63/47517 dst inside:192.168.150.77/210 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 24 2007 08:54:49: %ASA-4-106023: Deny tcp src outside:192.168.208.63/47522 dst inside:192.168.150.77/281 by
access-group "OUTSIDE" [0x5063b82f, 0x0]
To find more than one specific logging message, administrators can use regular expressions with the grep command as follows:
Firewall# show logging | grep (106023|106100) Aug 24 2007 09:02:37: %ASA-4-106023: Deny tcp src outside:192.168.208.63/51584 dst inside:192.168.150.77/239 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 24 2007 09:02:37: %ASA-4-106023: Deny tcp src outside:192.168.208.63/51585 dst inside:192.168.150.77/288 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 24 2007 09:02:37: %ASA-4-106023: Deny tcp src outside:192.168.208.63/51586 dst inside:192.168.150.77/147 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 24 2007 09:02:37: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.208.63(51587) ->
inside/192.168.150.77(104) hit-cnt 1 first hit [0x22e8ac21, 0x0]
The grep command and regular expressions can be extended to find a group of log messages. In the following example, the syslog messages ranging from 1060nn to 1062nn will be displayed using grep and the respective regular expressions:
Firewall# show logging | grep 106[0-2][0-9][0-9]: Aug 24 2007 09:14:54: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.208.63(38807) ->
inside/192.168.150.77(80) hit-cnt 1 first hit [0x22e8ac21, 0x0] Aug 24 2007 09:16:14: %ASA-6-106015: Deny TCP (no connection) from 192.168.150.65/2278 to 64.101.128.83/80
flags RST on interface inside Aug 24 2007 09:16:41: %ASA-4-106023: Deny icmp src outside:192.168.208.63 dst inside:192.168.150.77
(type 8, code 0) by access-group "OUTSIDE" [0xd3f63b90, 0x0] Aug 24 2007 09:16:41: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.208.63(38664) ->
inside/192.168.150.77(80) hit-cnt 1 first hit [0x22e8ac21, 0x0] Aug 24 2007 09:16:43: %ASA-4-106023: Deny icmp src outside:192.168.208.63 dst inside:192.168.150.77
(type 8, code 0) by access-group "OUTSIDE" [0xd3f63b90, 0x0] Aug 24 2007 09:16:43: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.208.63(38665) ->
inside/192.168.150.77(80) hit-cnt 1 first hit [0x22e8ac21, 0x0] Aug 24 2007 09:17:32: %ASA-1-106021: Deny ICMP reverse path check from 192.168.150.60 to 192.168.2.1 on
interface outside
As with the Cisco ASA, the show logging command combined with the use of certain keywords and search-specific filters is the most effective means of finding relevant information. It should be reiterated that the logging capabilities of IOS routers are much more limited than those of the Cisco ASA. The grep keyword does not exist in this context, so instead administrators must use the include keyword. In addition, either the log or log-input keyword must be present on each access control entry (ACE) that should generate logging messages. If neither log nor log-input is present on a given ACE, no log messages will be generated for it.
The following example (access list 185) will demonstrate how those log messages will be displayed. In this example, TCP ports 53 through 123 will not be logged, TCP ports 137 through 445 will use the log keyword, and TCP ports 500 through 1024 will use the log-input keyword. For this example, the access list counter and logging buffer will be cleared and connections will then be created to each port group. The router response demonstrates the different results that are obtained when using no keyword, the log keyword, or the log-input keyword. The first part of the example shows the access list counters, whereas the second part of the example shows the available logging messages.
For more information on the use of the log and log-input keywords, refer to the white paper Understanding Access Control List Logging.
Access List Counters
Router#show access-lists 185 Extended IP access list 185 10 deny tcp 172.16.1.0 0.0.0.255 host 192.168.2.1 range domain 123 (284 matches) 20 deny tcp 172.16.1.0 0.0.0.255 host 192.168.2.1 range 137 445 log (1236 matches) 30 deny tcp 172.16.1.0 0.0.0.255 host 192.168.2.1 range 500 1024 log-input (1574 matches) 40 permit ip any any (1550 matches) Router#
Available Logging Messages
For the logging messages that follow, note that there is no logging for ACE sequence ID 10 of access list 185 because a logging keyword was not used in this ACE. Note also the different syntax of the logs for which the log keyword was used (messages 2092 through 2094), versus those for which the log-input keyword was used (messages 2095 through 2097).
Router#show logging | include 185 002092: Mar 30 2010 11:41:48.681 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.92(59078) ->
192.168.2.1(417), 1 packet 002093: Mar 30 2010 11:41:49.681 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.95(14897) ->
192.168.2.1(427), 1 packet 002094: Mar 30 2010 11:41:50.681 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.182(16737) ->
192.168.2.1(437), 1 packet 002095: Mar 30 2010 11:41:56.985 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.219(14872)
(FastEthernet0/1 0007.8580.9edd) -> 192.168.2.1(500), 1 packet 002096: Mar 30 2010 11:41:57.984 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.208(7751)
(FastEthernet0/1 0007.8580.9edd) -> 192.168.2.1(510), 1 packet 002097: Mar 30 2010 11:41:58.984 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.26(41202)
(FastEthernet0/1 0007.8580.9edd) -> 192.168.2.1(520), 1 packet Router#
Regular Expressions
Even though Cisco IOS Software does not support the grep command in this context, regular expressions can still be used to identify potential intrusions. In the following example, a regular expression was used with the include keyword to identify logs for which the source IP address was 172.16.1.92 and the destination port was 137.
Router#show logging | include 172.16.1.92.*\-\>.*\(137\) 002064: Mar 30 2010 11:41:20.659 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.92(17587) ->
192.168.2.1(137), 1 packet 002065: Mar 30 2010 11:41:21.659 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.92(58564) ->
192.168.2.1(137), 1 packet 002067: Mar 30 2010 11:41:23.663 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.92(17755) ->
192.168.2.1(137), 1 packet Router#
Selecting Log Messages Generated by a Specific ACE
Introduced with Cisco IOS release 12.4(22)T, the ACL Syslog Correlation feature allows for every log message generated by an ACE to include a cookie. The value of the cookie can be either a system-generated hash or a user-defined string. The global configuration command ip access-list logging hash-generation enables the automatic generation of an MD5 hash value to be used as a cookie. The following is access list 185 after enabling this feature:
Router#show access-lists 185 Extended IP access list 185 10 deny tcp 172.16.1.0 0.0.0.255 host 192.168.2.1 range domain 123 (355 matches) 20 deny tcp 172.16.1.0 0.0.0.255 host 192.168.2.1 range 137 445 log (1432 matches) (hash = 0x279C8521) 30 deny tcp 172.16.1.0 0.0.0.255 host 192.168.2.1 range 500 1024 log-input (1574 matches) (hash = 0xC85A617C) 40 permit ip any any (1776 matches) Router#
ACE sequence ID 10 on access list 185 did not receive a cookie because it does not include the log or log-input keyword. The following is an example of the syslog messages that were generated when the ACL Syslog Correlation feature was enabled:
Router#show logging | include 185 002416: Mar 30 2010 11:51:07.149 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.14(11995) ->
192.168.2.1(418), 1 packet [0x279C8521] 002417: Mar 30 2010 11:51:08.153 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.14(7331) ->
192.168.2.1(428), 1 packet [0x279C8521] 002418: Mar 30 2010 11:51:09.153 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.49(36426) ->
192.168.2.1(438), 1 packet [0x279C8521] 002419: Mar 30 2010 11:51:15.353 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.229(45012)
(FastEthernet0/1 0007.8580.9edd) -> 192.168.2.1(500), 1 packet [0xC85A617C] 002420: Mar 30 2010 11:51:16.357 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.42(56954)
(FastEthernet0/1 0007.8580.9edd) -> 192.168.2.1(510), 1 packet [0xC85A617C] 002421: Mar 30 2010 11:51:17.357 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.2(25313)
(FastEthernet0/1 0007.8580.9edd) -> 192.168.2.1(520), 1 packet [0xC85A617C]
Referencing the cookie value allows the administrator to select only those syslog messages that were generated by ACE sequence ID 20 on access list 185:
Router#show logging | include 0x279C8521 002415: Mar 30 2010 11:51:06.150 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.72(5775) ->
192.168.2.1(408), 1 packet [0x279C8521] 002416: Mar 30 2010 11:51:07.149 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.14(11995) ->
192.168.2.1(418), 1 packet [0x279C8521] 002417: Mar 30 2010 11:51:08.153 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.14(7331) ->
192.168.2.1(428), 1 packet [0x279C8521] 002418: Mar 30 2010 11:51:09.153 EDT: %SEC-6-IPACCESSLOGP: list 185 denied tcp 172.16.1.49(36426) ->
192.168.2.1(438), 1 packet [0x279C8521] Router#
The impetus for analyzing system logs can come from several sources. Often the motivation can come from network-related issues, hits to classification ACL ACEs, or the need to gather further information to support activity from another source, such IPS signatures. Below is an example from an ACL on the Cisco ASA. For the purpose of this example, a deny ACE will be used to gather more data about the denied events. It is important to note that ACE counters can have a much longer lifetime than log entries that reside in the logging buffer, therefore it is likely that there could be ACL ACE counter numbers without associated logs in the logging buffer. To clear the ACL counters, administrators can use the command clear access-list access list name counters.
Firewall# show access-list OUTSIDE access-list OUTSIDE; 24 elements access-list OUTSIDE line 1 extended deny tcp host 192.168.208.63 host 192.168.150.77 range www 123 log
informational interval 300 (hitcnt=96)0x22e8ac21 access-list OUTSIDE line 2 extended deny tcp host 192.168.208.63 host 192.168.150.77 range netbios-ssn 445
(hitcnt=1842) 0x5063b82f access-list OUTSIDE line 3 extended deny icmp host 192.168.208.63 host 192.168.150.77 (hitcnt=6) 0xd3f63b90
When the impetus is an ACE counter from a classification ACL, it is easiest to use the ACL name to quickly identify entries instead of the logging-specific messages. This will show log messages from the ACL OUTSIDE regardless of whether the logkeyword was used in the ACL. Administrators must take care to use the correct alphanumeric case for the specific ACL, otherwise the Cisco ASA will display many connection-related messages.
Firewall# show logging | grep OUTSIDE Aug 24 2007 09:02:35: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.208.63(51063) ->
inside/192.168.150.77(83) hit-cnt 1 first hit [0x22e8ac21, 0x0] Aug 24 2007 09:02:35: %ASA-4-106023: Deny tcp src outside:192.168.208.63/51064 dst inside:192.168.150.77/214 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 24 2007 09:02:35: %ASA-4-106023: Deny tcp src outside:192.168.208.63/51065 dst inside:192.168.150.77/260 by
access-group "OUTSIDE" [0x5063b82f, 0x0]
Alternatively, if the ip audit feature is used, the regular expression could be expanded, which would enable display of logs from the access list named OUTSIDE and from IPS-related signatures.
Firewall# show logging | grep (OUTSIDE|4000[0-9][0-9]) ug 24 2007 10:07:47: %ASA-4-400014: IDS:2004 ICMP echo request from 192.168.208.63 to 192.168.150.77 on
interface outside Aug 24 2007 10:07:47: %ASA-4-106023: Deny icmp src outside:192.168.208.63 dst inside:192.168.150.77
(type 8, code 0) by access-group "OUTSIDE" [0xd3f63b90, 0x0] Aug 24 2007 10:07:47: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.208.63(57633) ->
inside/192.168.150.77(80) hit-cnt 1 first hit [0x22e8ac21, 0x0] Aug 24 2007 10:07:49: %ASA-4-400014: IDS:2004 ICMP echo request from 192.168.208.63 to 192.168.150.77 on
interface outside Aug 24 2007 10:07:49: %ASA-4-106023: Deny icmp src outside:192.168.208.63 dst inside:192.168.150.77
(type 8, code 0) by access-group "OUTSIDE" [0xd3f63b90, 0x0] Aug 24 2007 10:07:49: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.208.63(57634) ->
inside/192.168.150.77(80) hit-cnt 1 first hit [0x22e8ac21, 0x0]
It is also possible to look for specific ACE entries by searching for the hash (for example, 0x22e8ac21) from the ACE entry of our ACL. The hash (for example, 0x22e8ac21) identifier may not be present in some versions of code or in certain Cisco Security Appliances.
Firewall# show access-list OUTSIDE access-list OUTSIDE; 24 elements access-list OUTSIDE line 1 extended deny tcp host 192.168.208.63 host 192.168.150.77 range www 123 log
informational interval 300 (hitcnt=96) 0x22e8ac21 access-list OUTSIDE line 2 extended deny tcp host 192.168.208.63 host 192.168.150.77 range netbios-ssn 445
(hitcnt=1842) 0x5063b82f access-list OUTSIDE line 3 extended deny icmp host 192.168.208.63 host 192.168.150.77 (hitcnt=6) 0xd3f63b90 Firewall# show logging | grep OUTSIDE Aug 24 2007 09:02:35: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.208.63(51063) ->
inside/192.168.150.77(83) hit-cnt 1 first hit[0x22e8ac21, 0x0] Aug 24 2007 09:02:35: %ASA-4-106023: Deny tcp src outside:192.168.208.63/51064 dst inside:192.168.150.77/214 by
access-group "OUTSIDE" [0x5063b82f, 0x0]
If administrators wanted to see only the entries that corresponded to the specific ACE entry for line 2 of the ACL, they could use the hash 0x5063b82f:
Firewall# show logging | grep 5063b82f Aug 29 2007 11:14:55: %ASA-4-106023: Deny tcp src outside:192.168.208.63/35746 dst inside:192.168.150.77/389 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 29 2007 11:14:55: %ASA-4-106023: Deny tcp src outside:192.168.208.63/35746 dst inside:192.168.150.77/443 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 29 2007 11:14:55: %ASA-4-106023: Deny tcp src outside:192.168.208.63/35746 dst inside:192.168.150.77/256 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 29 2007 11:14:55: %ASA-4-106023: Deny tcp src outside:192.168.208.63/35746 dst inside:192.168.150.77/399 by
access-group "OUTSIDE" [0x5063b82f, 0x0]
It is important to note that, although ACL denied log messages are often of interest because they may indicate potential unauthorized attempts to access the network, ACL allowed messages may sometimes be more useful when administrators investigate incidents. In the following example, the impetus to search for ACL denied messages on the access list OUTSIDE is expanded. The results of the commands will reveal what activity has been generated that is against policy and then determine whether the attacker performed any activity that was allowed by the policy.
First administrators will use the command show logging | grep access list name to identify the source IP address that may be needed for further investigation. In this example, the source IP address of interest is 192.168.208.63.
Firewall# show logging | grep OUTSIDE Aug 24 2007 10:27:29: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.208.63(39675)->
inside/192.168.150.77(80) hit-cnt 1 first hit [0x22e8ac21, 0x0] Aug 24 2007 10:27:31: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.208.63(39676) ->
inside/192.168.150.77(80) hit-cnt 1 first hit [0x22e8ac21, 0x0]
Next administrators will use show logging grep ip address to find more information about the IP address (192.168.208.63):
Firewall# show logging | grep 192.168.208.63 Aug 24 2007 10:27:22: %ASA-4-400014: IDS:2004 ICMP echo request from 192.168.208.63 to 192.168.150.70 on<
interface outside Aug 24 2007 10:27:22: %ASA-6-302020: Built ICMP connection for faddr 192.168.208.63/15343 gaddr 192.168.150.70/0
laddr 192.168.150.70/0 Aug 24 2007 10:27:22: %ASA-6-106015: Deny TCP (no connection) from 192.168.208.63/49827 to
192.168.150.70/80 flags ACK on interface outside Aug 24 2007 10:27:22: %ASA-6-302020: Built ICMP connection for faddr 192.168.208.63/15343 gaddr 192.168.150.70/0
laddr 192.168.150.70/0 Aug 24 2007 10:27:22: %ASA-6-302015: Built inbound UDP connection 732748 for outside:192.168.208.63/49804
(192.168.208.63/49804) to inside:192.168.150.70/53 (192.168.150.70/53) Aug 24 2007 10:27:22: %ASA-6-302015: Built inbound UDP connection 732749 for outside:192.168.208.63/49804
(192.168.208.63/49804) to inside:192.168.150.70/123 (192.168.150.70/123) Aug 24 2007 10:27:22: %ASA-6-302015: Built inbound UDP connection 732750 for outside:192.168.208.63/49804
(192.168.208.63/49804) to inside:192.168.150.70/139 (192.168.150.70/139) Aug 24 2007 10:27:22: %ASA-6-302020: Built ICMP connection for faddr 192.168.208.63/0 gaddr 192.168.150.70/0
laddr 192.168.150.70/0 Aug 24 2007 10:27:22: %ASA-6-302021: Teardown ICMP connection for faddr 192.168.208.63/0 gaddr 192.168.150.70/0
laddr 192.168.150.70/0 Aug 24 2007 10:27:22: %ASA-6-302020: Built ICMP connection for faddr 192.168.208.63/0 gaddr 192.168.150.70/0
laddr 192.168.150.70/0 Aug 24 2007 10:27:22: %ASA-6-302021: Teardown ICMP connection for faddr 192.168.208.63/0 gaddr 192.168.150.70/0
laddr 192.168.150.70/0 Aug 24 2007 10:27:23: %ASA-6-302015: Built inbound UDP connection 732753 for outside:192.168.208.63/49805
(192.168.208.63/49805) to inside:192.168.150.70/53 (192.168.150.70/53) Aug 24 2007 10:27:24: %ASA-6-302021: Teardown ICMP connection for faddr 192.168.208.63/15343 gaddr
192.168.150.70/0 laddr 192.168.150.70/0 Aug 24 2007 10:27:24: %ASA-6-302021: Teardown ICMP connection for faddr 192.168.208.63/15343 gaddr
192.168.150.70/0 laddr 192.168.150.70/0 Aug 24 2007 10:27:29: %ASA-4-400014: IDS:2004 ICMP echo request from 192.168.208.63 to 192.168.150.77 on
interface outside Aug 24 2007 10:27:29: %ASA-4-106023: Deny icmp src outside:192.168.208.63 dst inside:192.168.150.77
(type 8, code 0) by access-group "OUTSIDE" [0xd3f63b90, 0x0] Aug 24 2007 10:27:29: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.208.63(39675) ->
inside/192.168.150.77(80) hit-cnt 1 first hit [0x22e8ac21, 0x0] Aug 24 2007 10:27:31: %ASA-4-400014: IDS:2004 ICMP echo request from 192.168.208.63 to 192.168.150.77 on
interface outside Aug 24 2007 10:27:31: %ASA-4-106023: Deny icmp src outside:192.168.208.63 dst inside:192.168.150.77
(type 8, code 0) by access-group "OUTSIDE" [0xd3f63b90, 0x0] Aug 24 2007 10:27:31: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.208.63(39676) ->
inside/192.168.150.77(80) hit-cnt 1 first hit [0x22e8ac21, 0x0]
It appears that, in addition to several denied connections, this IP address also triggered some IPS-related alarms as well as successful UDP connections that were not denied by OUTSIDE access-list policy.
Although these syslog events do not give the same information as a full sniffer log such as TCP dump, there still is some information administrators can use to learn about the attacker. In addition, where there is access to historical data, some of these techniques can be used to find information that may evade time-based or connection-based means of identification. The following examples illustrate methods that can be used to aid in the identification of vertical and horizontal network scans.
Note: Horizontal scans will scan for a specific port or ports across a range of hosts (IP addresses), whereas vertical scans will scan a range of IP/TCP/UDP ports on a specific host or hosts (IP address/IP addresses).
The following is an example of a horizontal scan for TCP port 80 across several hosts. Note the incrementing source port from the attacker (51606-51611). This is due to standard sockets behavior. If the host has been scanning several other hosts, administrators may not see contiguous source ports such as those shown in this example. Also note that the URL "/" was accessed on host 192.168.150.70 and that 1533 bytes were transferred.
Firewall# show logging | grep 192.168.208.63 Aug 24 2007 11:15:16: %ASA-6-302013: Built inbound TCP connection 733280 for outside:192.168.208.63/51606
(192.168.208.63/51606) to inside:192.168.150.70/80 (192.168.150.70/80) Aug 24 2007 11:15:16: %ASA-6-302014: Teardown TCP connection 733280 for outside:192.168.208.63/51606 to
inside:192.168.150.70/80 duration 0:00:00 bytes 0 TCP Reset-O Aug 24 2007 11:15:29: %ASA-6-302013: Built inbound TCP connection 733282 for outside:192.168.208.63/51607
(192.168.208.63/51607) to inside:192.168.150.60/80 (192.168.150.60/80) Aug 24 2007 11:15:29: %ASA-6-302014: Teardown TCP connection 733282 for outside:192.168.208.63/51607 to
inside:192.168.150.60/80 duration 0:00:00 bytes 0 TCP Reset-I Aug 24 2007 11:15:33: %ASA-6-302013: Built inbound TCP connection 733283 for outside:192.168.208.63/51608
(192.168.208.63/51608) to inside:192.168.150.63/80 (192.168.150.63/80) Aug 24 2007 11:15:33: %ASA-6-302014: Teardown TCP connection 733283 for outside:192.168.208.63/51608 to
inside:192.168.150.63/80 duration 0:00:00 bytes 0 TCP Reset-I Aug 24 2007 11:15:39: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.208.63(51609) ->
inside/192.168.150.77(80) hit-cnt 1 first hit [0x22e8ac21, 0x0] Aug 24 2007 11:15:40: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.208.63(51610) ->
inside/192.168.150.77(80) hit-cnt 1 first hit [0x22e8ac21, 0x0] Aug 24 2007 11:15:50: %ASA-6-302013: Built inbound TCP connection 733286 for outside:192.168.208.63/51611
(192.168.208.63/51611) to inside:192.168.150.70/80 <(192.168.150.70/80) Aug 24 2007 11:15:58: %ASA-5-304001: 192.168.208.63 Accessed URL 192.168.150.70:/ Aug 24 2007 11:15:59: %ASA-6-302014: Teardown TCP connection 733286 for outside:192.168.208.63/51611 to
inside:192.168.150.70/80 duration 0:00:09 bytes 1533 TCP FINs
The following example shows a vertical scan of a specific host. It reveals that several different ports were accessed and the source port of the attacking IP is nearly contiguous, which is standard socket behavior.
Firewall# show logging | grep 192.168.208.63 Aug 24 2007 11:23:11: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.208.63(52978) ->
inside/192.168.150.77(80) hit-cnt 1 first hit [0x22e8ac21, 0x0] Aug 24 2007 11:23:11: %ASA-6-302013: Built inbound TCP connection 734665 for outside:192.168.208.63/52979
(192.168.208.63/52979) to inside:192.168.150.77/53 (192.168.150.77/53) Aug 24 2007 11:23:11: %ASA-4-106023: Deny tcp src outside:192.168.208.63/52980 dst inside:192.168.150.77/256 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 24 2007 11:23:11: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.208.63(52981) ->
inside/192.168.150.77(113) hit-cnt 1 first hit [0x22e8ac21, 0x0] Aug 24 2007 11:23:11: %ASA-4-106023: Deny tcp src outside:192.168.208.63/52982 dst inside:192.168.150.77/443 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 24 2007 11:23:11: %ASA-6-302013: Built inbound TCP connection 734666 for outside:192.168.208.63/52983
(192.168.208.63/52983) to inside:192.168.150.77/636 (192.168.150.77/636) Aug 24 2007 11:23:11: %ASA-4-106023: Deny tcp src outside:192.168.208.63/52984 dst inside:192.168.150.77/389 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 24 2007 11:23:11: %ASA-6-302013: Built inbound TCP connection 734667 for outside:192.168.208.63/52985
(192.168.208.63/52985) to inside:192.168.150.77/554 (192.168.150.77/554) Aug 24 2007 11:23:11: %ASA-6-302013: Built inbound TCP connection 734668 for outside:192.168.208.63/52986
(192.168.208.63/52986) to inside:192.168.150.77/25 (192.168.150.77/25) Aug 24 2007 11:23:11: %ASA-6-302013: Built inbound TCP connection 734669 for outside:192.168.208.63/52987
(192.168.208.63/52987) to inside:192.168.150.77/23 (192.168.150.77/23)
For cases in which IP address spoofing is strongly suspected, grepping for an IP address may not work very well, depending on the extent of the IP address spoofing. In these cases, it is better to rely on filtering based on regular expressions. Because of the near-contiguous nature of the source ports from the attacking host, it is likely that this traffic is from a single host controlled by the attacker.
Firewall# show logging | grep (106[0-2][0-9][0-9]|4000[0-9][0-9]) Aug 24 2007 11:43:41: %ASA-4-106023: Deny tcp src outside:192.168.208.63/55453 dst inside:192.168.150.77/256 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 24 2007 11:43:41: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.20.55(55454) ->
inside/192.168.150.77(80) hit-cnt 1 first hit [0x22e8ac21, 0x0] Aug 24 2007 11:43:41: %ASA-4-106023: Deny tcp src outside:192.168.28.68/55455 dst inside:192.168.150.77/443 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 24 2007 11:43:41: %ASA-4-106023: Deny tcp src outside:192.168.18.13/55456 dst inside:192.168.150.77/389 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 24 2007 11:43:41: %ASA-6-106100: access-list OUTSIDE denied tcp outside/192.168.208.63(55460) ->
inside/192.168.150.77(113) hit-cnt 1 first hit [0x22e8ac21, 0x0] Aug 24 2007 11:43:41: %ASA-4-106023: Deny tcp src outside:192.168.11.31/55457 dst inside:192.168.150.77/197 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 24 2007 11:43:41: %ASA-4-106023: Deny tcp src outside:192.168.8.21/55458 dst inside:192.168.150.77/263 by
access-group "OUTSIDE" [0x5063b82f, 0x0] Aug 24 2007 11:43:41: %ASA-4-106023: Deny tcp src outside:192.168.16.231/55459 dst inside:192.168.150.77/445 by
access-group "OUTSIDE" [0x5063b82f, 0x0]
Syslog messages from transit network devices can provide additional context and verification during an incident response. Often, these devices are the only place this level of detailed information exists. Although the examples in this paper were exclusive to tools available natively on the device and messages located locally in the memory buffer of the device itself, much of this analysis can be done on historical logs using common tools. Historical analysis of logs often may provide details that were not revealed using other tools and/or other forms of monitoring.
Understanding Access Control List Logging
http://www.cisco.com/web/about/security/intelligence/acl-logging.html
Using the Command Line Interface
http://www.cisco.com/en/US/docs/security/fwsm/fwsm31/command/reference/usecli.html#wp1046322
IOS Regular Expressions
http://www.cisco.com/en/US/docs/ios/12_2/termserv/configuration/guide/tcfaapre_ps1835_TSD_Products_Configuration_Guide_Chapter.html
Cisco IOS Software Release 12.4 Mainline Error and System Messages Documentation
http://www.cisco.com/en/US/products/ps6350/products_system_message_guides_list.html
Cisco Security Appliance System Log Messages, Version 7.2
http://www.cisco.com/en/US/docs/security/asa/asa72/system/message/syslog.html
This document is part of the Cisco Security portal. Cisco provides the official information contained on the Cisco Security portal in English only.
This document is provided on an “as is” basis and does not imply any kind of guarantee or warranty, including the warranties of merchantability or fitness for a particular use. Your use of the information in the document or materials linked from the document is at your own risk. Cisco reserves the right to change or update this document without notice at any time.