Explains how to troubleshoot common device issues including connectivity problems, configuration errors, synchronization failures, and deployment challenges to ensure proper device functionality.
ASA fails to reconnect to Security Cloud Control after reboot
An ASA reconnection failure is a connectivity issue that
-
occurs when an ASA device fails to reconnect to Security Cloud Control after a reboot
-
happens because the ASA has fallen back to using an OpenSSL cipher suite that is not supported by Security Cloud Control's Secure Device Connector (SDC), and
-
requires troubleshooting to test for the cipher suite issue and apply remediation steps.
Symptoms
-
ASA reboots and Security Cloud Control and the ASA fail to reconnect. Security Cloud Control displays the message, "Failed to reconnect."
-
When attempting to onboard an ASA, Security Cloud Control displays the message: Certificate could not be retrieved for <ASA_IP_Address>.
Disable ASA client-side certificate authentication for onboarding
Disable client-side certificate authentication on ASA to enable successful onboarding to Security Cloud Control.
Environment: ASA is configured with client-side certificate authentication.
Solution: Disable client-side certificate authentication.
Details: ASAs support credential-based authentication as well as client-side certificate authentication. Security Cloud Control cannot connect to ASAs that use client-side certificate authentication. Before onboarding your ASA to Security Cloud Control, make sure it does not have client-certificate authentication enabled by using this procedure:
Procedure
Client-side certificate authentication is disabled on the ASA, allowing successful onboarding to Security Cloud Control.
Determine the OpenSSL cipher suite used by your ASA
This task helps you identify whether the OpenSSL cipher suite on your ASA is compatible with the Secure Device Connector (SDC).
Use this procedure to identify the OpenSSL cipher suite being used by your ASA. If the cipher suite named in the command output is not in the list of supported cipher suites, the SDC doesn't support that cipher suite and you will need to update the cipher suites on your ASA.
Procedure
The command output displays the OpenSSL cipher suite being used by your ASA, allowing you to verify its compatibility with the SDC.
Cipher suites supported by Security Cloud Control's Secure Device Connector
Security Cloud Control's Secure Device Connector uses node.js which only accepts the latest and most secure ciphers. As a result, Security Cloud Control's SDC only supports this list of ciphers:
-
ECDHE-RSA-AES128-GCM-SHA256
-
ECDHE-ECDSA-AES128-GCM-SHA256
-
ECDHE-RSA-AES256-GCM-SHA384
-
ECDHE-ECDSA-AES256-GCM-SHA384
-
DHE-RSA-AES128-GCM-SHA256
-
ECDHE-RSA-AES128-SHA256
-
DHE-RSA-AES128-SHA256
-
ECDHE-RSA-AES256-SHA384
-
DHE-RSA-AES256-SHA384
-
ECDHE-RSA-AES256-SHA256
-
DHE-RSA-AES256-SHA256
If the cipher suite you use on your ASA is not in this list, SDC does not support it and you will need to Update your ASA's cipher suite.
Update your ASA's cipher suite
Update the TLS cipher suites on your ASA to ensure secure communication by configuring specific encryption protocols that meet your security requirements.
You need to modify the TLS cipher suite configuration when strengthening the security posture of your ASA device or when compliance requirements mandate specific encryption standards.
Procedure
The ASA is now configured with the specified TLS cipher suites, and the configuration has been saved to memory.
Troubleshoot ASA using CLI commands
This section discusses some of the important commands you may want to use to troubleshoot the ASA and test basic connectivity. See CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide to learn about other troubleshooting scenarios and CLI commands. In the 'System Administration' section, navigate to the 'Testing and Troubleshooting' chapter.
You can use the Security Cloud Control CLI interface available for each ASA device to execute these commands.
See ASA command line interface to learn about how to use the CLI interface in Security Cloud Control.
NAT policy settings
Some of the important commands to determine the NAT settings are as follows:
-
To determine NAT policy statistics, use show NAT.
-
To determine the NAT pools, including the addresses and ports allocated, and how many times they were allocated, use show NAT pool.
For more commands related to NAT, see CLI Book 2: Cisco ASA Series Firewall CLI Configuration Guide, and navigate to the 'Network Address Translation (NAT)' chapter.
Test basic connectivity: pinging addresses
You can ping the ASA device using the ping <IP address> command using the ASA CLI interface.
Display the routing table
Use the show route command to view the entries in the routing table.
ciscoasa# show route
Example output for a routing table of an ASA:
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, V - VPN
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, + - replicated route
SI - Static InterVRF
Gateway of last resort is 192.168.0.254 to network 0.0.0.0
S* 0.0.0.0 0.0.0.0 [1/0] via 192.168.0.254, management
C 10.0.0.0 255.0.0.0 is directly connected, Outside
L 10.10.10.1 255.255.255.255 is directly connected, Outside
C 192.168.0.0 255.255.255.0 is directly connected, management
L 192.168.0.118 255.255.255.255 is directly connected, management
Monitor switch ports
-
show interface
Displays interface statistics.
-
show interface IP brief
Displays interface IP addresses and status.
-
show ARP
Shows dynamic, static, and proxy ARP entries. Dynamic ARP entries include the age of the ARP entry in seconds.
Example output of ARP entries:
management 10.10.32.129 0050.568a.977b 0
management 10.10.32.136 0050.568a.5387 21
LANFAIL 20.20.21.1 0050.568a.4d70 96
outsi 10.10.16.6 0050.568a.e6d3 3881
outsi 10.10.16.1 0050.568a.977b 5551
Troubleshoot missing information on the RA VPN monitoring page
This task resolves issues where information is missing on the RA VPN monitoring page by enabling the outside interface for Webvpn on an ASA device.
This issue may occur if the outside interface is not enabled for Webvpn when configuring remote access VPN on an ASA device.
Procedure
The outside interface is now enabled for Webvpn, and information should appear correctly on the RA VPN monitoring page.
ASA real-time logging
ASA real-time logging is a logging feature that
-
displays the last 20 seconds of logged data or the last 10 KB of logged data, whichever limit is reached first
-
temporarily changes the existing logging configuration to request debugging-level data during data retrieval, and
-
returns the logging configuration to its previous state after data collection completes.
Real-time logging operation
When Security Cloud Control retrieves the real-time data, it reviews the existing logging configuration on your ASDM, changes it to request debugging-level data, and then returns the logging configuration to your configuration. The logging Security Cloud Control displays reflects any logging filters you may have set in ASDM.
You can see the commands that Security Cloud Control sends to perform logging by reviewing the change log. The first entry (on the bottom) indicates that Security Cloud Control "turned on" logging with the logging enable command and changed the ASDM logging level to debugging. The second entry (on the top) shows that the logging configuration was returned to its previous state. Logging was "turned off" with the no logging enable command and the ASDM logging level was returned to informational.
View ASA real-time logs
View real-time logging data from ASA devices to monitor network activity and troubleshoot issues as they occur.
Real-time logs provide immediate visibility into ASA device activity, allowing you to monitor traffic, security events, and system status in real time. You can apply filters to refine the log data and focus on specific events or criteria.
Procedure
Real-time logging data is displayed based on your filtering criteria, allowing you to monitor ASA device activity and troubleshoot issues as they occur.
ASA packet tracer
ASA Packet Tracer is a network troubleshooting tool that
-
sends synthetic packets into the network to evaluate how existing routing configuration, NAT rules, and policy configurations affect that packet
-
works on live, online ASA devices either physical or virtual, and
-
evaluates packets based on the saved configuration on the ASA, not staged changes.
Troubleshooting use cases
Use this tool to troubleshoot these kinds of issues:
-
Users report that they cannot reach resources that they should be able to.
-
Users report that they can reach resources they should not be able to.
-
Test a policy to determine if it works as you expect.
Packet tracer can be used on a live, online, ASA device either physical or virtual. Packet Tracer does not work on ASA model devices. Packet tracer evaluates packets based on the saved configuration on the ASA. Staged changes on Security Cloud Control are not evaluated by packet tracer.
We consider it a best-practice to run packet tracer on an ASA that is in the synced state. Though packet tracer will run if the device is not synced, you could encounter some unexpected results. For example, if you deleted a rule in the staged configuration on Security Cloud Control, and this same rule was triggered on the ASA during packet tracing, Security Cloud Control won't be able to show you the result of the packet's interaction with that rule.
As packet tracer sends the packet through the routing configuration, NAT rules, and security policies of your ASA, it displays the packet's status at each step. If the packet is allowed by the policy it receives a green checkmark
. If a packet is denied and dropped, Security Cloud Control displays a red X
.
Packet tracer also displays a real time log of the result of the packet trace. In the example below, you can see where a rule denied a tcp packet.
Troubleshoot an ASA device security policy
Use packet tracer to troubleshoot security policies by simulating packet flow through your ASA device to identify potential connectivity issues and policy violations.
Packet tracer allows you to test how packets traverse through your ASA device's security policies without actually sending real traffic. This helps diagnose connection problems and verify policy configurations.
Procedure
The packet tracer results display the path the simulated packet takes through your ASA device, showing which security policies are applied and whether the packet is allowed or denied.
Troubleshoot an access rule
Use packet tracer troubleshooting to analyze how traffic flows through your network policy and identify issues with specific access rules.
When you need to verify that an access rule is working correctly or diagnose traffic flow issues, the packet tracer tool allows you to simulate network traffic and see how it interacts with your configured rules.
Procedure
The packet tracer runs and provides analysis results for the selected access rule, showing how traffic flows through your network policy.
Troubleshoot a NAT rule
This task enables you to diagnose and resolve issues with NAT rules by using packet tracer functionality to analyze traffic flow and rule behavior.
Use this procedure when you need to troubleshoot NAT rule configuration issues on ASA devices. The troubleshooting process helps identify why traffic is not being translated as expected.
Procedure
-
In the left pane, click and then click the ASA tab.
-
Select your ASA, and click View NAT Rules
in the Action pane. -
Select the rule from the NAT Rules table that you want to troubleshoot and click Troubleshoot
in the details pane. Notice that in the values panel of the Troubleshoot page, many of the fields are pre-populated with the attributes of the rule you chose. -
Enter information in the remaining required fields. Once you have completed all the required fields the Run Packet Tracer becomes active.
-
Click Run Packet Tracer.
-
Continue with Analyze packet tracer results.
The packet tracer runs and provides analysis results that help identify NAT rule issues and traffic flow problems.
Troubleshoot a twice NAT rule
Troubleshoot a Twice NAT rule to identify and resolve issues with packet translation by running packet tracer analysis on source or destination packet flows.
Use this procedure when you need to analyze NAT rule behavior and identify potential issues with packet translation in your ASA configuration. For bi-directional Twice NAT rules, you can troubleshoot either source packet translation or destination packet translation.
Procedure
-
In left pane, click .
-
Click the ASA tab, select your ASA, and click View NAT Rules
in the Action pane. -
Select the rule from the NAT Rules table that you want to troubleshoot and click Troubleshoot
in the details pane. For a bi-directional Twice NAT rule, this opens a dropdown where you choose to troubleshoot the source packet translation or the destination packet translation. -
Enter information in the remaining required fields. Once you have completed all the required fields the Run Packet Tracer becomes active.
-
Click Run Packet Tracer.
The packet tracer analysis runs and displays the results, showing how the NAT rule processes the specified packet flow and identifying any issues with the translation.
Analyze packet tracer results
This task helps you understand the reasons behind packet tracer results, whether packets are dropped or allowed, and enables you to make necessary configuration changes to achieve desired access results.
Packet tracer provides detailed information about packet processing decisions. When you run packet tracer, you receive results that show whether packets are allowed or denied, along with the specific rules and policies that influenced these decisions.
Procedure
You have successfully analyzed the packet tracer results and understand the reasons for packet processing decisions. Any necessary rule modifications have been deployed and verified through re-running packet tracer.
Cisco ASA Advisory cisco-sa-20180129-asa1
The Cisco Product Security Incident Response Team (PSIRT) published the security advisory cisco-sa-20180129-asa1 which describes a critical-severity ASA and Firepower security vulnerability. Read the entire PSIRT team advisory for a full explanation of what ASA and Firepower hardware, software, and configurations are affected.
If you determine that your ASAs are impacted by the advisory, you can upgrade your ASAs to the patched version using Security Cloud Control. Use this process:
Procedure
Confirming ASA running configuration size
Confirm the size of your running configuration file to ensure it meets system requirements or limitations.
Use this procedure to determine if your ASA running configuration exceeds the 4.5 MB size limit by copying it to flash storage and examining the file properties.
Before you begin
Follow these steps to confirm the size of your running configuration file:
Procedure
You have successfully determined the size of your ASA running configuration file. Files over 4718592 bytes exceed the 4.5 MB limit.
Container Privilege Escalation Vulnerability Affecting Secure Device Connector: cisco-sa-20190215-runc
The Cisco Product Security Incident Response Team (PSIRT) published the security advisory cisco-sa-20190215-runc which describes a high-severity vulnerability in Docker. Read the entire PSIRT team advisory for a full explanation of the vulnerability.
This vulnerability impacts all Security Cloud Control customers:
-
Customers using Security Cloud Control's cloud-deployed Secure Device Connector (SDC) do not need to do anything as the remediation steps have already been performed by the Security Cloud Control Operations Team.
-
Customers using an SDC deployed on-premise need to upgrade their SDC host to use the latest Docker version. They can do so by using the following instructions:
Update a Security Cloud Control-Standard SDC Host
This task updates Docker on your SDC host to ensure continued compatibility and security patching for your Secure Device Connector deployment.
-
The Secure Device Connector (SDC) has been installed on CentOS 7 virtual machines up to now. However, as CentOS has reached its end-of-life and is no longer supported by Firewall in Security Cloud Control, we recommend migrating all SDCs from CentOS 7 to an Ubuntu virtual machine.
For more information, see Migrate an On-Premises Secure Device Connector and Secure Event Connector from a CentOS 7 Virtual Machine to an Ubuntu Virtual Machine.
This procedure applies if you are still using CentOS.
-
Use these instructions if you deployed an SDC using the Security Cloud Control image.
Procedure
You have successfully upgraded to the latest, patched version of Docker on your SDC host.
Recommendation: migrate SDC from CentOS to ubuntu
Migrate all SDCs from CentOS 7 to an Ubuntu virtual machine. The Secure Device Connector (SDC) has been installed on CentOS 7 virtual machines up to now. However, as CentOS has reached its end-of-life and is no longer supported by Firewall in Security Cloud Control, we recommend migrating all SDCs from CentOS 7 to an Ubuntu virtual machine.
-
If you have created your own SDC host you will need to follow the instructions to update based on how you installed Docker. If you used CentOS, yum and Docker-ce (the community edition) the preceding procedure will work.
-
If you have installed Docker-ee (the enterprise edtion) or used an alternate method to install Docker, the fixed versions of Docker may be different. You can check the Docker page to determine the correct versions to install: Docker Security Update and Container Security Best Practices.
This procedure applies if you are still using CentOS.
For more information, see Migrate an On-Premises Secure Device Connector and Secure Event Connector from a CentOS 7 Virtual Machine to an Ubuntu Virtual Machine.
Bug tracking
This reference provides details on how Cisco tracks and updates vulnerability information through bug tracking systems, enabling users to access the latest information on security advisories and associated fixes.
Cisco is continuing to evaluate this vulnerability and will update the advisory as additional information becomes available. After the advisory is marked Final, you can refer to the associated Cisco bug for further details:
Limitation: large ASA running configuration files
Configuration file size limitations
Be aware that Security Cloud Control cannot predictably manage ASA devices with running configuration files that are 4.5 MB or larger, contain approximately 22,000 lines or more, or have too many access-list entries for a single access group.
This limitation affects ASA onboarding and ongoing management in Security Cloud Control. You may see behavior such as the ASA failing to onboard, Security Cloud Control not displaying all of the configuration defined in the ASA's running configuration file, or Security Cloud Control failing to write to the change log.
When you onboard an ASA to Security Cloud Control, Security Cloud Control stores a copy of the ASA's running configuration file in its database. Large configuration files exceed database storage and processing capabilities.
To confirm the size of your running configuration file, see Confirming ASA running configuration size. Contact your Cisco account team for help safely reducing the size of your configuration file without disrupting your security policies.