Table Of Contents
Configuration Recommendations for Scalability
Policy Tuning and Troubleshooting
Caching and Resetting Query Responses
Deployment Planning
Overview
This section provides information on deploying the product as part of pilot program and scaling the product to 100,000 agent deployments.
This section contains the following topics:
•
Configuration Recommendations for Scalability
•
•
Piloting the Product
Before deploying Cisco Security Agents (CSA) on a large scale, it is critical that you run a manageable and modest initial pilot of the product. Even in a CSA upgrade situation, a pilot program is required. Due to the unique configuration of every individual enterprise, the pre-configured policies that ship with CSA will not fit every site perfectly. A certain amount of policy tuning is always necessary. This tuning is best done on a small sample of systems that are representative of the whole.
Once the pilot is operating satisfactorily, with CSA protecting systems using properly tuned policies, you can turn your pilot into a larger deployment.
The following sections provide a guideline for conducting a pilot of CSA and deploying the product on a large scale.
Running a Pilot Program
Your pilot program should proceed in the following manner:
•
Details:
–
–
The pilot should include machines that you can access readily (either yourself or through a responsive end-user). If you will eventually be installing agents on multiple, supported operating systems, your pilot should include machines running those operating systems. Again, systems in your pilot should be representative of the whole deployment to which you intend to scale.
•
Details:
–
Scalable Deployments
The Cisco Security Agent V5.x release offers scaling of agents to 100,000 systems. To reach this deployment number, there are recommended multi-tiered CSA MC server system hardware, CPU, and memory requirements. Please refer to the following section.
Hardware Sizing
This section provides three server configuration examples and three hardware configuration examples. The server and hardware combinations will be charted in three tables providing information on how many agents can be deployed using each server and hardware configuration combination. This should give you an idea of how to configure CSA to scale up to a 100,000 agent deployment.
For the purpose of this guide, we will use three server configuration examples.
Server Configurations:
1.
2.
3.
We will use the following hardware configurations.
Hardware Configurations:
1.
2.
3.
4.
The following tables approximate the number of agents you could deploy with each server configuration installed on one of four hardware configurations provided.
Table 2-1 Server Configuration 1: Single Server
Hardware Configuration 1
2,500
Hardware Configuration 2
5,000
Hardware Configuration 3
10,000
Hardware Configuration 4
20,000
Table 2-2 Server Configuration 2: Two Servers
7,500
15,000
30,000
75,000
Table 2-3 Server Configuration 3: Three Servers
10,000
20,000
50,000
100,000
Software Considerations
•
•
Note
Configuration Recommendations for Scalability
If you intend to scale to a deployment of approximately 100,000 agents, there are some configuration recommendations you should consider.
Set Polling Interval
With 100,000 agents deployed across your enterprise, you want to ensure that no more than 20 agents are communicating with the MC approximately every second or so. Therefore, with a deployment of this size, it is recommended that you set the polling interval to no less than 1 hour. You can have some systems polling in every hour and others polling in later than that. But on average, a 1 hour or higher polling interval is appropriate. Be sure to have the polling hint functionality enabled, as well.
Use Content Engines
For large deployments, it is highly recommended that you use content engines with transparent web caching. It makes sense to direct groups of agents to different content engines in large deployment scenarios. Content engines reduce the load on the MC by caching rule downloads and software updates.
Factors in Network Sizing
You can use the following data points for computing product network usage. The following numbers average tasks based on the upper limit of a 100,000 agent deployment.
Agent and Configuration Statistics
•
•
•
•
Task Size Statistics
•
•
•
•
•
•
•
•
•
Tracker Agent Installation Statistics
•
•
•
Bandwidth Statistics
•
•
•
•
•
As an example of how you could compute network load using the data points provided here, take 100,000 agents, each generating an average of 3 events per day, and multiply Event update size, by number of Event updates, by number of agents, per a time frame of your choosing and average out a network load.
Factors in Database Sizing
You can use the following data points for computing database sizing. The following numbers average table size based on the upper limit of a 100,000 agent deployment.
•
•
•
•
Policy Tuning and Troubleshooting
Once you have started your CSA pilot, you need to tune the policies to suit your needs and troubleshoot any problems that occur.
Overall Guidelines
This section presents some overall guidelines for tuning and troubleshooting your CSA pilot. Please read through this section carefully and consider the specific needs and requirements of your pilot before moving on to actually using the techniques. Here are the most important guidelines to follow when tuning and troubleshooting policies:
•
•
Group membership is cumulative, which can be useful in tuning and troubleshooting. For example, at the beginning of a pilot, participating hosts that are Windows desktops would be attached to the All Windows and Desktops - All Types groups on the Systems -> Groups menu. Once you have tuned the basic desktop policies, you might attach some of those hosts to the Desktops - Remote or mobile group. Once you are satisfied with the performance of the remote/mobile policies, you could define a new group for a specific department's applications, attach hosts to the new group, and pilot those policies.
•
•
•
•
•
There are two general approaches to policy creation, and the approach you choose affects how you tune and troubleshoot the policies:
•
–
–
–
Then, you attach each group to the following policies:
–
–
Note
After attaching the Desktop and Server groups, Virus Scanner policy, and Installation Application policy, you are ready to create agent kits, start the pilot, examine the event log, and stage the next policy additions. For example, if you have a prioritized list of applications to protect, start with the first on the list, use the Analysis -> Application Behavior Investigation tool to understand the behavior of the application, craft a policy, place it in test mode on the pilot machines, and examine the event log. Use the techniques in the rest of this section to tune/troubleshoot that application's policy, re-examine the event log, and if you are satisfied with the result, place the application's policy in live mode on the pilot machines. You repeat these steps with each application on your prioritized list.
•
Using Test Mode
CSA policies can execute in live mode, where they enforce rules by denying or allowing events, or test mode, where they indicate in the event log what the action would have been to the given event. All entries in the event log for rules in test mode begin with the label
TESTMODE:to make it easy to scan for events relating to rules under test. In general, you start a pilot in test mode and gradually change over to live mode as you examine the performance of each policy. You can use test mode in two different ways:•
From the Systems->Groups menu, you use the supplied Systems - test mode group, which is available for Windows, Linux, and Solaris. You attach hosts (both desktops and servers) to each appropriate test mode group. You can make one or more agent kits available for download with the test mode groups. Be sure to include "test mode" in the name of the agent kit.
When the "test mode" phase of the pilot is completed, you can unattach hosts from the test mode groups to place the hosts in live mode.
•
If one of the rule modules within a policy is not behaving as expected, you can place it in test mode while still keeping the remaining rule modules in live mode. To do this, select the Test Mode checkbox on any Configuration -> Rule Modules -> <platform> Rule Modules -> <module name> page.
Note
Test mode is not intended to be used indefinitely because the area under test is completely vulnerable from a security standpoint. Groups and rule modules in test mode should move to live mode in a timely fashion. Once the pilot is over, you need to carefully control which hosts if any are in test mode. You can remove the test mode kits to ensure they do not get downloaded during deployment and periodically monitor the Systems - test mode group to ensure that all pilot participants have migrated to live mode agent kits. You want to avoid the situation where a security hole exists after deployment because some groups or rule modules were inadvertently left in test mode.
Disabling Specific Rules
When you examine the event log with the Events -> Event Log menu, the description of each event references the rule number. If you find a consistent pattern of false positives with the same specific rule number, you can disable that rule if desired. There are two different approaches to disabling rules:
•
•
The easiest way to disable a rule is by clicking on the rule number at the bottom of the event description in the event log. On the rule page, you click on the Enabled checkbox to uncheck it and disable the rule. Once you generate the rules, this rule will be disabled.
Caching and Resetting Query Responses
Rules can be configured with enforcement actions of allow, deny, terminate, or query the user. In some cases, there are rules that already query the user but do so repeatedly instead of caching the user's response to make it persistent. In other cases, there are rules that are generating a mix of false positives and valid enforcements in the event log and need to be modified so they query the user and cache the user's response for the false positives.
You set up a query and cache the answer with different MC menus:
•
•
Note
If a user has responded to a query inappropriately and the response is being cached, you can reset the user's cache by doing the following:
1.
2.
3.
Setting Up Exception Rules
In some cases, you need two or more different rules to completely specify the desired actions to a specific event. For example, you could have one rule that denies all applications from writing to the //blizzard/webdocs directory and another rule that allows the WebGuru application with authenticated user webmaster to write to the //blizzard/webdocs directory. The second rule allowing write access for WebGuru is considered an exception rule because it overrides a small part of the overall deny rule for the //blizzard/webdocs/ directory. The MC manipulates the precedence of exception rules so that they are evaluated before the rules that they override.
Although you can create exception rules with the MC rule pages, the easiest way to create exception rules is using the Event Management Wizard from the event log. The wizard tailors its behavior to the event from which you launch it. You can use the wizard to create two general types of exception rules:
•
•
To launch the wizard:
1.
2.
The wizard asks you questions about the following:
•
•
•
•
After you click Finish in the wizard, the MC displays the new exception rule. At this point, you should do the following:
1.
2.
3.
