Manage Logging

Where are Logs Stored?

When you create a policy, activity logs are by default saved to the North America – California, US location of the Secure Access data warehouse. You can change the location of the data warehouse to Europe at any time. For more information, see Change the Location of Event Data Logs.

You can also optionally configure logging so that logs are also stored to an Amazon S3 bucket—either your own or one managed by Cisco.

Logging to the Secure Access Data Warehouse

The Secure Access data warehouse is the virtual location where your instance of Secure Access stores its event data logs. By default, Secure Access saves your event data logs to Cisco's California location; however, you can change the location of the data warehouse from North America to Europe at any time. For more information, see Change the Location of Event Data Logs.

Logging to Amazon S3

As well as storing logs to one of its data warehouses, Secure Access can store logs to an Amazon S3 bucket.

By having your logs uploaded to an S3 bucket, you can then automatically download logs so that you can keep them in perpetuity in backup storage outside of the Secure Access data warehouse storage system. Saving to an S3 bucket also gives you the ability to ingest logs through your SIEM or another security tool. This can help you determine if any security events in your Secure Access logs coincide with events in other security tools.

Secure Access Amazon S3 options:

• A self-managed bucket—You own the Amazon S3 bucket, including its configuration and management.

• A Cisco-managed bucket—Cisco Secure Access owns the bucket and sets the configuration and management of it.

Advantages and Disadvantages of Configuring a Cisco-Managed Bucket

• Easy to set up and manage.

• Secure Access supports the rotation of the S3 bucket key every 90 days.

• Included with your Secure Access license.

• You cannot add anything to your S3 bucket besides log files from Secure Access. You cannot use your Cisco-managed S3 bucket for any other application.

• Some SIEM integration types may require advanced privileges for the user accessing the S3 bucket (beyond the basic Read permissions) and as such, may not work with this feature.

• You cannot get support from Amazon directly for advanced configuration assistance, such as automation or help with the command line.

• Data can only be stored offline for a maximum of 30 days.

Prerequisites

• Full Admin user role. For more information, see Manage Accounts.

• A login to Amazon AWS service (http://aws.amazon.com/console/). If you don't have an account, Amazon provides free sign-up for S3.

  Note

  Amazon requires a credit card in case your usage exceeds free plan usage.

• A bucket configured in Amazon S3 to be used for storing logs. For more information, see Amazon's S3 documentation.

  Note

  Periods in S3 bucket names and Object Lock in S3 buckets are not supported.

JSON Bucket Policy

When you set up your Amazon S3 bucket, you must add a bucket policy that accepts uploads from Secure Access. Copy the following preconfigured JSON and substitute your S3 bucket name for bucketname. Then, paste the Secure Access S3 bucket policy into your Amazon S3 bucket policy.

{
  "Version": "2008-10-17",
  "Statement": \[
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::568526795995:user/logs"
      },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::bucketname/*"
    },
    {
      "Sid": "",
      "Effect": "Deny",
      "Principal": {
        "AWS": "arn:aws:iam::568526795995:user/logs"
      },
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::bucketname/*"},
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::568526795995:user/logs"
      },
      "Action": "s3:GetBucketLocation",
      "Resource": "arn:aws:s3:::bucketname"
    },
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::568526795995:user/logs"
      },
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::bucketname"
    }
  \]
}

Sample S3 Bucket Data Path

my_company_name-us-west-1/dnslogs

Use the data path to your self-managed S3 bucket to:

• Download log files using the AWS CLI.
• Integrate your AWS S3 bucket with the Cisco Cloud Security App for Splunk. The Cisco Cloud Security App for Splunk enables you to analyze your Secure Access logs that are stored in your AWS S3 bucket. For more information, see Cisco Cloud Security App for Splunk.

Prerequisites

• Install the AWS CLI to your system.

To download files, run the following command. If you want to test the command without syncing files, add the --dryrun flag.

AWS CLI command syntax:

aws s3 sync s3://DATAPATH /path/to/local/directory/

Detailed sample command:

aws s3 sync s3://mycompany-us-west-1/dnslogs /opt/splunk/etc/apps/TA-cisco_umbrella/data/

Prerequisites

• Full Admin user role. For more information, see Manage Accounts.

• An AWS account. Sign up for free at http://aws.amazon.com/console/.

  Note

  Amazon requires a credit card if your usage exceeds the limits of the free plan.

• An Amazon S3 bucket configured for storing DLP violations. You must first configure a bucket in Amazon S3. For more information, see The Amazon S3 documentation.

  Note

  Periods are not supported in the names of S3 bucket and Object Lock settings.

Procedure

To enable logging of DLP violation evidence to your S3 bucket, perform these steps:

1. Configure the IAM policy for your S3 bucket

2. Configure your S3 bucket

Components of S3 Bucket Data Path

The components of the S3 bucket data path are:

<AWS S3 bucket name>-<AWS region>/<AWS S3 bucket directory prefix>

• AWS S3 bucket name and AWS region—the name of your AWS S3 bucket, followed by a dash and the AWS region.
• AWS S3 bucket directory prefix—the directory prefix (customer folder name) in the AWS S3 bucket.

Sample S3 Bucket Data Path

An example of a sample S3 bucket data path is:

my_company_name-us-west-1/dnslogs

Use the S3 bucket data path to:

• Download log files using the AWS CLI.
• Integrate your AWS S3 bucket with the Cisco Cloud Security App for Splunk. The Cisco Cloud Security App for Splunk enables you to analyze Secure Access logs stored in your AWS S3 bucket. For more information, see Cisco Cloud Security App for Splunk.

Prerequisites

Before downloading files from the S3 bucket, ensure that AWS CLI is installed on your system.

To download files from the S3 bucket, run the AWS CLI command.

aws s3 sync s3://DATAPATH /path/to/local/directory/

Note

Use the --dryrun flag to test the command without syncing files.

An example of an AWS CLI command is:

aws s3 sync s3://mycompany-us-west-1/dnslogs /opt/splunk/etc/apps/TA-cisco_umbrella/data/

About the Notifications for the IAM Key

• Secure Access provides periodic notifications to an organization when the IAM key for a Cisco-managed S3 bucket expires. Secure Access sends email notifications to the email account that was used during the initial purchase or renewal of your Cisco Secure Access license.
• An organization can view notifications about the expired IAM key on the Overview page in Secure Access. After the organization successfully rotates the IAM key for the Cisco-managed S3 bucket, Secure Access removes the IAM key notifications on the Overview page.

Sample Command

aws s3 sync s3://cisco-managed-us-west-1/2069997_6ff2802af17337def701c2e7816cf14913zf848a/ /opt/splunk/etc/apps/TA-cisco_umbrella/data/

Best Practices: Download Files From the Cisco-Managed S3 Bucket

• When you download files from a Cisco-managed S3 bucket, ensure that you only download one copy of each log file. If you download log files multiple times, Cisco reserves the right to suspend the download of logs from a Cisco-managed S3 bucket (through the rotation of keys or other methods).

• Once files are downloaded to a local directory, you can delete the files. Only delete files that are older than the Retention Duration value configured in Secure Access. Otherwise, the next sync command will download the deleted files again.

Detection and notification

Secure Access continuously checks the status of Amazon S3 log exports. Export failures are detected when any of the following conditions occur:

• The S3 bucket or path is renamed or deleted

• The access policy is modified or removed

• S3 access credentials expire or become invalid

When an export failure is detected, Secure Access is unable to deliver log files to the configured S3 destination. The Admin > Log Management page displays Last Sync timestamp for reference. The job status will become Not configured once the job is paused. This behavior is applicable only for user-managed buckets, not for Cisco-managed buckets.

Export job pause policy

When an export failure is detected and persists for twenty-four hours, the S3 log export job is automatically paused. Secure Access stops attempting to upload logs until the configuration is corrected and the job is manually restarted.

Retry, retention, and deletion policy

When an export failure is detected:

• Secure Access tries to upload affected log files three times in immediate succession.

• If all upload attempts fail, the failed log files are retained for 24 hours from the initial failure. This retention window allows time to correct any misconfiguration and prevent data loss

• If the S3 export misconfiguration is not resolved within 24 hours, the retained log files are permanently deleted from Secure Access. These files cannot be recovered.

Log export recovery considerations

• After a misconfiguration is corrected, the S3 log export job becomes available to restart from Admin > Log Management > ManageclickStart Logging. A success notification appears when you successfully start logging.

• Resolve any misconfiguration within 24 hours to prevent permanent loss of log data.

Data recovery considerations

• Logs deleted after the 24-hour retention window cannot be recovered through S3 export.

• Event data for the affected period may still be accessible through Secure Access reporting APIs within the standard data retention period, which is typically up to 90 days.

Frequently Asked Questions

Will I receive an email notification if my S3 export fails?

No. Email notifications are not sent for S3 export failures.

Can I recover logs deleted after a misconfiguration?

No. Deleted logs cannot be recovered through S3 export. However, event data may be available by using reporting APIs if it is still within the retention period.

How do I know if my S3 export is paused?

The Admin > Log Management page displays the current job status and timestamps for additional information.