Configure AWS VPC Flow Logs for CTB Input

Introduction

This document describes how to configure VPC Flow Logs as an input to Cisco Telemetry Broker (CTB).

Prerequisites 

Requirements

Cisco recommends that you have knowledge of these topics:

• Amazon Web Services (AWS) 
• CTB admininstration. 

Components Used

The information in this document is based on these software and hardware versions:

• CTB (v2.2.1+)
• AWS

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Configuration Steps

Step 1. Configure S3 Bucket in AWS

1: Log in to AWS management console with username and password.

2: Ensure you log in to appropriate region.

3: Navigate to search bar and type S3.

          AWS-Dashboard

Note: In demo, you have selected Ohio region with us-east-2 availability zone, it is visible right next to the gear icon.

4: Click create bucket.

             AWS-S3

5: Give bucket a name and leave every option as it is and click create.

             AWS-S3

             AWS-S3

6: Once bucket is successfully created, save the bucket ARN which is to be used later during the configuration.

         AWS-S3            

         AWS-S3

Step 2. Create IAM User with Access key and Attach S3 Bucket Policy

1: Launch the IAM from aws search bar. 

       AWS-IAM    

2: Navigate to users.

              AWS-IAM

3: Uncheck the AWS management console access box as the purpose of created user is to access S3 storage.

           AWS-IAM

4: Move ahead via click next.

           AWS-IAM

5: Click create user once entered details are verified. 

Note: By unchecking the AWS management console access box, it prevents the user from logging in to AWS account using web UI.

6: Assign policy by assigning it to the user, directly attaching it to a group or configuring it inline.

Note: For demonstration, you direclty assign policy to the user. For more information - Managing AWS Policies

7: Search for S3 full access and select AmazonS3full access, which allows the user to have full access for every S3 bucket created on its corresponding AWS account.

8: Check the box with policy name AmazonS3FullAccess and click next.

               AWS-IAM

               AWS-IAM

Note: You can create more granular policy by allowing only specific bucket as well, please navigate to  Policy creation to create your S3 bucket policy in json format.

9: Once user is created, list the user and navigate to security credential tab and click create access key.

             AWS-IAM

               AWS-IAM

10: Select the other radio button and optionally add a tag.

              AWS-IAM

           AWS-IAM

          AWS-IAM

11: Click Download .csv file. This is the Access key in a csv file and it is no longer available to download or view once you navigate away from this page.

                 AWS-IAM

Step 3. Configure VPC Flow Logs

1: Launch your VPC on your desired region and navigate to Your VPC option.

             AWS-Flow-Logs           

2: Select your VPC from the list showing on the screen.

              AWS-Flow-Logs

Note: You have selected VPC name SCA in this demo.

3: Navigate to Your VPCs under Virtual private cloud, switch to the Flow logs tab and click Create flow logs.

           AWS-Flow-Logs

4: Give your flow logs a name and share the S3 bucket ARN created earlier.

Note: For ARN, see Configure S3 bucket - Step 6

5: You have an option to go with AWS default log format or create custom log format in case if more fields are required.

             AWS-Flow-Logs

            AWS-Flow-Logs

7: Click create flow logs.

            AWS-Flow-Logs

            AWS-Flow-Logs

Step 4. Configure VPC Input to CTB

1: Access CTB Web UI, navigate to Explorer> Broker node tab > click open broker node >Data Flowtab > Click Add Input.

         CTB-Input-UI

2: Select Input type as AWS VPC Flow log and click next.

         CTB-Input-UI

3: Fill the details Input name, S3 Bucket Path, Region Code, Input IP Address, AWS Access Key ID, AWS Secret Access Key and click Add input.

        CTB-Input-UI

Note: For S3 Bucket Path, see Configure VPC flow logs - Step 7.

Note: For Region code, see AWS home page next to gear icon.

Note: Any IP Address configured as the Input IP Address (unique IP not shared by any other exporter) is reported as the exporter for the transformed netflow data.

Note: For AWS Access Key ID, see Configure IAM user for access key with S3 access policy, step 9

Verify

After a few minutes of configuring AWS VPC input, the status column becomes active if the AWS S3 bucket has data in it.

Verify the status of AWS VPC input using these steps.

1: Log in to CTB UI and navigate toExplorer> Broker node tab > click openbroker node >switch tab toInput >Click open  AWS input.

2: Verify that configured aws-flow logs have active status and received metric have rising graph.

             CTB-Input-UI