Have an account?

  •   Personalized content
  •   Your products and support

Need an account?

Create an account

Cisco ScanCenter Administrator Guide, Release 5.2

Book Contents
Find Matches in This Book
Available Languages
Download Options

Book Title

Cisco ScanCenter Administrator Guide, Release 5.2

Chapter Title

Log Extraction

Results

Updated:
April 11, 2019

Chapter: Log Extraction

Log Extraction

Overview

Cisco ScanCenter allows you to extract your data logs from the Cisco Cloud Web Security storage architecture. Configure secure and fully automated extraction of your web usage data for import and analysis using your SIEM platform S3-compatible Application Programming Interface (API). The log extraction service uses Amazon Simple Storage Service (S3) protocol only for the purpose of API compatibility; it does not use Amazon Web Services.

Data logs:

  • compiled in World Wide Web Consortium (W3C) text format

  • tab delimited with UTF-8 encoding

  • placed in a dedicated storage bucket assigned to your company for your secure extraction

  • allows you to choose what data you want to pull and when you want to extract the data

  • typically available within 15 minutes after the event

  • files deleted after five days

Note:

  • If you have anonymization rules in place, all anonymized fields will also apply in the extracted log data.

  • Visibility of the data in HTTPS traffic will be the same as in the WIRe reports.

  • All accounts within the delegated administration hierarchy log data to the same location. Account details are visible in the master account.

  • Child account data consolidated into a single bucket. Reporting attribute x-ss-company-id used to identify child accounts.

Reporting Attributes

25 commonly-used reporting attributes are included in the data logs. Four optional attributes can also be included.

The field prefixes are:

  • c—client

  • s—server

  • cs—client to server

  • sc—server to client

  • x-ss—Cisco Cloud Web Security custom field


Note

The contents of the majority of attributes are normalized to lower case. However, for some attributes you may wish to view the original string as entered by the user. Attributes listed with “Original” in parentheses are available in normalized and original form.

datetime

Date and time

c-ip

Client IP address

cs(X-Forwarded-For)

Value of the X-Forwarded-For header

cs-username

(Original) Username logged in the form of WinNT://<username> or a custom text name

cs-method

(Original) Request method: CONNECT, GET, POST

cs-uri-scheme

Referrer protocol: FTP, HTTP, HTTPS

cs-host

(Original) Host part of the URL string; for example, in news.example.com/sport the host is news.example.com

cs-uri-port

Port number of web request

cs-uri-path

Path part of the URL string; for example, in news.example.com/sport the path is /sport

cs-uri-query

Query part of the URL string; for example, in http://www.example.com/search?hl=en&q=free+screensavers&btnG=Example+Search&meta=&aq=f&oq= the query is hl=en&q=free+screensavers&btnG=Example+Search&meta=&aq=f&oq=

cs(User-Agent)

(Original) Complete user agent string; for example, Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1) (more information on user agent strings can be found at http://www.useragentstring.com/pages/useragentstring.php)

cs(Content-Type)

(Original) Request content type; for example, image/gif, application/pdf, text/html, application/EDI-X12

cs-bytes

Bytes sent

sc-bytes

Bytes received

sc-status

Response Status Code; for example, to find all web requests to pages that did not exist, you can filter by 404 (more information on status codes can be found at http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html)

sc(Content-Type)

(Original) Response content type

s-ip

Destination IP address

x-amp-score

(Optional) AMP score of the request, if available

x-amp-sha

(Optional) Request/response SHA256 for the executable/PDF

x-avc-app

(Optional) Human readable AVC application name

x-avc-app-id

(Optional) AVC application ID, if detected

x-ss-category

Web filtering category; the category codes that apply to the x-ss-category reporting attribute are:

Code Description Code Description Code Description
c:adlt Adult c:gamb Gambling c:pnet Professional Networking
c:adv Advertisements c:game Games c:pol Politics
c:alc Alcohol c:gov Government and Law c:porn Pornography
c:art Arts c:hack Hacking c:ref Reference
c:astr Astrology c:hate Hate Speech c:rel Religion
c:auct Auctions c:hlth Health and Nutrition c:rest Real Estate
c:aud Streaming Audio c:ilac Illegal Activities c:saas SaaS and B2B
c:busi Business and Industry c:ildl Illegal Downloads c:sci Science and Technology
c:card Digital Postcards c:img Photo Search / Images c:scty Society and Culture
c:cell Mobile Phones c:infr Infrastructure and Content Delivery c:shop Shopping
c:chat Chat and Instant Messaging c:job Job Search c:snet Social Networking
c:comm Online Communities c:kids Safe for Kids c:socs Social Science
c:comp Computers and Internet c:ling Lingerie and Swimsuits c:sprt Sports and Recreation
c:csec Computer Security c:lol Humor c:srch Search Engines and Portals
c:date Dating c:lotr Lotteries c:swup Software Updates
c:drug Illegal Drugs c:mail Web-based E-mail c:sxed Sex Education
c:dyn Dynamic / Residential c:natr Nature c:tob Tobacco
c:edu Education c:news News c:trad Online Trading
c:ent Entertainment c:ngo Non-governmental Organizations c:tran Web Page Translation
c:extr Extreme c:nsn Non-sexual Nudity c:trns Transportation
c:fash Fashion c:osb Online Storage and Backup c:trvl Travel
c:filt Filter Avoidance c:p2p Peer File Transfer c:vid Streaming Video
c:fnnc Finance c:park Parked Domains c:voip Internet Telephony
c:food Dining and Drinking c:pem Organizational E-mail c:weap Weapons
c:free Freeware and Shareware c:pers Personal Sites c:whst Web Hosting
c:fts File Transfer Services c:plag Cheating and Plagarism unclassified Unclassified

x-ss-last-rule-name

(Original) Policy rule name

x-ss-last-rule-action

Five rule actions to choose from: allow, authenticate, block, warn, inspect

x-ss-block-type

Type of block or pattern, specified in the filter, that generated the block

x-ss-block-value

String value that matched the block pattern

x-ss-referrer-host

(Original) Host part of the referrer URL string; for example, in news.example.com/sport the host is news.example.com

x-ss-external-ip

External IP address that Cisco Cloud Web Security gets from the customer (also known as the egress IP address); can also be the subnet of the external IP address (also known as the egress IP address subnet)

x-ss-company-id

Child Company ID

Issuing a Key Pair

Issue a key pair from the Log Extraction page in Cisco ScanCenter. Copy both keys into your SIEM platform S3-compatible API to enable extraction of data logs from your assigned bucket.


Note

We do not provide technical support for configuring SIEM devices or third-party products. In the event of an issue, consult the vendor-specific support team.

Before you begin

  • The log extraction service must be enabled and provisioned for your company.

  • Only your super user administrator can access the Log Extraction page.

Procedure

Step 1

Click the Admin tab to display the administration menus.

Step 2

In the Your Account menu, click Log Extraction to display the Log Extraction page.

Step 3

In the Bucket column of the Connection Details area, a dedicated bucket with a unique name based on your Company ID is created to hold your private data.

Step 4

In the Actions column of the Credentials area, click the Issue Key button.

Step 5

In the Warning dialog box, click the Issue & Download button.

Step 6

Save the resulting CSV file named keypair.csv to your downloads folder. The Access Key will be updated to show the generated key ID, but the secret key is not displayed anywhere in Cisco ScanCenter for the purpose of security. To maintain privacy, Cisco Customer Support will not ask for the secret key.

Step 7

Open the saved CSV file named keypair.csv to view your accessKey and secretKey. The accessKey is a 20-character string, while the secretKey is a 40-character string.

Step 8

(Optional) The Log Extraction Attributes area lists some additional attributes you may select to add to exported logs. Additional services may need to be activated to see data in these fields. Click Save.

Attribute Field Name

Description

x-avc-app-id

AVC application ID, if detected

x-avc-app

Human readable AVC application name

x-amp-score

AMP score of the request, if available

x-amp-sha

Request/response SHA256 for the executable/PDF

What to do next

Copy the pair of keys into your API client and configure it for extraction.

For example, s3cmd is a free command-line tool that can be used as a client to test connectivity to the log extraction service.

After downloading and installing the s3cmd tool, edit the following variables in the s3cmd configuration file:

  • Set access_key to your accessKey

  • Set secret_key to your secretKey

  • Set host_base to vault.scansafe.com

  • Set host_bucket to %(bucket)s.vault.scansafe.com

  • Set use_https to True

  • Set signature_v2 to True

As an example result, the variables would then appear as:

  • access_key=12345678901234567890

  • secret_key=1234567890123456789012345678901234567890

  • host_base=vault.scansafe.com

  • host_bucket=%(bucket)s.vault.scansafe.com

  • use_https=True

  • signature_v2=True


Note

The %(bucket)s.vault.scansafe.com value in the host_bucket variable does not use your dedicated bucket name.

For information on how to use the s3cmd tool, see:

http://s3tools.org/usage

http://s3tools.org/s3cmd-howto

http://s3tools.org/s3cmd-sync


Note

Only read commands are allowed to access your bucket. For example, the ls, get, debug, sync, and info read commands are allowed, while the put write command is not allowed.


Note

If the client application being used to access the Log Extraction API sends requests through the Cisco Cloud Web Security service, it may be challenged for authentication. If the client application is unable to respond to the authentication challenge correctly, the requests to the Log Extraction API will be blocked by the Cisco Cloud Web Security service. In this case, allow the client application direct access to the Log Extraction API by adding an exception for the URL vault.scansafe.com on TCP port 443.

Was this Document Helpful?

FeedbackFeedback

Contact Cisco

Related Cisco Community Discussions

About Us
Contact Us
Work with Us