Bulk Statistics

This chapter provides configuration information for:

Configuring Communication with the Collection Server

Two configuration methods are available for defining how bulk statistics are collected and managed. A “standard” configuration allows the system to automatically assign a number to the bulk statistics file. Optionally, a number can be specified by an administrator in the optional configuration method. Command details and descriptions of keywords and variables for commands in this chapter are located in the Bulk Statistics Configuration Mode Commands and Bulk Statistics File Configuration Mode Commands chapters in the Command Line Interface Reference.

Configuring Standard Settings

The configuration example in this section defines basic operation of the bulk statistics feature. Use the following example configuration to set up the system to communicate with the statistic collection server:

configure
   bulkstats mode
      schema name format format_string
      sample-interval time_interval
      transfer-interval xmit_time_interval
      limit mem_limit
      exit
   bulkstats collection
   end

Configuring Optional Settings

This section describes optional commands that can be used within the Bulk Statistics Configuration mode. Specifically, bulk statistic “files” under which to group the bulk statistic configuration are configured using commands in this section. “Files” are used to group bulk statistic schema, delivery options, and receiver configuration. Because multiple “files” can be configured, this functionality provides greater flexibility in that it allows you to configure different schemas to go to different receivers.

configure
   bulkstats mode
      file number
         receiver ip_address { primary | secondary } [ mechanism { { { ftp | sftp } login user_name [ encrypted ] password pwd } | tftp } } ] }
         receiver mode { redundant | secondary-on-failure }
         remotefile format naming_convention [ both-receivers | primary-receiver | secondary-receiver ]
         header format header_format
         footer format footer_format
         exit
      name schema format format_string
      sample-interval time_interval
      transfer-interval xmit_time_interval
      limit mem_limit
      exit
   bulkstats collection
   end

Configuring Bulk Statistic Schemas

In each configuration example described in Configuring Standard Settings and Configuring Optional Settings, the following is the primary command used to configure the type of schema and the statistics collected:
name schema format format_string

Refer to the Bulk Statistics Configuration Mode Commands and Bulk Statistics File Configuration Mode Commands chapters in the Command Line Interface Reference for more information regarding supported schemas, available statistics, and proper command syntax.

Verifying Your Configuration

After configuring support for bulk statistics on the system, you can check your settings prior to saving them.

Follow the instructions in this section to verify your bulk statistic settings. These instructions assume that you are at the root prompt for the Exec mode.

Check your collection server communication and schema settings by entering the following command:
show bulkstats schemas

The following is an example command output:

Bulk Statistics Server Configuration:
  Server
State:                 Enabled 
  File
Limit:                   6000
KB 
  Sample
Interval:              15
minutes (0D 0H 15M)
  Transfer
Interval:            480
minutes (0D 0H 15M)
  Collection
Mode:              Cumulative 
  Receiver
Mode:                Secondary-on-failure 
  Local
File Storage:           None
Bulk Statistics Server
Statistics:
  Records
awaiting transmission: 114
  Bytes
awaiting transmission:   8092
  Total
records collected:       59926
  Total
bytes collected:         4190178 
  Total
records transmitted:     59812
  Total
bytes transmitted:       4188512 
  Total
records discarded:       0
  Total
bytes discarded:         0
  Last
collection time required: 2 second(s)
  Last
transfer time required:   0 second(s)
  Last
successful transfer:      Wednesday
December 7 12:14:30 EDT 2011
  Last
successful tx recs:       190
  Last
successful tx bytes:      13507
  Last
attempted transfer:       Wednesday
December 7 12:14:30 EDT 2011
File 1
  Remote
File Format:     /users/ems/server/data/chicago/bulkstat%date%%time%.txt
  File
Header:                  "CHI_test %time%"
  File
Footer:                  ""
Bulkstats Receivers:
  Primary: 192.168.0.100
using FTP with username administrator
  Records
awaiting transmission: 0
  Bytes
awaiting transmission:   0
  Total
records collected:       0
  Total
bytes collected:         0 
  Total
records transmitted:     0 
  Total
bytes transmitted:       0 
  Total
records discarded:       0 
  Total
bytes discarded:         0 
  Last
transfer time required:   0
second(s) 
  No
successful data transfers
  No
attempted data transfe


File 2 not configured


File 3 not configured


File 4 not configured

Saving Your Configuration

Save the configuration as described in the Verifying and Saving Your Configuration chapter.

Viewing Collected Bulk Statistics Data

The system provides a mechanism for viewing data that has been collected but has not been transferred. This data is referred to as “pending data”.

View pending bulk statistics data per schema by entering the following:
show bulkstats data

The above command also shows the statistics of remote files, if configured as described in Configuring Optional Settings.

The following is a sample output:

Bulk Statistics Server
Statistics:
  Records
awaiting transmission: 1800
  Bytes
awaiting transmission:   163687 
  Total
records collected:       1800
  Total
bytes collected:         163687 
  Total
records transmitted:     0
  Total
bytes transmitted:       0
  Total
records discarded:       0
  Total
bytes discarded:         0
  Last
collection time required: 2 second(s) 
  Last
transfer time required:   0
second(s) 
  No
successful data transfers
  Last
attempted transfer:       Monday
February 14 15:12:30 EST 2011 


File 1
  Remote
File Format:              %date%%time% 
  File
Header:                     "Format 4.5.3.0"
  File
Footer:                     ""
  Bulkstats
Receivers:
    Primary: 192.168.1.200
using FTP with username root
  File
Statistics:
    Records
awaiting transmission: 1800 
    Bytes
awaiting transmission:   163687 
    Total
records collected:       1800 
    Total
bytes collected:         163687 
    Total
records transmitted:     0
    Total
bytes transmitted:       0
    Total
records discarded:       0
    Total
bytes discarded:         0
    Last
transfer time required:   0
second(s) 
    No
successful data transfers
    Last
attempted transfer:       Monday
February 14 15:12:30 EST 2011


  File
2 not configured


  File
3 not configured


  File
4 not configured

Manually Gathering and Transferring Bulk Statistics

There may be times where it is necessary to gather and transfer bulk statistics outside of the scheduled intervals. The system provides commands that allow you to manually initiate the gathering and transferring of bulk statistics.

These commands are issued from the Exec mode.

To manually initiate the gathering of bulk statistics outside of the configured sampling interval, enter the following command:
bulkstats force gather
To manually initiate the transferring of bulk statistics prior to reaching the of the maximum configured storage limit, enter the following command:
bulkstats force transfer

Clearing Bulk Statistics Counters and Information

It may be necessary to periodically clear counters pertaining to bulk statistics in order to gather new information or to remove bulk statistics information that has already been collected. The following command can be used to perform either of these functions:

clear bulkstats { counters | data }

The clear bulkstats data command clears any accumulated data that has not been transferred. This includes any "completed" files that have not been successfully transferred.

Bulk Statistics Event Log Messages

The stat logging facility captures several events that can be useful for diagnosing errors that could occur with either the creation or writing of a bulk statistic data set to a particular location.

The following table displays information pertaining to these events.
Table 1. Logging Events Pertaining to Bulk Statistics
Event Event ID Severity Additional Information
Local File Open Error 31002 Warning "Unable to open local file filename for storing bulkstats data"
Receiver Open Error 31018 Warning "Unable to open url filename for storing bulkstats data"
Receiver Write Error 31019 Warning "Unable to write to url filename while storing bulkstats data"
Receiver Close Error 31020 Warning "Unable to close url filename while storing bulkstats data"