Table Of Contents
Common Cisco Info Mediator Properties and Command Line Options
Editing Cisco Info Mediator Properties
How Properties and Command Line Options Are Processed at Startup
Cisco Info Mediator Rules File Processing
Elements, Fields, Properties, and Arrays in Rules Files
Assigning Values to Cisco Info Server Fields
Assigning Temporary Elements in Rules Files
Conditional Statements in Rules Files
Including Multiple Rules Files
Rules File Functions and Operators
Connecting to Multiple Cisco Info Servers
Host and Process Utility Functions
Update on Deduplication Function
Monitoring Cisco Info Mediator Loads
Strings and Numerics in One Expression
Using Load Functions to Monitor Nodes
Troubleshooting Cisco Info Mediators
The Cisco Info Mediator Will Not Start
The Cisco Info Mediator Is Not Sending Alerts to the Cisco Info Server
The Cisco Info Mediator Is Losing Events
The Cisco Info Mediator Is Consuming Too Much CPU Time
The Event List Fields Are Not Being Populated Properly
Cisco Info Mediator Information and Error Messages
ProbeWatch and TSMWatch Messages
Cisco Info Mediator Reference
This appendix contains reference information for Cisco Info Mediators, including:
•
Common Cisco Info Mediator Properties and Command Line Options
•
•
•
The information in this appendix applies to all Cisco Info Mediators and TSMs. For introductory information about Cisco Info Mediators, see "Info Mediators."
For more information about specific Cisco Info Mediators, see the individual guides available for each Cisco Info Mediator on the Cisco Systems Support Site.
Common Cisco Info Mediator Properties and Command Line Options
This section describes the properties and command line options common to all Cisco Info Mediators. For the properties and command line options specific to a particular Cisco Info Mediator, see the individual guides for each Cisco Info Mediator available on the Cisco Systems Support Site.
Editing Cisco Info Mediator Properties
You can edit Cisco Info Mediator properties using a text editor or the Properties Editor. The Properties Editor enables you to edit multiple properties files, validates changes you make, and saves the file in the correct format in the correct directory. The Properties Editor is described in Properties Editor.
You can edit the properties file while the Cisco Info Mediator is running. If you do this, the changes you make will take effect the next time you run the Cisco Info Mediator. For an introduction to Cisco Info Mediator properties, see Properties File.
How Properties and Command Line Options Are Processed at Startup
There are default values for each property. In an unedited properties file, all properties are set to the default values, and are commented out with a hash symbol (#) to the beginning of the line.
If you change a setting in the properties file and remove the hash symbol, this overrides the default.
If you change a setting on the command line, this overrides both the default value and the setting in the properties file. To avoid errors, add as many properties as possible to the properties file rather than using the command line options.
If -name is specified, its value determines the name used for the properties file ($OMNIHOME/probes/arch/name.props), rules file ($OMNIHOME/probes/arch/name.rules), and log file ($OMNIHOME/log/name.log).
If -propsfile is specified, its value overrides the name setting for the properties file.
Table E-1 lists the common properties and command line options available to all Cisco Info Mediators, and their default settings.
Table E-1 Common Cisco Info Mediator Properties and Command Line Options
AuthPassword <string>
N/A
The password associated with the username used to authenticate the Cisco Info Mediator when it connects to a Cisco Info Server running in secure mode. This password must be encrypted with the nco_crypt utility.
Secure mode is described in Secure Mode.
AuthUserName <string>
N/A
A username used to authenticate the Cisco Info Mediator when it connects to a Cisco Info Server running in secure mode.
Secure mode is described in Secure Mode.
AutoSAF <integer>
-autosaf
-noautosaf
Turns on automatic store and forward mode. By default, automatic store and forward mode is not enabled.
Store and forward mode is described in Store and Forward Mode.
Buffering <boolean>
-buffer
-nobuffer
Specifies whether or not to use buffering when sending alerts. By default, buffering is not enabled.
BufferSize <integer>
-buffersize <integer>
Number of alerts to buffer. The default is 10.
N/A
-help
Displays the supported command line options and exits.
LookupTableMode <integer>
-lookupmode <integer>
Specifies how table lookups are performed. It can be set to 1, 2, or 3. The default is 1.
If set to 1, all external lookup tables are assumed to have a single value column. Tabs are not used as column delimiters.
If set to 2, all external lookup tables are assumed to have multiple columns. If the number of columns on each line is not the same, an error is generated that includes the file name and the line on which the error occurred.
If set to 3, the rules engine attempts to determine the number of columns in the external lookup table. An error is generated for each line that has a different column count from the previous line. The error includes the file name and the line on which the error occurred.
Lookup tables are described in Lookup Table Operations.
Manager <string>
-manager <string>
Specifies the value of the Manager field for the alert.
MaxLogFileSize <integer>
N/A
Specifies the maximum size the log file can grow. The default is 1 Mbyte. Once the size of the log file reaches the size specified, a second log file is started. When the second file reaches the maximum size, the first file is overwritten with a new log file and the process starts again.
MaxRawFileSize <integer>
N/A
Specifies the maximum size of the raw capture file in Kbytes. The default is unlimited (-1).
If RawCaptureFileBackup is disabled, this property is ignored.
If RawCaptureFileBackup is enabled, this specifies the approximate file size at which a new file will be started.
Raw capture mode is described in Raw Capture Mode.
MaxSAFFileSize <integer>
N/A
Specifies the maximum size the store and forward file can grow to. The default is
1 Mbyte.Store and forward mode is described in Store and Forward Mode.
MessageLevel <string>
-messagelevel <string>
Specifies the lowest level of messages to be written to the message log. Values in ascending order are: debug, info, warn, error, fatal. The default is warn.
MessageLog <string>
-messagelog <string>
Specifies the name of the file to which to send debug and other information messages. This can also be set to STDOUT and STDERR. The default is $OMNIHOME/log/name.log.
MsgDailyLog
-msgdailylog
If enabled, the MsgDailyLog property turns on daily logging. By default, daily logging is not enabled.
Because the time is checked regularly, when MsgDailyLog is set, there will be a slight reduction in performance.
MsgTimeLog
-msgtimelog <string>
Specifies the MsgTimeLog property specifies the time after which the daily log is created. It is set to 0000 (midnight) by default. If MsgDailyLog is not set, this property is ignored.
Name <string>
-name <string>
Specifies the name of the Cisco Info Mediator. If set on the command line, the properties file, rules file, and message log file used will be: $OMNIHOME/probes/arch/name.props, $OMNIHOME/probes/arch/name.rules, and $OMNIHOME/log/name.log, respectively.
NetworkTimeout <integer>
-networktimeout <integer>
Specifies a time in seconds after which the connection to the Cisco Info Server will time out, should a network failure occur. The default value for this property is 0, meaning that no time out occurs.
PollServer <integer>
-pollserver <integer>
If the Cisco Info Mediator is connected to a backup Cisco Info Server because a failover occurred, the Cisco Info Mediator will periodically attempt to reconnect to the primary Cisco Info Server. This property specifies the frequency in seconds at which the Cisco Info Mediator polls for the return of the primary Cisco Info Server.
The default is 0, meaning that no polling occurs.
The BackupInfo Server property, used to designate a backup Cisco Info Server, is described in Specifying Cisco Info Server Properties.
N/A
-propsfile <string>
Specifies the name of the properties file. The default is $OMNIHOME/probes/arch/name.props.
RawCapture <boolean>
-raw
-noraw
Controls the raw capture mode. By default, raw capture mode is disabled.
Raw capture mode is described in Raw Capture Mode.
RawCaptureFile <string>
-capturefile <string>
Specifies the name of the raw capture file. The default is $OMNIHOME/var/name.cap.
Raw capture mode is described in Raw Capture Mode.
RawCaptureFileAppend <boolean>
N/A
Specifies whether new data is appended to the existing raw capture file, instead of overwriting it. By default, the file is overwritten.
Raw capture mode is described in Raw Capture Mode.
RawCaptureFileBackup <boolean>
N/A
Specifies whether the raw capture file is backed up when it exceeds its maximum file size, determined by the MaxRawFileSize property. By default, the file is not backed up.
Raw capture mode is described in Raw Capture Mode.
RetryConnectionCount <integer>
N/A
Specifies the number of events the Cisco Info Mediator should process in store and forward mode before trying to reconnect to the Cisco Info Server. The default is 15.
Store and forward mode is described in Store and Forward Mode.
RetryConnectionTimeOut <integer>
N/A
Specifies the number of seconds the Cisco Info Mediator should process events in store and forward mode before trying to reconnect to the Cisco Info Server. The default is 30.
RulesFile <string>
-rulesfile <string>
Specifies the name of the rules file. The default is: $OMNIHOME/probes/arch/name.rules.
SAFFileName <string>
-saffilename <string>
Specifies the name of the store and forward file. The default is $OMNIHOME/var/name.server.store
In this file name, name is the name of the Cisco Info Mediator and server is the name of the target Cisco Info Server.
Store and forward mode is described in Store and Forward Mode.
Server <string>
-server <string>
Specifies the name of the Cisco Info Server that receives alerts. The default is NCOMS. You can specify more than one Cisco Info Server by using colons to separate the names, for example:
Server: "NCOMS:NCOMS_HIGH:NCOMS_LOW". This enables you to use multiple Cisco Info Servers in Cisco Info Mediator rules files, as described in Connecting to Multiple Cisco Info Servers.
Server Backup <string>
N/A
Specifies a secondary Cisco Info Server should the primary Cisco Info Server connection fail. If NetworkTimeout is set and a virtual Cisco Info Server is in use, use ServerBackup to refer to your secondary Cisco Info Server.
StoreAndForward <boolean>
-saf
-nosaf
Controls the store and forward operations. By default, store and forward mode is enabled.
Store and forward mode is described in Store and Forward Mode.
N/A
-version
Displays version information and exits.
Cisco Info Mediator Rules File Processing
A Cisco Info Mediator takes an event stream and parses it into elements. Event elements are converted into Cisco Info Server fields using Cisco Info Mediator rules. The Identifier field, used by the Cisco Info Server for deduplication, is also created. The fields are then inserted into alerts.status table as an alert.
For introductory information about rules files, see The Rules File. For information about the Identifier field, see Creating a Unique Identifier.
This section describes rules file syntax. It contains the following sections:
•
•
•
•
Changes made to the rules file require the Cisco Info Mediator to be restarted or made to re-read the rules file (the latter is preferable, because the Cisco Info Mediator will not lose events). You can force the Cisco Info Mediator to re-read the rules file by issuing a kill -HUP <pid> command on the Cisco Info Mediator process ID (PID). See the ps and kill man pages for more information.
Elements, Fields, Properties, and Arrays in Rules Files
A Cisco Info Mediator splits the event stream into elements, indicated by the $ symbol in the rules file. For example, $Node is an element containing the node name of the event source. You can assign elements to Cisco Info Server fields, indicated by the @ symbol in the rules file.
Assigning Values to Cisco Info Server Fields
You can assign values to Cisco Info Server fields in the following ways:
•
•
•
Numeric values can be expressed in decimal or hexadecimal form. The following statements, which set the Class field to 100, are equivalent:
•
•
In addition to assigning elements to fields, you can use the processing statements, operators, and functions described in this chapter to manipulate these values in rules files before assigning them. Elements are stored as strings, so you need to use the int function, described in Math Functions, to convert them into integers before performing numeric operations.
Assigning Temporary Elements in Rules Files
You can create a temporary element in a rules file by assigning it to an expression, for example:
$tempelement = "message"
An element, tempelement, is created and assigned the string value message.
If you refer to an element that has not been initialized in this way, the element is set to the null string ("").
To create the element $b and set it to setnow:
$b="setnow"
To then set the element $a to setnow:
$a=$b
In the following example, temporary elements are used to extract information from the Summary element, which has the string value: The Port is down on Port 1 Board 2.
$temp1 = extract ($Summary, "Port ([0-9]+)")
$temp2 = extract ($Summary, "Board ([0-9]+)")
@AlertKey = $temp1 + "." + $temp2The extract function is used to assign values to temporary elements temp1 and temp2. Then these elements are concatenated with a . separating them, and assigned to the AlertKey field. After these statements are executed, the AlertKey field has the value 1.2. The extract function is described in String Functions, and the concatenate operator (+) is described in Math and String Operators.
Assigning Properties
You can assign the value of a Cisco Info Mediator property, as defined in the properties file or on the command line, to a field value. A property is represented by a % symbol in the rules file. For example, you could add the following statement to your rules file:
@Summary = "Server = " + %ServerIn this example, when the rules file is processed, the Cisco Info Mediator searches for a property named Server. If the property is found, its value is concatenated to the text string and assigned to the Summary field. If the property is not found, a null string ("") is assigned.
You can also assign values to a property in the rules file. If the property does not exist, it is created. For example, you could create a property called Counter to keep track of the number of events that have been processed as follows:
if (match(%Counter,""))
{%Counter = 1}
else {%Counter = int(%Counter) + 1}These properties retain their values across events and when the rules file is re-read.
Most Cisco Info Mediators read properties once at startup, so changing Cisco Info Mediator properties after startup does not usually affect Cisco Info Mediator behavior. However, the RawCapture property can be set in the rules file, so that you can send the raw event data to a file only when certain conditions are met.
# Top of rules processing
%RawCapture=0
if (condition) {
# Want this event written to raw capture file.
%RawCapture=1
}The IF statement is described in Conditional Statements in Rules Files.
You can also enable raw capture mode by setting the -raw command line option or the RawCapture property in the Cisco Info Mediator properties file, as described in Raw Capture Mode.
Using Arrays
You must define arrays at the start of a rules file, before any processing statements. (You must also define tables, described in Lookup Table Operations, before any processing statements.) To define an array, use the following syntax:
array node_arr;
Arrays are one dimensional. Each time an assignments is made for a key value that already exists, the previous value is overwritten. In the example:
node_arr["myhost"] = "a";
node_arr["yourhost"] = "b";
node_arr["myhost"] = "c";After the preceding statements have been executed, there are two items in the node_arr array. The item with the key myhost is set to c, and the item with the key yourhost is set to b. You can make assignments using Cisco Info Mediator elements, for example:
node_arr[$Node] = "d";
Note
Conditional Statements in Rules Files
The IF and SWITCH statements provide condition testing for processing elements in rules files.
The IF Statement
A condition is a combination of expressions and operations that resolve to either TRUE or FALSE. The IF statement allows conditional execution of a set of one or more assignment statements by executing only the rules for the condition that is TRUE. It has the following syntax:
if (<condition>) {
rules
} [else if (<condition>) {
rules
} ...]
[else (<condition>) {
rules
}]You can combine conditions into increasingly complex conditions using the logical AND operator (&&), which is true only if all of its inputs are true, and OR operator (||), which is true if any of its inputs are true. For example:
if match ($Enterprise, "Acme") && match ($trap-type, "Link-Up") {
@Summary = "Acme Link Up on " + @Node
}The operators used in this example are described in Rules File Functions and Operators.
The SWITCH Statement
A SWITCH statement transfers control to a set of one or more rules assignment statements depending on the value of an expression. It has the following syntax:
switch (expression) {
case "stringliteral":
rules
case "stringliteral":
rules
...
default:
[rules]
}The SWITCH statement tests for exact matches only. This statement should be used wherever possible in place of an IF statement because SWITCH statements can be processed more efficiently and therefore execute more quickly.
Note
The expression can be any valid expression. For example:
switch($node)The stringliteral can be any string value. For example:
case "jupiter":You can have more than one stringliteral separated by the | operator. For example:
case "jupiter" | "mars" | "venus":This case is executed if the expression matches any of the specified strings.
Including Multiple Rules Files
You can include a number of rules files in your main rules file using the include statement:
include "rulesfile"
Specify the path to the rules file as an absolute path. A relative path is relative to the Cisco Info Mediator working directory, which can vary depending on how the Cisco Info Mediator is started. You cannot use environment variables in the path.
if(match(@Manager, "ProbeWatch"))
{
include "/opt/Omnibus/probes/solaris2/probewatch.rules"
}
else
...Rules File Functions and Operators
You can use the operators and functions described in this section to manipulate elements in rules files before assigning them to Cisco Info Server fields.
Table E-2 lists the operators described in the following sections.
Table E-3 lists the functions described in the following sections.
Math and String Operators
You can use math operators to add, subtract, divide, and multiply numeric operands in expressions. Table E-4 describes the math operators supported in rules files.
You can use string operators to manipulate character strings. Table E-5 describes the string operator supported in rules files.
Table E-5 String Operator
+
Concatenates two or more strings.
@field = $<element1> + "<message>" + $<element2>
Bit Manipulation Operators
You can use bitwise operators to manipulate integer operands in expressions. Table E-6 describes the bitwise operators supported in rules files.
These operators manipulate the bits in integer expressions. For example, in the statement:
$result2 = int($<number3>) >> 1
If <number3> has the value 17, <result2> resolves to 8, as shown:
16 8 4 2 1
>>1 0 0 0 1
0 1 0 0 0
Note the bits do not wrap around; when they are pushed off one end, they are replaced on the other end by a 0.
Bitwise operators only work with integer expressions. Elements are stored as strings, so you need to use the int function, described in Math Functions, to convert them into integers before performing these operations.
Comparison Operators
You can use comparison operators to test numeric values for equality and inequality. Table E-7 describes the comparison operators supported in rules files.
Logical Operators
You can use logical operators on boolean values to form expressions that resolve to TRUE or FALSE. Table E-8 describes the logical operators supported in rules files.
Connecting to Multiple Cisco Info Servers
The setobjectserver function enables you to specify the Cisco Info Server to which an alert will be sent. The setdefaultobjectserver function enables you to specify the default Cisco Info Server to which alerts are sent when none is specified.
The syntax to set the Cisco Info Server to which alerts are sent is:
setobjectserver(<string_expression>)
For example:
setobjectserver(upper($ServerName))The syntax to set the default Cisco Info Server is:
setdefaultobjectserver(<string_expression>)For example:
setdefaultobjectserver("NCOMS2")Any Cisco Info Servers specified in these functions must first be identified in the Server property or -server command line option, described in Common Cisco Info Mediator Properties and Command Line Options. An example properties file entry is:
Server: "TEST1:TEST2:TEST3"
The first Cisco Info Server specified, in this case TEST1, is the default Cisco Info Server. You can change both the default Cisco Info Server and the Cisco Info Server to which specific alerts are sent, as shown in the following example:
# Once an event of Major severity or higher comes in, set the default Info Mediator to TEST2
if(int(@Severity) > 3)
{
setdefaultobjectserver("TEST2")
}
# Send all clear events to TEST3
if (int(@Severity) = 0)
{
setobjectserver("TEST3")
}Existence Function
You can use the exists function to test for the existence of an element, with the following syntax:
exists ($<element>)
The function returns TRUE if the element was created for this particular event; otherwise it returns FALSE.
Deleting Elements or Events
You can use functions to remove elements from an event, discard an entire event, and recover a discarded event. Table E-9 describes these functions.
String Functions
You can use string functions to manipulate string elements, typically field or element names. Table E-10 describes the string functions supported in rules files.
Table E-10 String Functions
upper(<expression>)
Converts an expression to uppercase.
$Node = upper($Node)
lower(<expression>)
Converts an expression to lowercase.
$Node = lower($Node)
length(<expression>)
Calculates the length of an expression and returns the numeric value.
$LengthNode = length($Node)
rtrim(<expression>)
Removes white space from the right of an expression.
$TrimNode = rtrim($Node)
ltrim(<expression>)
Removes white space from the left of an expression.
$TrimNode = ltrim($Node)
scanformat(<expression>, "<string>")
Converts the expression according to the following formats, similar to the scanf family of routines in C. Conversion specifications are:
%% - literal %; do not interpret.
%d - matches an optionally signed decimal integer.
%u - same as %d; no check is made for sign.
%o - matches an optionally signed octal number.
%x - matches an optionally signed hexadecimal number.
%i - matches an optionally signed integer.
%e, %f, %g - matches an optionally signed floating point number.
%s - matches a string terminated by white space or end of string.
$element = "Foo is up in 15 seconds"
[$node, $state, $time] = scanformat($element,
"%s is %s in %d seconds")This sets $node, $state, and $time to Foo, up, and 15, respectively.
substr(<expression>,<n>,
<len>)Extracts a substring, starting at the position specified in the second parameter, for the number of characters specified by the third parameter.
$Substring = substr($Node,2,10)
printable(<expression>)
Converts any non-printable characters in the given expression into a space character.
$Print = printable($Node)
match(<expression>,
"string")TRUE if the expression value matches the string exactly.
if match($Node, "New")
nmatch(<expression>,
"string")TRUE if the expression starts with the specified string.
if nmatch($Node, "New")
regmatch(<string>, "regexp")
Full regular expression matching of the value in the regular expression in the string (which can be a field, element, or string expression).
For more information on regular expressions, see "Regular Expressions."
if (regmatch($enterprise, "^Acme Config:[0-9]"))
extract(<string>,"regexp")
The extract function returns the part of the string (which can be a field, element, or string expression) that matches the parenthesized section of the regular expression. Regular expression pattern matching is described in "Regular Expressions.".
extract ($expr,"ab([0-9]+)cd")
If $expr is "ab123cd" then the value returned is 123.
expand("string")
The expand function returns the string (which must be a literal string) with escape sequences expanded. Possible expansions are:
\" - double quote
\NNN - octal value of NNN
\\ - backslash
\a - alert (BEL)
\b - backspace
e - escape (033 octal)
f - form feed
n - new line
r - carriage return
\t - horizontal tab
v - vertical tab
This function cannot be used as the regular expression argument in the regmatch or extract functions.
log(debug, expand("Rules file with embedded \\\""))
sends the following to the log:
Sun Oct 21 19:56:15 2001 Debug: Rules file with embedded \"
Math Functions
You can use math functions to perform numeric operations on elements. Elements are stored as strings, so you must use these functions to convert them into integers before performing these operations. Table E-11 describes the math functions supported in rules files.
In the following example, the severity of an alert that monitors disk space usage is set depending on the amount of available disk space.
if (int($PercentFull) > 80 && int($PercentFull) <=85)
{
@Severity=2
}
else if (int($PercentFull)) > 85 && int($PercentFull) <=90)
{
@Severity=3
}
else if (int($PercentFull > 90 && int($PercentFull) <=95)
{
@Severity=4
}
else if (int($PercentFull) > 95)
{
@Severity=5
}The percentage of disk space is not always provided in the event stream. The percentage of disk space can be calculated in the rules file as follows:
if (int($total) > 0)
{
@DiskSpace=(100*int($diskspace))/int($total)
}This can also be calculated using the real function:
if (int($total) > 0)
{
@DiskSpace=(real($diskspace)/real($total))*100
}The previous example can then be used to set the severity of the alert.
Date and Time Functions
You can use date and time functions to obtain the current time or to perform date and time conversions. Times are specified in Coordinated Universal Time (UTC)—the number of elapsed seconds since 1 January 1970. Table E-12 describes the date and time functions supported in rules files.
Host and Process Utility Functions
You can use utility functions to obtain information about the environment in which the Cisco Info Mediator is running. Table E-13 describes the host and process functions supported in rules files.
Lookup Table Operations
Lookup tables provide a way to add extra information in an event. A table is made up of a list of keys and values. It is defined by using the table keyword and accessed using the lookup keyword. You can create a lookup table directly in the rules file or in a separate file.
The lookup keyword evaluates the expression in the keys of the named table and returns the associated value. If the key is not found, an empty string is returned. The lookup function has the following syntax:
lookup(<expression>,<tablename>)
Table definitions must appear at the start of a rules file, before any processing statements. You can define multiple tables in a rules file.
You must restart the Cisco Info Mediator before any changes made to a lookup table take effect.
Defining Lookup Tables in the Rules File
You can create a lookup table directly in the rules file using the following format:
table tablename={{"<key>","<value>"},{"<key>","<value>"}...}
For example, to create a table that matches a node name to the department the node is in:
table dept={{"node1","Technical"},{"node2","Finance"}}You can access this lookup table in the rules file as follows:
@ExtraChar=lookup(@Node,dept)This example uses the @Node field as the key. If the value of the @Node field matches a key in the table, @ExtraChar is set to the corresponding value.
A lookup table can also have multiple columns. For example:
table example_table =
{{"key1", "value1", "value2", "value3"},
{"key2", "val1", "val2", "val3"}}You can obtain values from a multiple value lookup table as follows:
[@Summary, @AlertKey, $error_code] = lookup("key1", "example_table")Defining Lookup Tables in a Separate File
Rather than including the lookup table directly in the rules file, you can create the table in a separate file. If you are specifying a single value, the file must be in the format:
key[TAB]value
key[TAB]valueTo create a table in which the node name is matched to the department the node is in, use the following format:
node1[TAB]Technical
node2[TAB]FinanceDefine the table file, preceding any processing statements, by specifying the full path to the file as follows:
table dept="/opt/Omnibus/probes/solaris2/Dept"You can then use this lookup table in the rules file as follows:
@ExtraChar=lookup(@Node,dept)For multiple values, the format is:
key1[TAB]<value1>[TAB]<value2>[TAB]<value3>
key2[TAB]<val2>[TAB]<val3>[TAB]<val3>
The number of values in each row of the lookup table must match the number of values obtained from the lookup command. If they do not match, an error is generated.
The following example is incorrect, because the first entry has three values while the second only has two values:
table example_table =
{{"key1", "value1", "value2", "value3"},
{"key2", "val1", "val2"}}The following example is also incorrect, because the lookup table holds three values, but only two fields are being set by the lookup command:
table example_table =
{{"key1", "value1", "value2", "value3"},
{"key2", "val1", "val2", "val3"}}[@Summary, @AlertKey] = lookup("key1", "example_table")
Specifying Default Table Values
You can specify a default option to handle an event that does not match any of the key values in a table. The default statement must follow the specific table definition.
The following is an example for a table in the rules file:
table example_table =
{{"key1", "value1", "value2", "value3"},
{"key2", "val1", "val2", "val3"}}
default = {"defval1", "defval2", "defval3"}The following is an example for a table in a separate file:
table dept="/opt/Omnibus/probes/solaris2/Dept"
default="UNKNOWN"Update on Deduplication Function
The deduplication process is managed by the Cisco Info Server, but it can be configured in the Cisco Info Mediator rules file. In each rules file you can specify which fields of an alert are to be updated if the alert is deduplicated using the update function. This allows deduplication rules to be set on a per-alert basis.
The update function has the following syntax options:
update (<fieldname>)
or
update (<fieldname>, [TRUE | FALSE])
If set to TRUE, update on deduplication is enabled. If set to FALSE, update on deduplication is disabled. The default is TRUE.
For example, to ensure the Severity field is updated on deduplication, you can add the following to the rules file:
update (@Severity)You can also disable update on deduplication for a specific field. For example:
update (@Tally, FALSE)
Note
Details Function
Details are extra elements created by a Cisco Info Mediator to display alert information that is not stored in a field of the alerts.status table. Alerts do not have detail information unless it is added.
Detail elements are stored in the Cisco Info Server details table, alerts.details. You can view details by double-clicking an alert in the Event List.
You can add information to the details table using the details function. The detail information is added when an alert is inserted, but not if it is deduplicated.
The following example adds the elements $a and $b to the alerts.details table:
details($a,$b)The following example adds all of the alert information to the alerts.details table:
details ($*)
Note
In the following example, $Summary is compared to the strings Incoming and Backup. If there is no match, $Summary is set to the string, and all of the information for the event is added to the details table:
if (match($Summary, "Incoming"))
{
@Summary = "Received a call"
}
else if(match($Summary, "Backup"))
{
@Summary = "Attempting to back up"
}
else
{
@Summary = "Please see details"
details($*)
}Message Logging Functions
You can use the log function to log messages during rules processing. You can also set a log level using the setlog function, and only messages equal to or above that level are logged. There are five log levels: DEBUG, INFO, WARNING, ERROR, and FATAL, in order of increasing severity. For example, if the log level is set to WARNING, only WARNING, ERROR, and FATAL messages are logged, however, if the logging is set to ERROR, only ERROR and FATAL messages are logged.
Log Function
The log function sends a message to the log file.
The syntax of the log function is:
log([DEBUG | INFO | WARNING | ERROR | FATAL],"<string>")
Note
Setlog Function
The setlog function sets the minimum level at which messages are logged during rules processing. By default, the level for logging is WARNING and above.
The syntax of this function is:
setlog([DEBUG | INFO | WARNING | ERROR | FATAL])
Message Logging Example
The following is a sequence of log functions executed in the rules file:
setlog(WARNING)
log(DEBUG,"A debug message")
log(WARNING,"A warning message")
setlog(ERROR)
log(WARNING,"Another warning message")
log(ERROR,"An error message")This produces log output of:
A warning message.
An error message.
The DEBUG level message is not logged, because the logging setting is set higher than DEBUG. The second WARNING level message is not logged, because the preceding setlog function has set the log level higher than WARNING.
Service Function
Use the service function to define the status of a service before alerts are forwarded to the Cisco Info Server. The status changes the color of the alert when it is displayed in the Event List and Service windows.
The syntax of the service function is:
service(<service_identifier>, <service_status>)
The <service_identifier> identifies the monitored service, for example, $host.
Table E-14 lists the service status levels.
Service Function Example
If you want a Ping Info Mediator to return a service status for each host it monitors, you can use the service function in the rules file to assign a service status to each alert. The following example shows an extract from a rules file that assigns a service status to each alert based on the value of the status element.
switch ($status)
{
case "unreachable":
@Severity = "5"
@Summary = @Node + " is not reachable"
@Type = 1
service($host, bad) # Service Entry
case "alive":
@Severity = "3"
@Summary = @Node + " is now alive"
@Type = 2
service($host, good) # Service Entry
case "noaddress":
@Severity = "2"
@Summary = @Node + " has no address"
service($host, marginal) # Service Entry
case "removed":
@Severity = "5"
@Summary = @Node + " has been removed"
service($host, marginal) # Service Entry
case "slow":
@Severity = "2"
@Summary = @Node + " has not responded within trip time"
service($host, marginal) # Service Entry
case "newhost":
@Severity = "1"
@Summary = @Node + " is a new host"
service($host, good) # Service Entry
case "responded":
@Severity = "0"
@Summary = @Node + " has responded"
service($host, good) # Service Entry
default:
@Summary = "Ping Info Mediator Error details: " + $*
@Severity = "3"
service($host, marginal) # Service Entry
}Monitoring Cisco Info Mediator Loads
To monitor load, it is necessary to obtain time measurements and calculate the number of events processed over time. The updateload function takes a time measurement each time it is called and the getload function returns the load as events per second.
Each time the updateload function is executed, the current time stamp, recorded in seconds and microseconds, is added to the beginning of a series of time stamps. The remaining time stamps record the difference in time from the previous time stamp. For example, to take a time measurement and update a property called load with a new time stamp:
%load = updateload(%load)
Note
You can specify a maximum time window for which samples are kept, and a maximum number of samples. By default, the time window is one second and the maximum number of samples is fifty. You can specify the number of seconds for which load samples are kept and the maximum number of samples in the format:
<time window in seconds>.<max number of samples>
For example, to set or reset these values for the load property:
%load = "2.40"When the number of seconds in the time window is exceeded, any samples outside of that time window are removed. When the number of samples reaches the limit, the oldest measurement is removed.
The getload function calculates the current load, returned as events per second. For example, to calculate the current load and assign it to a temporary element called current_load:
$current_load = getload(%load)In the following example, the load property is initialized the first time the rules file is executed with a sample time window of three seconds and a maximum of ten samples. For each following execution, the current load is logged to the Cisco Info Mediator log file.
if(match(%load, "")){
%load="3.10"
}
%load = updateload(%load)
$current_load = getload(%load)
log(DEBUG, "Events per second is " + $current_load)Testing Rules Files
You can test the syntax of a rules file using the Syntax Info Mediator: nco_p_syntax. This is more efficient than actually running the Cisco Info Mediator to test whether the syntax of the rules file is correct.
To run the Syntax Info Mediator, enter:
nco_p_syntax -rulesfile /test/rulesfile
In this command, /test/rulesfile is the rules file. The Syntax Info Mediator always runs in debug mode. It connects to the Cisco Info Server, tests the rules file, displays any errors to the screen, and then exits. If no errors are displayed, the syntax of the rules file is correct.
Debugging Rules Files
When making changes to the rules file, adding new rules, or creating lookup tables, it is useful to test the Cisco Info Mediator by executing it in debug mode. This shows exactly how an event is being parsed by the Cisco Info Mediator and any possible problems with the rules file.
You can set the Cisco Info Mediator to run in debug mode on the command line or in the properties file. To enable debug mode on the command line, enter the command:
$OMNIHOME/probes/arch/<Info Mediator name> -messagelevel DEBUG -messagelog STDOUT
Alternatively, you can enter the following in the properties file:
MessageLevel: "DEBUG"
MessageLog: "STDOUT"
If you omit the MessageLog property or command line option, the debug information is sent to the Cisco Info Mediator log file in the $OMNIHOME/log directory rather than to the screen.
Rules File Examples
The following sections show examples of typical rules file segments.
Enhancing the Summary Field
This example rule tests if the $trap-type element is "Link-Up". If it is, the @Summary field is populated the with a string made up of "Link up on ", the name of the node from the record being generated, " Port ", and the value of the $ifIndex element:
if(match($trap-type,"Link-Up"))
{
@Summary = "Link up on " + @Node + " Port " + $ifIndex
}Populating Multiple Fields
This example rule is similar to the previous rule except the @AlertKey and @Severity fields are also populated:
if(match($trap-type, "Link-Up"))
{
@Summary = "Link up on " + @Node + " Port " + $ifIndex
@AlertKey = $ifIndex
@Severity = 4
}Nested IFs
This example rule first tests if the trap has come from an Acme manager and then tests if it is a "Link-Up". If both conditions are true, the @Summary field is populated with extra information:
if(match($enterprise,"Acme"))
{
if(match($trap-type, "Link-Up"))
{
@Summary= "Acme Link Up on " + @Node + " Port " + $ifIndex + " Reason: "+$ifLocReason
}}Regular Expression Match
This example rule tests for a line starting with "Acme Configuration:" followed by a single digit:
if (regmatch($enterprise,"^Acme Configuration:[0-9]"))
{
@Summary="Generic configuration change for "+@Node
}Regular Expression Extract
This example rule tests for a line starting with "Acme Configuration:" followed by a single digit. Then it extracts that single digit and places it in the @Summary field:
if (regmatch($enterprise,"^Acme Configuration:[0-9]"))
{
@Summary="Acme error "+extract($enterprise,"^Acme Configuration: ([0-9])")+"on"+@Node
}Numeric Comparisons
This example rule tests the value of an element called $freespace as a numeric value by converting it to an integer and performing a numeric comparison:
if (int($freespace) < 1024)
{
@Summary="Less than 1024K free on drive array"
}Simple Numeric Expressions
This example rule creates an element called $tmpval. The value of $tmpval is derived from the $temperature element which is converted to an integer and then has 20 subtracted from it. The string element $tmpval contains the result of this calculation:
$tmpval=int($temperature)-20Strings and Numerics in One Expression
This example rule creates an element called $Kilobytes. The value of $Kilobytes is derived from the $DiskSize element, which divided by 1024 before being cast back into a string type with the letter K appended:
$Kilobytes = string(int($DiskSize)/1024) + "K"Using Load Functions to Monitor Nodes
This example shows how to measure load for each node that is generating events. If a node is producing more than five events per second, a warning is written to the Cisco Info Mediator log file. If more than 80 events per second are generated for all nodes being monitored by the Cisco Info Mediator, events are sent to an alternate Cisco Info Server and a warning is written to the Cisco Info Mediator log file.
# set the following in the probes properties file: Server: "HIGHLOAD:LOWLOAD"
# declare an array, loads, at the top of the rules file array loads;
# initialize array items with the number of seconds samples may span and
# number of samples to maintain.
if (match("", loads[@Node])){
loads[@Node] = "2.50"
}
if (match("" , %general_load)){
%general_load="2.50"
}
loads[@Node] = updateload(loads[@Node])
%general_load=updateload(%general_load)
if (int(getload(loads[@Node])) > 5){
log(WARN, $Node + " is creating more than 5 events per second")
}
if (int(getload(%general_load)) > 80){
log(WARN, "Info Mediator is creating more than 80 events per second - switching to HIGHLOAD")
setobjectserver("HIGHLOAD")
}Troubleshooting Cisco Info Mediators
This section describes some of the common problems experienced by Cisco Info Center users and explains possible causes and solutions.
This troubleshooting information is divided into two sections:
Table E-15 Host and Process Utility Functions
This section contains a list of common problem causes. If you are unsure what your problem is, you should start by reading this section and following the instructions. If you cannot solve your problem by following the instructions in this part, move on to What To Do If.
This section describes common symptoms caused by Cisco Info Mediator problems and step-by-step instructions to help you locate and solve the problem. If none of the headings in this section match the symptoms of your problem, read through the lists of instructions and make sure you have tried all of the most likely solutions listed there.
If you have tried all of the suggested problem solutions and your Cisco Info Mediator still does not work, see your support contract and contact the help desk.
Common Problem Causes
The most common causes of Cisco Info Mediator problems are:
•
•
•
For information about setting the OMNIHOME and NETCOOL_LICENSE_FILE variables, see the Cisco Info Center Installation and Configuration Guide, 3.6.
For information about solving rules file problems, see Testing Rules Files and Debugging Rules Files.
For information about Cisco Info Mediator properties, see Common Cisco Info Mediator Properties and Command Line Options. Ensure all of the properties are set correctly in the Cisco Info Mediator properties file. For example, ensure the Server property contains the correct Cisco Info Server name and the RulesFile property contains the correct rules file name.
If you cannot solve the problem, read through the next section and make sure you have tried all of the most likely solutions listed there.
What To Do If
The headings in this section describe the most common symptoms of Cisco Info Mediator problems. Find the heading that most closely describes your problem, then follow the instructions until you locate the cause and solve the problem.
Table E-16 Cisco Info Mediator Problems
The Cisco Info Mediator will not start.
The Cisco Info Mediator is not sending alerts to the Cisco Info Server.
The Cisco Info Mediator Is Not Sending Alerts to the Cisco Info Server.
The Cisco Info Mediator is losing events.
The Cisco Info Mediator is consuming too much CPU time.
The Event List fields are not being populated properly.
If none of the headings match the symptoms of your problem, read through the lists of instructions and make sure you have tried all of the most likely solutions listed there. If you have tried all of the suggested problem solutions and your Cisco Info Mediator still does not work, see your support contract and contact the help desk.
The Cisco Info Mediator Will Not Start
If the Cisco Info Mediator does not start:
Step 1
./<Info Mediator name> -server <servername> -messagelevel DEBUG &
where <Info Mediator name> is the name of the Cisco Info Mediator that is not working and <servername> is the name of the server the Cisco Info Mediator is attempting to send alerts. Any errors are recorded in the $OMNIHOME/log/<Info Mediatorname>.log debug log file. Check the errors recorded in the debug log file to locate the problem.
For more information about debugging rules files, see Debugging Rules Files.
Step 2
If you can log in successfully, the Cisco Info Server is running. If the Cisco Info Server is not running, this is likely to be the cause of the problem. For more information about using the Cisco Info Server and nco_sql, see Direct Access Using the SQL Interactive Interface (nco_sql).
Step 3
Step 4
/usr/bin/ps -ef | grep nco
A list of Cisco Info Center processes is displayed. Ensure none of the processes correspond to the same type of Cisco Info Mediator. Cisco Info Center does not allow you to run two identical Cisco Info Mediator configurations because this would duplicate all of the events forwarded to the Cisco Info Server.
Step 5
$NCLICENSE/bin/nc_print_license
If you do not have enough licences to run another Cisco Info Mediator and you cannot stop any of the other Cisco Info Mediators, contact the help desk to request another license.
Step 6
Step 7
See Testing Rules Files and Debugging Rules Files for more information about how to do this.
Step 8
See the df and top man pages for more information about using these commands.
Step 9
If your disk is full, the Cisco Info Mediators and Cisco Info Servers will not be able to work properly.
Note
Step 10
If the store and forward file has been corrupted, there should be an error message in the $OMNIHOME/log/<Info Mediatorname>.log log file. If the file is corrupted, delete it and restart the Cisco Info Mediator.
Step 11
$OMNIHOME/bin/arch/<Info Mediatorname> -version
Step 12
If You Are Running the Cisco Info Mediator on a Remote Host
Step 1
See the Cisco Info Center Installation and Configuration Guide, 3.6 for more information about how to do this.
Step 2
Try to ping the Cisco Info Server host machine using the host name and the IP address. See the ping man page for more information about how to do this.
If you cannot connect to the Cisco Info Server host using the ping command, there is a problem with the connection between your Cisco Info Mediator host and your Cisco Info Server host.
Step 3
See the Cisco Info Center Installation and Configuration Guide, 3.6 for more information.
Step 4
Step 5
The Cisco Info Mediator Is Not Sending Alerts to the Cisco Info Server
If the Cisco Info Mediator is not sending alerts to the Cisco Info Server:
Step 1
ps -ef | grep nco
A list of Cisco Info Center processes is displayed. If the Cisco Info Mediator is not running, start the Cisco Info Mediator from the command line.
Step 2
ps -ef | grep nco
A list of Cisco Info Center processes is displayed. Ensure none of the processes correspond to the same type of Cisco Info Mediator. Cisco Info Center does not allow you to run two identical Cisco Info Mediator configurations because this would duplicate all of the events forwarded to the Cisco Info Server.
Step 3
For example, ensure the Server property contains the correct Cisco Info Server name and the RulesFile property contains the correct rules file name.
Step 4
Step 5
Step 6
Try to ping the Cisco Info Server host machine using the host name and the IP address. See the ping man page for more information about how to do this.
If you cannot connect to the Cisco Info Server host using the ping command, there is a problem with the connection between your Cisco Info Mediator host and your Cisco Info Server host.
Step 7
See the documentation supplied with your Element Manager for more information about how to do this.
Step 8
Step 9
To do this, check the $OMNIHOME/var/<Info Mediatorname>.saf and $OMNIHOME/var/<Info Mediatorname>.reco files to see if they are growing. If they are, disable store and forward in the Cisco Info Mediator's properties file. See Properties File for more information.
Step 10
You can do this using df -k or top commands. See the df and top man pages for more information about using these commands.
If You Are Running the Cisco Info Mediator on a Remote Host
Step 1
Try to ping the Cisco Info Server host machine using the host name and the IP address. See the ping man page for more information about how to do this.
If you cannot connect to the Cisco Info Server host using the ping command, there is a problem with the connection between your Cisco Info Mediator host and your Cisco Info Server host.
Step 2
See the Cisco Info Center Installation and Configuration Guide, 3.6 for more information.
Step 3
Step 4
The Cisco Info Mediator Is Losing Events
If not all events are being forwarded to the Cisco Info Server:
Step 1
./<Info Mediatorname> -server <servername> -messagelevel DEBUG &
In this command, <Info Mediatorname> is the name of the Cisco Info Mediator that is not working and <servername> is the name of the Cisco Info Server the Cisco Info Mediator is attempting to send alerts. Any errors are recorded in the debug rules file. Check the errors recorded in the debug log file to locate the problem.
For more information about debugging rules files, see Debugging Rules Files.
Step 2
See the documentation supplied with your Element Manager for more information about how to do this.
Step 3
Step 4
For example, ensure the Server property contains the correct Cisco Info Server name and the RulesFile property contains the correct rules file name.
The Cisco Info Mediator Is Consuming Too Much CPU Time
If the Cisco Info Mediator is consuming too much CPU time:
Step 1
./<Info Mediatorname> -server <servername> -messagelevel DEBUG &
where <Info Mediatorname> is the name of the Cisco Info Mediator that is not working and <servername> is the name of the Cisco Info Server the Cisco Info Mediator is attempting to send alerts to. Any errors are recorded in the debug log file. Check the errors recorded in the debug log file to locate the problem.
For more information about debugging rules files, see Debugging Rules Files.
Step 2
Step 3
Step 4
If your disk is full, the Cisco Info Mediators and Cisco Info Servers will not be able to work properly.
Note
Step 5
If it has been corrupted, an error message should exist in the
$OMNIHOME/log/<Info Mediatorname>.log log file.Step 6
The Event List Fields Are Not Being Populated Properly
If the Cisco Info Mediator is detecting events and forwarding them to the Cisco Info Server but the Event List fields are not being populated correctly:
Step 1
./<Info Mediatorname> -server <servername> -messagelevel DEBUG &
In this command, <Info Mediatorname> is the name of the Cisco Info Mediator that is not working and <servername> is the name of the Cisco Info Server the Cisco Info Mediator is attempting to send alerts. Any errors are recorded in the debug rules file. Check the errors recorded in the debug log file to locate the problem.
For more information about debugging rules files, see Debugging Rules Files.
Step 2
See Cisco Info Mediator Rules File Processing for more information about configuring rules files.
Step 3
See "Cisco Info Server SQL," for more information about using the SQL interactive interface to view the table information.
Cisco Info Mediator Information and Error Messages
This section lists all messages not Cisco Info Mediator-specific, including ProbeWatch and TSMWatch messages.
See the individual Cisco Info Mediator guides for information about Cisco Info Mediator-specific messages.
Generic Error Messages
Cisco Info Mediators can generate the following types of messages:
•
•
•
•
•
Fatal Level Messages
The Cisco Info Mediator automatically terminates when a fatal message is issued. See Table E-17 for a list and description of the fatal messages.
Error Level Messages
The Cisco Info Mediator is likely to terminate when an error message, as described in Table E-18, is issued.
Warning Level Messages
These messages, shown in Table E-19, are issued as warnings, however, they should not cause the Cisco Info Mediator to terminate.
Information Level Message
This message is for information purposes.
Table E-20 Information Level Cisco Info Mediator Message
Using stderr for logging.
The Cisco Info Mediator was unable to open a log file, so it is writing messages to stderr.
Debug
The debug messages, listed and described in Table E-21, give detailed error and information messages on the internal functioning of the Cisco Info Mediator. These messages are aimed at Cisco Info Mediator developers, however, they are listed here for completeness.
ProbeWatch and TSMWatch Messages
In certain situations, a Cisco Info Mediator or TSM generates events of their own. These events can provide information (such as startup or shutdown messages) or identify problems. This section describes the elements common to all ProbeWatch and TSMWatch messages.
ProbeWatch and TSMWatch messages are processed in the rules file and converted into alerts, like other events. Table E-22 shows the elements common to ProbeWatch and TSMWatch events.
Table E-23 describes summary strings common to all Cisco Info Mediators and TSMs.
See the Cisco Info Mediator Reference, 3.6 for additional summary strings for each Cisco Info Mediator.
TSMWatch messages are in the same format as ProbeWatch messages. Table E-24 describes summary strings common to all TSMs.
