Understanding System Messages

About System Messages

During operation, a fault or event in the Cisco Application Centric Infrastructure (ACI) system can trigger the sending of a system log (syslog) message to the console and, optionally, to a logging server on another system. A system message typically contains a subset of information about the fault or event, and the message is sent by syslog, by an SNMP trap, or by a Cisco Call Home message.

Many system messages are specific to the action that a user is performing or the object that a user is configuring or administering. These messages can be the following:

  • Informational messages, providing assistance and tips about the action being performed

  • Warning messages, providing information aboutsystem errorsrelated to an object,such as a user account or service profile, that the user is configuring or administering

  • Finite state machine (FSM) status messages, providing information about the status of an FSM stage

A system message can contain one or more variables. The information that the system uses to replace these variables depends upon the context in which you see the message.

System messages are created by various sources, such as the Application Policy Infrastructure Controller (APIC) or the spine and leaf switches in the ACI fabric. System messages from the switches can be generated by either of the following processes:

  • The underlying NX-OS operating system of the spine and leaf switches

  • ACI-related processes in the switch

Additional details about the faults or events that generate these system messages are described in the Cisco APIC Faults, Events, and System Messages Management Guide and are listed in the Cisco APIC Management Information Model Reference, a web-based document.


Note

Not all system messages indicate problems with your system. Some messages are purely informational, while others may help diagnose problems with communications lines, internal hardware, or the system software.

About NX-OS System Messages

NX-OS System Message Structure

System messages generated by the underlying NX-OS operating system are structured as follows:

timestamp Nexus: FACILITY-SEVERITY-MNEMONIC: Message-text

The fields in the messsage are described in the following sections.

timestamp

The year, month, date, and time of day when the message was generated.

FACILITY

The facility code consists of two or more uppercase letters that indicate the facility to which the message refers. A facility can be a hardware device, a protocol, or a module of the system software.

SEVERITY

The severity level is a single-digit code from 0 to 7 that reflects the severity of the condition. The lower the number, the more serious the situation. The following table lists the message severity levels.

Table 1. NX-OS Message Severity Levels

Level

Severity Level (NX-OS)

ITU Level (ACI)

Description

0

emergency

System is unusable

1

alert

Critical

Immediate action required

2

critical

Major

Critical condition

3

error

Minor

Error condition

4

warning

Warning

Warning condition

5

notification

Cleared

Normal but significant condition

6

informational

Informational message only

7

debugging

Message that appears during debugging only

MNEMONIC

The MNEMONIC code uniquely identifies the error message.

Message-text

Message-text is a text string that describes the condition. The text string sometimes contains detailed information about the event, including terminal port numbers, network addresses, or addresses that correspond to locations in the system memory address space. Because variable fields change from message to message, they are represented here by short strings enclosed in square brackets ([ ]). A decimal number, for example, is represented as [dec]. The following table lists the variable fields in messages.

Table 2. Representation of Variable Fields in NX-OS Messages

Representation

Type of Information

[chars] or [char]

Character string

[dec]

Decimal

[hex]

Hexadecimal integer

[int]

Integer

[num]

Number

NX-OS System Message Example

The following example shows a typical NX-OS system message:


2014 Jan 25 21:42:07 Nexus: ETHPORT-5-IF_DOWN_ADMIN_DOWN:
Interface Ethernet3/1 is down (Administratively down))

In this system message:

  • Nexus indicates that this message is generated by NX-OS.

  • ETHPORT is the facility code.

  • The number 5 is the severity level, indicating a ‘notification,’ a normal but significant condition.

  • IF_DOWN_ADMIN_DOWN is the mnemonic code.

  • “Interface Ethernet3/1 is down (Administratively down))” is the message text.

About ACI System Messages

ACI System Message Structure

System messages generated by ACI components other than NX-OS are structured as follows:


timestamp host %LOG_LOCALn-severity-SYSTEM_MSG [code][lifecycle state][rule][severity text][DN of affected MO]
Message-text

The fields in the messsage are described in the following sections.

timestamp

The year, month, date, and time of day when the message was generated.

host

The hostname or IP address of the host that generated the message, such as ‘apic1’ or ‘spine3.’

%LOG_LOCALn

The local facility code ‘n’ is a single-digit code from 0 to 7 that reflects indicate the local facility of the message. This number can be configured and is used to sort received messages.

severity

The severity level is a single-digit code from 1 to 5 that reflects the severity of the condition. The lower the number, the more serious the situation. Unlike NX-OS system messages, ACI system messages follow the ITU Perceived Severity values described in RFC5674. The following table lists the message severity levels.

Table 3. ACI Message Severity Levels

Level

Severity Level (NX-OS)

ITU Level (ACI)

Description

0

emergency

System is unusable

1

alert

Critical

Immediate action required

2

critical

Major

Critical condition

3

error

Minor

Error condition

4

warning

Warning

Warning condition

5

notification

Cleared

Normal but significant condition

6

informational

(Info)

Informational message only

7

debugging

(Not used)

Message that appears during debugging only

code

The unique fault or event code.

lifecycle state

The current lifecycle state of the fault. Faults are stateful, and a fault transitions through more than one state during its life cycle. Events are stateless, and this field is omitted in event system messages.

rule

The action or condition that caused the event, such as a component failure or a threshold crossing.

severity text

The text translation of the numeric severity value. For example “major.”

DN of affected MO

The distinguished name (DN) of the managed object (MO) affected by the fault condition or event.

Message-text

Message-text is a text string that briefly describes the condition. The text string sometimes contains detailed information about the fault or event, including interface port numbers or network addresses.

ACI System Message Examples

Example: Fault system message

The following example shows an ACI system message generated by a fabric node failure:


July 22 22:45:28 apic1 %LOG_LOCAL0-2-SYSTEM_MSG [F0110][soaking][node-failed][critical][topology/pod-1/node-102/fault-F0110]
Node 102 not reachable. unknown

In this system message:

  • apic1 indicates that this message is generated by the controller.

  • The number 2 is the severity level, indicating a critical condition.

  • F0110 is the fault code, which we can look up as ‘fltFabricNodeFailed.’

  • soaking is the current lifecycle state of the fault.

  • node-failed is the cause of the fault.

  • topology/pod-1/node-102 is the DN of the affected object, which is node 102 in pod 1.

  • fault-F0110 is the DN of the fault object, which is a child of the affected object.


    Tip

    Using the Visore object browser, you can inspect the properties of the fault object for more details about the fault condition.

  • “Node 102 not reachable. unknown” is the message text.

Example: Event system message

The following example shows an ACI system message generated by a fabric node failure:


July 22 22:45:27 apic1 %LOG_LOCAL0-6-SYSTEM_MSG [E4208219][link-state-change][info]
[subj-[topology/pod-1/lnkcnt-1/lnk-101-1-1-to-1-1-3]/rec-4294968577]
Link State of Fabric Link is set to ok

In this system message:

  • apic1 indicates that this message is generated by the controller.

  • The number 6 is the severity level, indicating an informational message.

  • E4208219 is the fault code, which we can look up as ‘fltFabricNodeFailed’.

  • link-state-change is the cause of the event.

  • topology/pod-1/lnkcnt-1/lnk-101-1-1-to-1-1-3 is the DN of the affected object, which is a link.

  • ec-4294968577 is the event record object, which is a child of the affected object.


    Tip

    Using the Visore object browser, you can inspect the properties of the event record object for more details about the fault condition.

  • “Link State of Fabric Link is set to ok” is the message text.