Analyzing and Troubleshooting Your Network

Associated policies

Associated policies lists the various objects or policies available. When the policies are viewed in a top to down manner, the lists start with the node that has the maximum utilization followed by the next lower utilization. For Cisco ACI fabrics, each item in each column can be chosen to show relevant associations and relationships between the tenants, contracts, and EPGs. Similarly, for NX-OS fabrics, these columns show the associations and relationships between the Security group, Contracts, and Filters.

Click View All to view all the nodes or switches for the chosen object in a side panel.

These objects or policies are available for fabrics.

Click any of the objects to show all related objects and policies.

Policy CAM Statistics

The policy CAM statistics displays all the nodes and associated rules, and you can drill into details for a specific node here. Click the checkboxes for objects you want to see in the table.

These objects are available.

You can filter the table based on these attributes.

The table also shows the hit count in these timelines. * For Cisco ACI fabrics: 1 month 1 week 1 day Cumulative

The gear icon allows you to toggle columns to customize the table as per your view.

Policy CAM Rules

In the Policy CAM Rules table, you can view the listings for all of the nodes based on the chosen snapshot.

You can filter the table based on these attributes.

These details are available in the Rules table.

For NX-OS fabrics:

The gear icon allows you to toggle columns to customize the table as per your view.

All Anomalies

In the Anomalies table, you can view the anomalies that are generated in the chosen snapshot of time, individually by nodes or as an aggregate.

The Anomalies table is not supported on the NX-OS fabrics.

You can filter the anomalies based on the following attributes:

Types of compliance

There are two compliance types:

Examples of compliance

{
  "fvTenant":
  {
    "attributes":
    {
      "SELECT(name)": "EXACT(tenantABC)"
    },
    "children":
    [
      {
        "fvBD":
        {
          "attributes":
          {
            "MATCH(type)": "EXACT(regular)",
            "SELECT(name)": "REGEX(.*ABC.*)"
          }
        }
      },
      {
        "fvBD":
        {
          "attributes":
          {
            "MATCH(type)": "EXACT(fc)",
            "SELECT(name)": "REGEX(.*XYZ.*)"
          }
        }
      }
    ]
  }
}

Compliance rules

Compliance rules are created to generate anomalies where compliance can be violated or satisfied. Once you create compliance rules, you can generate the Compliance Report to check how much the fabrics and networks align to the rules.

Click Manage > Anomaly and Compliance Rules > Compliance Rules. This is where all the created rules are listed. The Compliance Rules page allows you to view all the rules created in one place.

You can perform the following actions on this page:

Interpretation of compliance rules

The following table lists some examples of compliance rules and what condition they create.

Compliance analysis

The Actions button allows you to re run the analysis.

The Summary displays the number of violations, the top rules by anomaly count, the anomalies from violations and the violations by rule type. You can click on any of the rules in 'Top rules by Violation' to view more details and click the count under 'Number of anomalies from violations' to view the list of anomalies.

The Anomalies from Violations lists all the anomalies that were triggered by the rules created. Click any rule in the 'Grouped' view to see the list of anomalies categorized under that group. If you click any rule in the 'Ungrouped' view, you will be redirected to the compliance rule detail page. This can be listed in a group view for all fabrics or individual view for a specific fabric. The table lists the severity level of the anomaly, the type of rule that triggered the anomaly, the detection time, and the status.

When you click any rule, it takes you to a slide-in that gives you a summary of your rule ( What’s wrong, What triggered this anomaly, What’s the impact?, How do I fix it? ).

Use search to filter by attributes like App Profile Name, BD Name, Category, Compliance Object Name, Compliance Object Type, Contract Name, EPG Name, Filter Name, L2 Out Name, L3 Out Name, Level, Rule Name, Subject Name, Tenant Name, and VRF Name. The gear icon is used to customize the columns in the table.

The Compliance Rules table shows a summary of the rules enforced and violated along with the number for each rule type. The table lists all the rules used to generate the current report. The table specifies whether it’s a configuration rule or a communication rule and the number of anomalies from violations for each rule.

Use search to filter by attributes like Name, Rule Type, Enforcement Status, and Verified. The Create Compliance Rule button takes you to the rule creation page.

Compliance anomalies

In the UI, you specify your compliance rules and Nexus Dashboard will verify in the subsequent snapshots, whether the compliance rules are satisfied by the policy that is configured on Cisco APIC.

The number of anomalies raised is defined by the number of rules associated with a snapshot. For example, if an assurance group runs a compliance analysis on a snapshot every 15 minutes, and there are two rules associated with the snapshot, two anomalies will be raised.

Guidelines and limitations for compliance

Create compliance communication rule

You can view/edit Direction based traffic settings from the Direction settings column.

Create compliance rule with snapshot selection

Import configuration compliance

Create compliance rule with manual configuration

For BDs in context to VRFs, an extra requirement is needed. The EPG association requirement is to be added which requires an EPG association count. This can be equal to/at least/at most. However you can choose to add either the EPG Association Requirement or the Name and Attribute Requirement for BD. You cannot have all the attributes selected. See Manual configuration compliance.

Create template-based compliance

For more information about template syntax, see Templates to configure object selectors. For information on how to configure object selectors for the template, see Templates to configure object selectors.

Trigger a compliance analysis

The Compliance Analysis will internally trigger assurance analysis and generate compliance anomalies.

Templates to configure object selectors

When you create a configuration rule using manual configuration, only a few specific object selectors are supported (such as BD, EPG, VRF). By using a template, you can select any object and apply match criteria on its attributes.

An object can be any managed object from Cisco APIC, and its selection is based on the distinguished name of the object. If you prefer to have a different attribute as the selection criteria, you can use any valid attribute of that object. You can configure object selectors for selection and match criteria and based on tags and annotations.

Selection, status, and match criteria

For naming compliance, the compliance rules are on the name and nameAlias fields that are indicated by MATCH.

• STATUS defines the state of the specific object in the template, whether an object exists or does not exist. The STATUS criteria can be defined using one of the following keywords.

  • MUST_EXIST

  • MUST_NOT_EXIST

    The following is a syntax example:

  {
  "vzBrCP":
   {
  "attributes":
    {
  "STATUS": "(<status selected>)",
  "SELECT(name)":"<KEY_WORD>(<value>)",
  "MATCH(nameAlias/name)":"<KEY_WORD>(<value>)"
    }
   }
  }

• The SELECT and MATCH criteria can be defined using one of the following keywords. The MATCH criteria is used to define the Compliance rule. SELECT allows to define a criteria to select group of objects and MATCH allows to define attributes and values that those selected objects must have. These compliance rules will be applied on objects that are selected using the SELECT criteria.

  • STARTS_WITH

  • ENDS_WITH

  • EXACT

  • OR

  • REGEX

    Syntax for SELECT:

    SELECT(<attribute_name>): KEY_WORD(<value>)

    Syntax for MATCH:

    MATCH(<attribute_name>): KEY_WORD(<value>)

    

    ---

    Attribute_name can be any attribute of the object. REGEX(<value>) - where the value must follow the standard regex expression syntax "SELECT(name)": "REGEX(Ctrct_[1-3])". For more details about keyword regular expressions, see Summary of Regular-Expressions Constructs.

    ---

    The following is a syntax example:

    {
       "<object>":
       {
          "attributes":
          {
             "SELECT(dn)":"<KEY_WORD>(<value>)",
             "MATCH(nameAlias/name)":"<KEY_WORD>(<value>)"
          }
       }
    }

  If SELECT is not specified for an attribute, then rn and dn will be considered as SELECT by default.

The following is a syntax example where if the KEY_WORD is not defined, the default behavior is EXACT. When you use MATCH(dn) and MATCH(rn), they are defined as match criteria.

---

If an attribute (other than dn and rn) does not have MATCH or SELECT specified, it will be considered as MATCH by default.

---

{
  "fvAEPg":
  {
    "attributes":
    {
      "SELECT(dn)": "uni/tn-aepg_vzanycons_imd_ctx_pass_7/ap-CTX1_AP1/epg-CTX1_BD1_AP1_EPG7",
      "MATCH(isAttrBasedEPg)": "EXACT(no)",
      "prio": "OR(unspecified, prio1)"
    }
  }
}

In the above example, by default, "prio" will be a MATCH.

Example template to configure a Naming Compliance to match selected objects to name or nameAlias:

{
   "vzSubj":
   {
      "attributes":
      {
         "SELECT(dn)":"EXACT(subj1)",
         "MATCH(nameAlias)":"STARTS_WITH(ABC)"
      }
   }
}

As the attribute dn is always considered as SELECT by default and any other attribute is always considered as MATCH, the above template can be simplified as displayed in the example below. Additionally, if the keyword is not defined, the default behavior is EXACT.

{
   "vzSubj":
   {
      "attributes":
      {
         "dn":"subj1""nameAlias":"STARTS_WITH(ABC)"
      }
   }
}

---

In the above template, you can use any object instead of "vzSubj", and you can use any attribute instead of "dn".

---

• Template Syntax for {}

  The following is a syntax example of a generic template where the KEY_WORD is {}. You can use this template to customize your requirements, select attributes, regular expresssions.

  The KEY_WORD values can be as follows:

  • STARTS_WITH

  • ENDS_WITH

  • EXACT

  • OR

  • REGEX

  {
    "<MO type>":
    {
      "attributes":
      {
        "SELECT(<attribute>)": "KEY_WORD(<expression>)",
        "MATCH(<attribute>)": " KEY_WORD (<value>)"
      },
      "children":
      [
        {
          "<MO type>":
          {
            "attributes":
            {
              "SELECT(<attribute>)": " KEY_WORD (<value>)",
              "MATCH(<attribute>)": " KEY_WORD (<value>)"
  
            },
            "children":
            [
              {
                "<MO type>":
                {
                  "attributes":
                  {
                    "SELECT((<attribute>)": " KEY_WORD (<value>)",
                    "MATCH(<attribute>)": " KEY_WORD (<value>,<value>)"
  
                  }
                }
              }
            ]
          }
        }
      ]
    }
  }

• Template With Attribute Value NULL or EMPTY

  The following are examples of templates where the attribute value is null or empty.

  "REGEX(^.{0}$)"
  "EXACT()"
  "OR(test, )" <— use space

  {
    "fvTenant":
    {
      "attributes":
      {
        "MATCH(annotation)": "OR(orchestrator:msc, )",
        "SELECT(name)": "REGEX(aepg_aepg_imd_tnt_pass_[0-9]+)",
      }
    }
  }

For the procedure to configure Object Selectors for Naming Compliance using the above template, see Create template-based compliance.

Tags and annotations

As an APIC user, you can create tags on managed objects (MOs) that result in creating child objects of type tagInst or tagAnnotation (based on which APIC version is in use).

Therefore, if you select objects based on a tag created in APIC, you can follow the templates provided in this section to configure object selectors on tags and annotations.

Example that displays the child object as type tagInst:

{
   "<object>":
   {
      "attributes":
      {
         "MATCH(<attribute_name>)":"<KEY_WORD(<value>)"
      },
      "children":
      [
         {
            "<tagInst>":
            {
               "attributes":
               {
                  "SELECT(<attribute_name>)":"<KEY_WORD(<value>)"
               }
            }
         }
      ]
   }
}

Example that displays the child object as type tagAnnotation:

{
   "<object>":
   {
      "attributes":
      {
         "MATCH(<attribute_name>)":"<KEY_WORD(<value>)"
      },
      "children":
      [
         {
            "<tagAnnotation>":
            {
               "attributes":
               {
                  "SELECT(<key or value>)":"<KEY_WORD(<value>)"
               }
            }
         }
      ]
   }
}

An object can be any valid APIC object with tagAnnotation or tagInst as a child. Object selection is defined in the tagInst or tagAnnotation object using SELECT on the name in the case of tagInst, and key or value in the case of tagAnnotation.

The selection criteria can be any of the following keywords:

• STARTS_WITH

• ENDS_WITH

• EXACT

• OR

• REGEX

Compliance rules are defined at the parent object level using MATCH and the criteria can be defined using any KEY_WORD. tagInst or tagAnnotation do not participate in compliance rules as they only provide the selection criteria.

Example template where you SELECT all the fVBDs where the tag is “BDs_in_cisco”, and those BDs must have name as BD or app1BD.

{
   "fvBD":
   {
      "attributes":
      {
         "MATCH(name)":"OR(BD, app1BD)"
      },
      "children":
      [
         {
            "tagInst":
            {
               "attributes":
               {
                  "SELECT(name)":"EXACT(BDs_in_cisco)"
               }
            }
         }
      ]
   }
}

For the procedure to configure object selectors based on Tags and Annotations using a template, see Create template-based compliance.

---

When using the steps to Create template-based compliance, to configure object selectors for tags and annotations, you must perform an additional step. Before you click Save, in Create New Rule, you must check the checkbox for the field Enable Object Selection Based on tagAnnotation/tagInst. Therefore, if any object has a tag annotation or tagInst, the parent based on the selection criteria in these two objects will be selected.

---

Communication compliance

Manual configuration compliance

Matching criteria

Define matching criteria objects

Matching Criteria Object Type 1	How to define
Tenant	tn - operator value Object type 2 (Could be either VRF or BD) If you select VRF, the rule is further defined as tn - operator value ctx - operator value If you select BD, the rule is further defined as tn - operator value bd - operator value
VRF	tn - operator value ctx - operator value
BD	tn - operator value bd - operator value
EPG	tn - operator value ap - operator value epg - operator value
App Profile	tn - operator value ap - operator value
L3 Out	tn - operator value out - operator value
L3 InstP	tn - operator value out - operator value instp - operator value
L2 Out	tn - operator value l2out - operator value
L2 InstP	tn - operator value l2out - operator value instp - operator value
Contract	tn - operator value brc - operator value
Subject	tn - operator value brc - operator value subj - operator value
Filter	tn - operator value flt - operator value

---

operator and value can be set to anything.

---

Access conformance report

Navigate to Analyze > Analysis Hub > Conformance.

Choose a fabric from the drop-down list.

OR

Navigate to Manage > Fabrics.

Choose a fabric.

In the General section, click Conformance.

Click View Report.

View conformance report

You can save conformance report as a PDF with the browser print option (Only supported on Chrome and Firefox).

Connectivity Analysis options

The checks performed by Connectivity Analysis include:

Layer 2 ToR support

For standalone NX-OS fabrics, with Layer 2 ToR support, Connectivity Analysis offers the following capabilities:

Supported topologies

Guidelines and limitations for all fabrics

Guidelines and limitations for Cisco ACI fabrics

Guidelines and limitations for standalone NX-OS fabrics

Create connectivity analysis

Follow these steps to create a connectivity analysis.

View connectivity analysis

Follow these steps to view connectivity analysis.

Manage connectivity analysis

Follow these steps to manage connectivity analysis.

You can also stop, re-run, run reverse, and rename an analysis by clicking the ellipses (…​) for each job from the Connectivity Analysis Jobs table in the Connectivity Analysis page.

Filtering information

In some cases, you might be able to filter results to find information more easily.

For example, you might have a situation where there a large number of endpoints under a single leaf switch, but you are only interested in endpoints that have a certain VLAN value.

You could filter the information to show only those specific endpoints in this situation.

Use the following operators for the filter refinement:

Traffic analytics conversations

A TCP conversation is a 4-tuple including source IP address, destination IP address, destination port, and protocol. A non-TCP conversation is a 3-tuple including source IP address, destination IP address, and protocol. In case a single client establishes multiple communication flows initiated by multiple source ports toward a service endpoint, all related statistics would be aggregated as a single entry in the Traffic analytics table. A service endpoint is defined by an IP address, a port, and a protocol.

An anomaly is raised after the conversation rate limit is exceeded. Navigate to Admin > System Settings > Flow Collection. In the Traffic analytics status for the last hour area, you can view if the conversation rate approaches or exceeds the limits. You can also view if there are any Traffic analytics record drops.

Traffic analytics support for north-south filters

By default, Nexus Dashboard enables traffic analytics on all L3Out interfaces to monitor traffic flows. You can create rules and apply them to external interfaces as an additional option to choose traffic that needs monitoring. Applying interface rules can filter traffic collection and prevent overloading. It also allows for visibility into specific subnets and enables targeted monitoring of key traffic. For more information on applying interface filter rules, see Apply filters to restrict traffic analytics for north-south filtering.

---

• Data Center Interconnect (DCI) rules are not required for monitoring traffic within a fabric. They are only needed for monitoring traffic between multiple ACI sites. When you enable DCI rules for the first time, the system automatically creates a default rule on the spine Inter-Pod Network and Inter-Site Network (IPN/ISN) interfaces with the subnet 0.0.0.0/00::0/0. YYou can update these subnets to monitor only the traffic that needs monitoring.

• Egress TCAM carving must be configured on NX-OS fabrics when using the interface filter rule feature. For detailed instructions, refer to the "Performing Egress NetFlow TCAM Carving" section in the Cisco Nexus 9000 Series NX-OS System Management Configuration Guide. This configuration is required the first time interface filters are created. If Egress TCAM carving is not performed, the interface filter rule configuration fails on the switch, and the issue is reported as a configuration failure anomaly.

• Traffic analytics filter rules do not apply on east-west traffic.

• Configuring interface filter rules will apply in the hardware/ASIC, this will also limit third-party NetFlow monitoring on that interface.

---

Guidelines for traffic analytics

These guidelines apply for traffic analytics:

Configure traffic analytics flow collection

If flow telemetry is already enabled on the fabric, you must first disable flow telemetry for all the fabrics and remove all flow rules before enabling traffic analytics.

View traffic analytics

Manage service categories

In the Manage Service Categories area, you can view the ports that have been assigned to categories based on standard networking defaults and any categories you may have created. If a port has not been assigned to a category, you can assign it to one of the existing categories or create a new category. This helps you to organize and manage your network ports more efficiently.

With this release, in Nexus Dashboard, you can view the UDP conversations in the traffic analytics table along with the TCP conversations. You can create a new category or modify existing category and choose UDP or TCP protocol to view and monitor the conversations.

Follow these steps to manage service categories.

View traffic analytics for endpoints

Follow these steps to view Traffic analytics for endpoints.

Use the flow troubleshoot workflow

The flow troubleshoot workflow enables you to collect all the flow records between two endpoints. Nexus Dashboard allows you to specify the duration for flow collection and then collect records between specific endpoints for the specified duration. As a result you can view the path visualization, 5-tuple flow information, and any issues seen on individual flows.

The flow troubleshoot workflow feature is not supported in Traffic analytics compatibility mode.

Follow these steps to use the flow troubleshoot workflow.

Cisco Energy Manager

The Cisco Energy Manager is a service developed by Cisco that collects data from various data providers and consolidates the GHG emissions and the source of the energy from the data. The Cisco Energy Manager is hosted in a Cisco Intersight cloud.

View the sustainability report for switches

View the sustainability report for PDUs

Health Delta

Health Delta analyses the difference in the health of the fabric across the two snapshots.

See Health Delta for more details.

Policy Delta for ACI

Policy Delta for ACI fabrics analyzes the differences in the policy between the two snapshots and provides a co-related view of what has changed in the ACI Fabric.

See Policy Delta for more details.

Policy Delta for standalone NX-OS

Policy Delta for standalone NX-OS fabrics analyzes the changed nodes or switches across two snapshots and obtains a co-related view of what has changed in the NX-OS switches.

See View policy delta analysis for more details.

Guidelines and limitations for delta analysis

Create delta analysis

For ACI Assurance Group users, APIC admin writePriv privileges allow information collection on the APIC host and leaf switches. You must have APIC admin writePriv privileges to configure the APIC Configuration Export Policy. This is not a requirement for NX-OS fabrics.

Choose Analyze > Analysis Hub > Delta Analysis > Create Delta Analysis.

You can perform one delta analysis at a time. To perform another delta analysis, you must stop the current delta analysis and then start the another delta analysis.

If there are any errors in the creation of a delta rule, it will be displayed on the summary page of the rule creation as a banner.

View delta analysis

The delta analysis page displays the analysis in a tabular form. The analysis are sorted by status. The Create Delta Analysis button lets you create a new delta analysis. Click any delta analysis to view more details.

The status of analysis can be either Aborted, Pending, Scheduled, Stopped, Stopping, Success, Failed, Partially Failed, Queued, Completed or In progress.

The filter bar allows you to filters the analysis by the following factors:

The delta analysis dashboard displays the general details of the analysis along with the health and policy delta.

View health delta analysis

Health Delta analyses the difference in the health of the fabric across the two snapshots. The results are displayed in the following areas:

The toggle for 'Include Acknowledged Anomalies' allows you to filter out the acknowledged anomalies from the results displayed if enabled. If it is disabled, manually acknowledged anomalies are included in the Anomaly Count.

The first count represents the anomalies found only in the earlier snapshot. The second count represents the anomalies common in both the snapshots. The third count represents the anomalies found only in the later snapshot.

Delta analysis now performs an object delta rather than a count delta. So along with the count, you can now view how many anomalies were cleared, how many are unchanged and how many are new anomalies.

The anomaly count also displays the difference for the different types of anomalies. It is displayed for Critical, Major and Warning.

If you click any of the counts for the Count and Health Delta resources, you can view the list of resources along with the node information. For SVI, the node column provides the VNI association.

For ACI fabrics, the Anomalies can be listed for the following kinds of snapshots:

For NX-OS fabrics, the Anomalies can be listed for the following types:

The anomalies are displayed in a tabular form with the following fields:

For ACI fabrics, you can filter the results based on the following attributes:

For NX-OS fabrics, you can filter the results based on the following attributes:

Choose an anomaly to view the anomaly details.

View policy delta analysis

Click Policy Delta to view the policy changes across the two snapshots. Policy Delta includes Changed Policy Object and Policy Viewer. ACI fabrics also include Audit Log.

A correlated view of what has change in the data center is displayed in the Audit Log panel. When you choose a particular object in the Changed Policy Objects panel, the relevant difference is highlighted in the Policy Viewer panel and the relevant audit log is highlighted in the Audit Log panel. APIC audit logs are records of user-initiated events such as logins and logouts or configuration changes that are required to be auditable. For every snapshot, the audit log history is limited to last 24 hrs.

Pre-change analysis options

The following list specifies the options you can choose on your pre-change analysis job. Only the objects listed are supported.

For Fabric Access Policies, you can choose to add the following to your pre-change analysis job:

Guidelines and limitations for pre-change

When using Pre-Change Analysis follow these guidelines and limitations:

Support for multiple objects in pre-change analysis

In addition to multiple tenants, you can also add multiple infrastructure objects as part of a Pre-Change Analysis JSON or XML job. The Pre-Change Analysis upload path allows you to add, modify, and delete multiple objects across the policy universe. There are no additional configurations required to use this feature. Your Pre-Change Analysis job for multiple objects will run, based upon the files you upload.

The following file upload formats are accepted:

Known issues for pre-change analysis

Create pre-change analysis job

General

Change

Depending upon your selection, the relevant fields are displayed for you to populate.

Complete the selections as appropriate, and click Save or Save & Run.

After a Pre-Change Analysis job is completed, the Pre-Change Analysis table displays the status for the job as completed.

Click the Pre-Change Analysis Name for which you want to view the details. In a sidebar to the right, the details are displayed in a column including the general information such as the name of the job, snapshot, and change definition type. The list of changes modeled for the job are also available. If you are viewing a completed job, the anomalies that were generated as a result of the changes are displayed at the top of this page.

For completed jobs, click the icon on the top right of the sidebar to navigate to the results page. Further details about the job are available here under the specific tabs for Dashboard, Delta Analysis, Compliance Analysis.

Download pre-change analysis job

You can download an existing Pre-Change Analysis as follows:

In the downloaded file, you can view all the attributes which include attributes that are modified and those that are not modified. If desired, the downloaded file can be uploaded to your Cisco APIC.

Device connectivity notifier for TAC-initiated collector

Nexus Dashboard uses the device connectivity issue notifier on Nexus Dashboard to communicate with the devices. The notifier checks for TAC triggered on-demand collection of logs. In case the fabric is not configured properly to communicate with the device, Nexus Dashboard notifies the following:

If the node interaction is not healthy on the device, you cannot choose the device for Log Collector to collect logs. In the GUI, the device is greyed out.

Guidelines and Limitations

Log collector dashboard

Navigate to Analyze > Analysis Hub > Log Collector.

The Log Collector dashboard displays a graph of Logs by Job status for a particular fabric and displays the latest log collections.

The filter bar allows you to filters the logs by Status, Name, Node, start time, and end time. For standalone NX-OS fabrics, you can also filter by Type.

Use the following operators for the filter refinement:

The page also displays the log collection jobs in a tabular format. The jobs are sorted by status. Choose the log collection job in the table to view additional details.

General

This displays the status of the job along with a graph showing the number of devices by status.

Details

The following information is listed:

Selected Nodes

This displays the list of nodes in a tabular form along with the status of each job and the upload status for the files uploaded.

Upload All Files allows you to upload all the files.

TAC-initiated log collector

The TAC initiated log collector enables Cisco TAC to trigger on-demand collection of logs for specified user devices in the Cisco Intersight Cloud to the Device Connector.

When the TAC assist job is complete, the new job appears in the Log Collector table. Choose the log collection job in the table to display additional details. The Log Collection status displays information such as status, general information, and node details.

You can save TAC assist job details as a PDF with the browser print option (Only supported on Chrome and Firefox).

Upload logs to Cisco Intersight Cloud

Choose Analyze > Analysis Hub > Log Collector > Create Log Collector Job.

Bug scan schedule

Bug Scan runs for all the fabrics onboarded to Nexus Dashboard with a timer auto-scheduled every 7 or 14 days for each device, depending on the number of nodes and fabrics on the Nexus Dashboard cluster. This schedule is fixed and is not customizable.

Bug Scan for all switches of a fabric are included in the auto-schedule once they are onboarded in Nexus Dashboard. After that initial bug scan run occurs, it is then run again every 7 or 14 days. In situations where the bug scan either executes correctly or fails to execute on the device (for example, if the CPU and memory of the switch is higher than the target threshold), bug scan will run again in the next auto-scheduled timer.

On-demand Bug Scans are a user-triggered collection of technical support information from selected switches of a fabric. Those on-demand scans are prioritized over auto-scheduled runs and do not consider the CPU and memory metrics. On demand bug scans can be executed for up to 10 switches concurrently. If auto-scheduled Bug Scan is in progress and on-demand Bug Scan is initiated on other switches, based on the available resources in the Nexus Dashboard nodes the on-demand Bug Scan will start while the current Bug Scan is in progress or after the current Bug Scan is completed.

Only one Bug Scan at the time can run on a specific device.

A Bug Scan is triggered automatically in the following scenarios:

View active and known bugs

The Bug Scan feature collects technical support logs from devices in a fabric and scans them for bugs. You can view the active and known bugs affecting your network after the Bug Scan is completed.

• Active Bugs - Bugs present in the software version that are detected in your network based on its configuration and tech support files based on bug signatures continuously updated through Cisco Nexus Dashboard Metadata files. A signature can identify a bug on a fabric or switch with a confidence level. The confidence level indicates the percentage of confidence the signature has in detecting that bug accurately. Active bugs are detected in Nexus Dashboard Insights only when the confidence level of a signature is 100% for ACI fabrics and over 75% for NX-OS fabrics. Note that not all bugs have a digital signature and some active bugs might not be identified as such.

• Known Bugs - Bugs present in the software version that may potentially impact your network. These bugs do not require a bug signature but are based only on the software version match and not on switch configurations or tech-support.

---

Only Severity 1 to Severity 3 bugs attached to the Release Notes of a certain ACI or NX-OS release are considered known bugs.

---

Backup and Restore

EPL only backups data for fabrics that EPL has been configured. If EPL is disabled for a fabric (even if EPL has previously been configured there), then you cannot backup the data for that fabric. Also, you can backup only historical data (data on the Endpoint Search page).

If a backup is initiated when EPL is enabled, then when restoring the backup, the same external data IPs that EPL was using must be available on ND. If those IPs are not available, then select the Ignore External Service IP Configuration option in the restore backup form. However, there are chances that the EPL pods will be brought up with different IPs, so any existing EPL policies become invalid. If EPL was previously configured with the Configure My Fabric option, you need to disable and enable EPL so that the old policy is cleaned up and an updated policy is deployed. If you did not use the Configure My Fabric option, then manually update their config with the new IPs.

EPL Connectivity Options

Sample topologies for the various EPL connectivity options are as given below.

Configuring Endpoint Locator

The Nexus Dashboard OVA or the ISO installation comes with two interfaces:

(Out-of-band or OOO) connectivity of switches via switch mgmt0 interface can be through data or Management interface. For more information refer to the Nexus Dashboard Installation and Upgrade Guide.

The Management interface provides reachability to the devices via the mgmt0 interface either Layer-2 or Layer-3 adjacent. This allows Nexus Dashboard to manage and monitor these devices including POAP. EPL requires BGP peering between the Nexus Dashboard and the Route-Reflector. Since the BGP process on Nexus devices typically runs on the default VRF, in-band IP connectivity from the Nexus Dashboard to the fabric is required. The data network interface can be configured during Nexus Dashboard installation. You can’t modify the configured in-band network configurations.

The setup of Data network interface on the Nexus Dashboard is a prerequisite of any application that requires the in-band connectivity to the devices within fabric. This includes EPL and Network Insights Resources (NIR).

On the fabric side, for a standalone Nexus Dashboard deployment, if the Nexus Dashboard data network port is directly connected to one of the front-end interfaces on a leaf, then that interface can be configured using the epl_routed_intf template.

However, for redundancy purposes, it is always advisable to have the server on which the Nexus Dashboard is installed to be dual-homed or dual-attached. With the OVA Nexus Dashboard deployment, the server can be connected to the switches via a port-channel. This provides link-level redundancy. To also have node-level redundancy on the network side, the server may be attached to a vPC pair of Leaf switches. In this scenario, the switches must be configured such that the HSRP VIP serves as the default gateway of the Data Network interface on the Nexus Dashboard.

For the HSRP configuration on terry-leaf3, the switch_freeform policy may be employed.

You can deploy a similar configuration on terry-leaf3 while using IP address 10.3.7.2/24 for SVI 596. This establishes an in-band connectivity from the Nexus Dashboard to the fabrics over the Data Network interface with the default gateway set to 10.3.7.3.

After you establish the in-band connectivity between the physical or virtual Nexus Dashboard and the fabric, you can establish BGP peering.

During the EPL configuration, the route reflectors (RRs) are configured to accept Nexus Dashboard as a BGP peer. During the same configuration, the Nexus Dashboard is also configured by adding routes to the BGP loopback IP on the spines/RRs via the Data Network Interface gateway.

Cisco Nexus Dashboard queries the BGP RR to glean information for establishment of the peering, such as ASN, RR, and IP.

To configure Endpoint Locator from the Cisco Nexus Dashboard Web UI, in the fabric Overview page, choose Actions > Configuration > Configure Endpoint Locator. Similarly, you can configure EPL on the Topology page. Right-click on the required fabric, then click More > Configure Endpoint Locator. The Endpoint Locator window appears.

You can enable EPL for one fabric at a time.

Select the switches on the fabric hosting the RRs from the drop-down list. Cisco Nexus Dashboard will peer with the RRs.

By default, the Configure My Fabric option is selected. This option only configures EPL as a BGP neighbor of the switch and this option does not configure network reachability between EPL and the switch. This knob controls whether BGP configuration will be pushed to the selected spines/RRs as part of the enablement of the EPL feature. If the spine/RR needs to be configured manually with a custom policy for the EPL BGP neighborship, then this option should be unchecked. For external fabrics that are only monitored and not configured by Nexus Dashboard, this option is greyed out as these fabrics are not configured by Nexus Dashboard.

Select the Process MAC-Only Advertisements option to enable processing of MAC-Only advertisements while configuring the EPL feature.

If EPL is enabled on a fabric with or without selecting the Process Mac-Only Advertisements checkbox and you want to toggle this selection later, then you have to first disable EPL and then click Database Clean-up to delete endpoint data before re-enabling EPL with the desired Process Mac-Only Advertisements setting.

Select Yes under Collect Additional Information to enable collection of additional information such as PORT, VLAN, VRF etc. while enabling the EPL feature. To gather additional information, NX-API must be supported and enabled on the switches, ToRs, and leafs. If the No option is selected, this information will not be collected and reported by EPL.

For all fabrics except external fabrics, NX-API is enabled by default. For external fabrics, you have to enable NX-API in the external fabric settings by selecting the Enable NX-API checkbox in the Advanced tab of the External_Fabric_11_1 fabric template.

Click the i icon to view a template of the configuration that is pushed to the switches while enabling EPL. This configuration can be copied and pasted on spines or border gateway devices to enable EPL on external monitored fabrics.

Once the appropriate selections are made and various inputs have been reviewed, click Submit to enable EPL. If there are any errors while you enable EPL, the enable process aborts and the appropriate error message is displayed. Otherwise, EPL is successfully enabled.

The Nexus Dashboard Data Service IP is used as BGP neighbor.

When the Endpoint Locator feature is enabled, there are a number of steps that occur in the background. Nexus Dashboard contacts the selected RRs and determines the ASN. It also determines the interface IP that is bound to the BGP process. Also, appropriate BGP neighbor statements are added on the RRs or spines in case of eBGP underlay, to get them ready to accept the BGP connection that will be initiated from the Nexus Dashboard. The external Nexus Dashboard Data Service IP address that is assigned to the EPL pod will be added as the BGP neighbor. Once EPL is successfully enabled, the user is automatically redirected to the EPL dashboard that depicts operational and exploratory insights into the endpoints that are present in the fabric.

For more information about the EPL dashboard, see Monitoring Endpoint Locator.

Flushing the Endpoint Database

After you enable the Endpoint Locator feature, you can clean up or flush all the Endpoint information. This allows starting from a clean-slate with respect to ensuring no stale information about any endpoint is present in the database. After the database is clean, the BGP client re-populates all the endpoint information learnt from the BGP RR. You can flush the endpoint database even if you have not re-enabled the EPL feature on a fabric on which the EPL feature was previously disabled.

To flush all the Endpoint Locator information from the Cisco Nexus Dashboard Web UI, perform the following steps:

Configuring Endpoint Locator for Single VXLAN EVPN Site

Before you begin:

In the following figure, the Nexus Dashboard service application is attached to the VPC pair of Leaf switches as it provides the link and node-level redundancy. The BGP instance running on EPL container establishes iBGP peering with the fabric spines. The iBGP peering is between Spine loopback addresses (loopback0) and EPL container persistent IP addresses. The loopback0 address of Spines is reachable via VXLAN Underlay, therefore, EPL container IP must have IP reachability towards the spines. We can configure an SVI on Leaf switches that can provide IP connectivity. The SVI will be a non-VXLAN enabled VLAN and will only participate in the underlay.

To configure endpoint locator for single VXLAN EVPN site, perform the following steps:

1. You must configure persistent IP addresses on Cisco Nexus Dashboard. On Nexus Dashboard, choose Admin > System Settings.

2. In the External Pools card, click Edit.

   The External Service Pools window appears.

3. In the Persistent data IPs area, click + Add IP Address and enter the necessary persistent IP addresses. Click the checkmark for each IP address that you enter.

   

   ---

   The IP address must be associated with Nexus Dashboard Data Pool. A single persistent IP address is required to visualize and track EPs for a single site.

   ---

4. Configure SVI using FHRP for ND Data Interface and Underlay IP connectivity.

   You can use switch_freeform policy on fabric Leaf 1.

   To create a freeform policy, perform the following steps:

   1. Choose Manage > Fabrics, then click on the required fabric.

      The fabric Overview page for this fabric appears.

   2. Click Configuration Policies > Policies.

   3. Click Actions > Add policy.

      The Create Policy window appears.

   4. Choose the appropriate leaf switch from the Switch List drop-down list and click Choose Template.

   5. On Select Policy Template window, choose switch_freeform template and click Select.

      Apply FHRP configurations and save the template.

      Deploy the template configuration.

      In this example, SVI 100 with HSRP gateway created on fabric Leaf 1. Similarly, repeat the steps for fabric Leaf 2.

      The following is a configuration example:

      feature hsrp
      vlan 100
      name EPL-Inband
      interface Vlan100
        no shutdown
        no ip redirects
        ip address 192.168.100.252/24
        no ipv6 redirects
        ip router ospf 100 area 0.0.0.0
        hsrp 100
          ip 192.168.100.254

5. Verify IP reachability between Nexus Dashboard Data Interface and fabric switches.

6. Enable EPL at fabric level.

   1. In the fabric Overview page, choose Actions > Configuration > Configure Endpoint Locator.

   2. Choose the appropriate switches on the fabric hosting the Spine/Route Reflector RRs from the drop-down list.

      Choose Configure my Fabric option for knob controls.

      Whether BGP configuration will be pushed to the selected Spines/RRs as part of the enablement of the EPL feature. If the Spine/RR needs to be configured manually with a custom policy for the EPL BGP neighborship, then this option should be unchecked. For external fabrics that are only monitored and not configured on Nexus Dashboard this option is grayed out. As these fabrics are not configured on Nexus Dashboard.

      Choose Process MAC-Only Advertisements option to enable processing of MAC-Only advertisements while configuring the EPL feature.

      

      ---

      If EPL is enabled on a fabric with or without choosing the Process Mac-Only Advertisements checkbox and if you want to toggle this selection later, then you must disable EPL and click Database Clean-up to delete endpoint data before re-enabling EPL with the desired Process Mac-Only Advertisements setting.

      Choose Yes in Collect Additional Information to enable collection of additional information such as PORT, VLAN, and VRF while enabling the EPL feature. To access additional information, NX-API must be supported and enabled on the switches, ToRs, and leafs. If you choose No option, this information won’t be collected and reported by EPL.

      ---

      

      ---

      For all fabrics except external fabrics, NX-API is enabled by default. For external fabrics, you must enable NX-API in the external fabric settings, choose Enable NX-API checkbox in the Advanced tab of the External_Fabric_11_1 fabric template.

      ---

      Click the Preview icon to view a template of the configuration that is pushed to the switches enabling EPL. This configuration can be copied and pasted on spines or border gateway devices to enable EPL on external monitored fabrics.

      Once the appropriate selections are made and various inputs have been reviewed, click Save Config to enable EPL. If there are any errors while you enable EPL, the enable process aborts and the appropriate error message are displayed. Otherwise, EPL is successfully enabled. Once the EPL is enabled, the Persistent IP will be in-use.

Configuring Endpoint Locator for interconnected VXLAN fabrics

Before you begin:

The below figure enables EPL for interconnected VXLAN fabrics. The BGP peering’s are established between the Spines/RRs of each interconnected VXLAN fabric and Nexus Dashboard EPL Container. The Persistent IPs are required based on the number of VXLAN EVPN sites. The Nexus Dashboard application hosted on Cisco ND Cluster is located on Site 1. The routing information to reach the Spines/RRs deployed in the remote site must be exchanged across the interconnected VXLAN fabrics. Once the BGP session is formed, local EPs of Fabric 2 can be visualized and tracked.

By default, Nexus Dashboard data Interface and Site 2 Spines/RRs loopback prefixes are not advertised across the BGWs. Therefore, prefixes must be exchanged using custom route maps and prefix lists across the sites. At the same time, route redistribution between OSPF and BGP is required as Spines/RRs loopback prefixes are part of OSPF protocol while BGWs peer with each other using BGP.

To configure endpoint locator for interconnected VXLAN fabrics, perform the following steps:

1. You must configure persistent IP addresses on Cisco Nexus Dashboard. On Nexus Dashboard, choose Admin > System Settings.

2. In the External Pools card, click Edit.

   The External Service Pools window appears.

3. In the Persistent data IPs area, click + Add IP Address and enter the necessary persistent IP addresses. Click the checkmark for each IP address that you enter.

   

   ---

   Ensure that the IP addresses are associated with Nexus Dashboard Data Pool. Two persistent IP addresses are required to visualize and track EPs for a multisite with two member fabrics. One Persistent Data IP address is used as EPL container IP to establish BGP session with Site 1 fabric. A new Persistent IP address is configured that can be used to peer with Site 2 fabric.

   ---

4. Configure Route Redistribution for VXLAN EVPN Fabrics.

   Route Redistribution for Fabric 1

   The following switch_freeform policy can be used on Fabric 1 BGWs. To create a new switch_freeform policy, refer to the above examples.

   The example below shows a sample configuration:

   ip prefix-list site-2-rr seq 5 permit 20.2.0.1/32 >> Site 2 RR
   ip prefix-list site-2-rr seq 6 permit 20.2.0.2/32 >> Site 2 RR
   ip prefix-list epl-subnet seq 5 permit 192.168.100.0/24 >> EPL Subnet
   
   route-map bgp-to-ospf permit 10
      match ip address prefix-list site-2-rr
   route-map ospf-to-bgp permit 10
      match ip address prefix-list epl-subnet
   
   router ospf 100
      redistribute bgp 100 route-map bgp-to-ospf
   
   router bgp 100
      address-family ipv4 unicast
          redistribute ospf 100 route-map ospf-to-bgp

   Route Redistribution for Fabric 2

   The following switch_freeform policy can be used on Fabric 2 BGWs. To create a new switch_freeform policy, refer to the above examples.

   The example below shows a sample configuration:

   ip prefix-list site-2-rr seq 5 permit 20.2.0.1/32 >> Site 2 RR
   ip prefix-list site-2-rr seq 6 permit 20.2.0.2/32 >> Site 2 RR
   ip prefix-list epl-subnet seq 5 permit 192.168.100.0/24 >> EPL Subnet
   
   route-map bgp-to-ospf permit 10
     match ip address prefix-list epl-subnet
   route-map ospf-to-bgp permit 10
     match ip address prefix-list site-2-rr
   
   router ospf 200
     redistribute bgp 200 route-map bgp-to-ospf
   
   router bgp 200
       address-family ipv4 unicast
           redistribute ospf 200 route-map ospf-to-bgp

5. To configure EPL, in the fabric Overview page, choose Actions > Configuration > Configure Endpoint Locator.

6. Choose the appropriate switches on the fabric hosting the Spine/Route Reflector RRs from the drop-down list.

   Once the appropriate selections are made and various inputs have been reviewed, click Save Config to enable EPL. If there are any errors while you enable EPL, the enable process aborts and the appropriate error message is displayed. Otherwise, EPL is successfully enabled. Once the EPL is enabled, the Persistent IP will be in-use.

   You can view EPL enabled for fabric-1 and fabric-2 successfully. To view and track EPs, see Monitoring Endpoint Locator.

Configuring Endpoint Locator for External Fabrics

In addition to Easy fabrics, Nexus Dashboard allows you to enable EPL for VXLAN EVPN fabrics comprising of switches that are imported into the external fabric. The external fabric can be in managed mode or monitored mode, based on the selection of Fabric Monitor Mode flag in the External Fabric Settings. For external fabrics that are only monitored and not configured by Nexus Dashboard, this flag is disabled. Therefore, you must configure BGP sessions on the Spine(s) via OOB or using the CLI. To check the sample template, click ℹ icon to view the configurations required while enabling EPL.

In case the Fabric Monitor Mode checkbox in the External Fabric settings is unchecked, then EPL can still configure the spines/RRs with the default Configure my fabric option. However, disabling EPL would wipe out the router bgp config block on the spines/RRs. To prevent this, the BGP policies must be manually created and pushed onto the selected spines/RRs.

Configuring Endpoint Locator for eBGP EVPN Fabrics

You can enable EPL for VXLAN EVPN fabrics, where eBGP is employed as the underlay routing protocol. Note that with an eBGP EVPN fabric deployment, there is no traditional RR similar to iBGP. The reachability of the in-band subnet must be advertised to the spines that behave as Route Servers.

To configure EPL for eBGP EVPN fabrics from the Cisco Nexus Dashboard Web UI, perform the following steps:

Monitoring Endpoint Locator

Information about the Endpoint Locator is displayed on a single landing page or dashboard. The dashboard displays an almost real-time view of data (refreshed every 30 seconds) pertaining to all the active endpoints on a single pane. The data that is displayed on this dashboard depends on the scope selected by you from the SCOPE drop-down list. The Nexus Dashboard scope hierarchy starts with the fabrics. Fabrics can be grouped into a VXLAN fabric group. A group of VXLAN fabric groups constitute a Data Center. The data that is displayed on the Endpoint Locator dashboard is aggregated based on the selected scope. From this dashboard, you can access Endpoint History, Endpoint Search, and Endpoint Life.

Disabling Endpoint Locator

To disable endpoint locator from the Cisco Nexus Dashboard Web UI, perform the following steps:

First Published: 2025-01-31
Last Modified: 2025-01-31