Cisco Catalyst SD-WAN Troubleshooting Guide

PDF

Cisco Catalyst SD-WAN Troubleshooting Guide

On-Demand Troubleshooting

Want to summarize with AI?

Log in

Lists the development milestones and release information for On-demand troubleshooting, including feature description.


Table 1. Feature History

Feature Name

Release Information

Description

On-Demand Troubleshooting

Cisco IOS XE Catalyst SD-WAN Release 17.6.1a

Cisco SD-WAN Release 20.6.1

Cisco vManage Release 20.6.1

This feature lets you view detailed information about the flow of traffic from a device. You can use this information to assist with troubleshooting.

Enhancement to On-Demand Troubleshooting

Cisco vManage Release 20.11.1

You can view the detailed troubleshooting progress of the flow of traffic from a device.

Information About On-Demand Troubleshooting

On-demand troubleshooting lets you view detailed information about the flow of traffic from a device.

By default, Cisco SD-WAN Manager captures aggregated information about flows. You can obtain detailed information for specific devices and for specific historical time periods by adding an on-demand troubleshooting entry. When you add an entry, Cisco SD-WAN Manager compiles detailed information according to parameters that you configure.

To conserve system resources, Cisco SD-WAN Manager compiles detailed information only when you request it by adding an entry. In addition, Cisco SD-WAN Manager stores the information for a limited time (3 hours by default), then removes it. You can request the same information again, if needed.

Note

On a Cisco SD-WAN Manager cluster setup, only a connected node can remove an on-demand troubleshooting task or mark it as complete.

Restrictions for On-Demand Troubleshooting

Ensure that no Cisco or third-party APIs that instruct on-demand troubleshooting to stop are called when you are using on-demand troubleshooting. These APIs prevent on-demand troubleshooting from compiling information.

Page Elements

The On Demand Troubleshooting window provides options for configuring and adding an on-demand troubleshooting entry. The On Demand Troubleshooting window displays information about existing on-demand troubleshooting entries and provides the following information and options.

Item (Field)

Description

ID

System-assigned identifier of the entry.

Device ID

System IP of the device to which the entry applies.

Data Type

Type of data for which the entry provides detailed information.

Creation Time

Date and time that you added the entry.

Expiration Time

Date and time that the entry expires.

At this expiration time, the entry is removed from the table automatically, and the corresponding detailed information is no longer available.

By default, an entry is removed 3 hours after its creation time.

Data Backfill Start Time

Start date and time of the data backfill period.

Data Backfill End Time

End date and time of the data backfill period.

Status

Status of the entry:

  • IN_PROGRESS: Detailed troubleshooting information is in the process of being compiled.

  • QUEUED: Detailed troubleshooting information is queued for compilation.

  • COMPLETED: Detailed troubleshooting information has been compiled.

Configure On-Demand Troubleshooting

You can configure on-demand troubleshooting for a device from the Tools > On Demand Troubleshooting window in Cisco SD-WAN Manager. This window provides options for adding an on-demand troubleshooting entry, and for managing existing entries.

Cisco vManage Release 20.6.1 and earlier: You can configure on-demand troubleshooting for a device from the Monitor > On Demand Troubleshooting window in Cisco SD-WAN Manager.

You can also start on-demand troubleshooting from various locations in the Monitor > Devices window for a device. See View On-Demand Troubleshooting Information for a Device.

Cisco vManage Release 20.6.1 and earlier: You can start on-demand troubleshooting from various locations in the Monitor > Network window for a device.

On-demand troubleshooting is qualified for troubleshooting entries for up to 10 devices concurrently.

Add an On-Demand Troubleshooting Entry

Adding an entry in the On Demand Troubleshooting window instructs Cisco SD-WAN Manager to compile detailed troubleshooting information for the device that you specify, using the parameters that you configure.

To add an on-demand troubleshooting entry, follow these steps:

  1. From the Cisco SD-WAN Manager menu, choose Tools > On Demand Troubleshooting.

    Cisco vManage Release 20.6.1 and earlier: From the Cisco SD-WAN Manager menu, choose Monitor > On Demand Troubleshooting.

  2. From the Select Device drop-down list, choose the Cisco IOS XE Catalyst SD-WAN device or the Cisco vEdge device for which you want to enable on-demand troubleshooting.

  3. From the Select Data Type drop-down list, choose SAIE or ConnectionEvents.

  4. Choose an option for the data backfill period:

    • Last 1 hour: Provides detailed stream information for the period beginning 1 hour before you add the troubleshooting entry and ending at the time that you add the entry.

    • Last 3 hours: Provides detailed stream information for the period beginning 3 hours before you add the troubleshooting entry and ending at the time that you add the entry.

    • Custom Date and Time Range: Use the Start date and time and the End date and time fields to designate the backfill period that you want. Note that the End date and time value cannot be later than the current date and time.

  5. Click Add.

    The troubleshooting entry appears in the table of entries. When the value in the Status field for the entry shows the value Completed, you can view the troubleshooting information from the Monitor > Devices window, as described in View On-Demand Troubleshooting Information for a Device.

Update an On-Demand Troubleshooting Entry

Update an on-demand troubleshooting entry to make changes to its configuration settings. For example, update an entry to adjust its backfill period.

Only entries that are in the QUEUED state can be updated.

To update an on-demand troubleshooting entry, follow these steps:

  1. From the Cisco SD-WAN Manager menu, choose Tools > On Demand Troubleshooting.

    Cisco vManage Release 20.6.1 and earlier: From the Cisco SD-WAN Manager menu, choose Monitor > On Demand Troubleshooting.

  2. In the table of entries, click … adjacent to the entry that you want to update and choose Update.

  3. In the Update Troubleshoot Status dialog box that is displayed, configure the settings as needed, and click Add.

Delete an On-Demand Troubleshooting Entry

Deleting an on-demand troubleshooting entry removes the entry from Cisco SD-WAN Manager. After you delete an entry, you can no longer view its detailed information.

Deleting an entry can help free resources in Cisco SD-WAN Manager.

To delete an on-demand troubleshooting entry, follow these steps:

  1. From the Cisco SD-WAN Manager menu, choose Tools > On Demand Troubleshooting.

    Cisco vManage Release 20.6.1 and earlier: From the Cisco SD-WAN Manager menu, choose Monitor > On Demand Troubleshooting.

  2. In the table of entries, click … adjacent to the entry that you want to delete and choose Delete on demand queue.

  3. In the Delete On Demand Status window that is displayed, click OK.

View On-Demand Troubleshooting Information for a Device

You can view on-demand troubleshooting information for a device from the Network window for that device.

Before you can view this information, at least one on-demand troubleshooting entry must exist for the device. Add an entry from the On Demand Troubleshooting window as described in Add an On Demand Troubleshooting Entry, or add an entry from the Network window as described in the following procedure.

  1. From the Cisco SD-WAN Manager menu, choose Monitor > Devices.

    Cisco vManage Release 20.6.1 and earlier: From the Cisco SD-WAN Manager menu, choose Monitor > Network.

  2. In the Hostname column, click the device for which you want to view the information.

  3. Perform either of these actions:

    • To view the troubleshooting information for an SAIE application:

      1. Click SAIE Applications.

        Note

        In Cisco vManage Release 20.7.1 and earlier releases, SAIE Applications is called DPI Applications.

      2. In the Applications Family table, click an application family.

      3. In the Applications table, click an application.

    • To view troubleshooting information for a specific metric, in the left pane, under ON-DEMAND TROUBLESHOOTING click an option. Not all options apply to all device types.

      • FEC Recovery Rate

      • SSL Proxy

      • AppQoe TCP Optimization

      • AppQoE DRE Optimization

      • Connection Events

        From Cisco Catalyst SD-WAN Manager Release 20.16.1, if you enable unified logging for your device, you can view the firewall security connection events for inspect, pass, and drop actions. The connection event information also includes the reason for traffic drop by the firewall policy.

      • WAN Throughput

      • Flows

      • Top Talkers

    The Flows and Top Talkers metrics are only for TCP Optimized flows.

    If on-demand troubleshooting is configured for the device, detailed troubleshooting information appears. This information includes traffic statistics and metrics such as source IP address, destination IP address, number of packets, number of bytes, and more. Use the options that are available and hover your cursor over elements on the graphs to view the information that you need.

    Note

    Starting from Cisco IOS XE Release 17.9.1a, use the policy ip visibility features enable command to manually enable or disable the feature fields in Flexible Netflow (FNF). Use the show sdwan policy cflowd-upgrade-status command to check which features were enabled before the version upgrade. You have to manually control the features after a version upgrade using the disable or enable commands.

    For more information, see policy ip visibility command page.

    If on-demand troubleshooting information is not configured, the Enable On Demand Troubleshooting option is displayed. Continue to Step 4.

  4. If the Enable On Demand Troubleshooting option is displayed, perform these actions to start this feature for the selected device:

    1. Click Enable On Demand Troubleshooting.

    2. Choose one of the following options:

      • Quick Enable: Starts an on-demand troubleshooting entry with a backfill period of 3 hours. With this option, detailed stream information for the past 3 hours becomes available.

        After you choose this option, click Refresh to view the detailed troubleshooting information. It can take a few minutes for this information to become available. Alternatively, click Go to On Demand Troubleshooting to display the On Demand Troubleshooting window that includes the entry that you just added.

      • Go to On Demand Troubleshooting: Displays the On Demand Troubleshooting window. Add an entry in this window as described in Add an On Demand Troubleshooting Entry. Repeat Steps 1 to Step 3 in this procedure to view the detailed information.

View Progress of On-Demand Troubleshooting

Minimum supported release: Cisco vManage Release 20.11.1

After you enable on-demand troubleshooting, the On-demand Troubleshooting in Progress message appears on the Monitor > Devices page. The message remains until the troubleshooting is complete.

Click a chart option to view the troubleshooting progress in a graphical format. Select a time period to display data or click Custom to display a selection of a custom time period.

You can use the request nms olap-db command to start, stop, or restart the Cisco SD-WAN Manager online analytical processing (OLAP) database or view the status of the database.

For more information about this command, see request nms olap-db.

View Detailed Top Source Data

After on-demand troubleshooting is configured, you can view detailed information about top application usage for a device. To do so, follow these steps:

  1. From the Cisco SD-WAN Manager menu, choose Monitor > Overview > Top Applications.

    Cisco vManage Release 20.6.1 and earlier: From the Cisco SD-WAN Manager menu, choose Dashboard > Main Dashboard > Top Applications.

  2. In the SAIE Application tab, click an application usage bar in the chart.

    Note

    In Cisco vManage Release 20.7.1 and earlier releases, SAIE Application is called DPI Application.

  3. In the chart for the application that you selected, click the device usage bar.

    If on-demand troubleshooting is configured for the device, detailed top source data appears.

    If on-demand troubleshooting information is not configured, the Go to On Demand Troubleshooting option appears. Continue to Step 4.

  4. If the Go to On Demand Troubleshooting option appears, perform these actions:

    1. Click Go to On Demand Troubleshooting to display the On Demand Troubleshooting window.

    2. In the On Demand Troubleshooting window, add an entry, as described in Add an On Demand Troubleshooting Entry.

    3. Repeat Step 1 to Step 3 in this procedure to view the detailed information.