Cisco Crosswork Data Gateway 7.2 Installation and Configuration Guide for Cloud Applications

PDF

Cisco Crosswork Data Gateway 7.2 Installation and Configuration Guide for Cloud Applications

Crosswork Data Gateway VMs troubleshooting

Want to summarize with AI?

Log in

Outlines the Troubleshooting menu options for Crosswork Data Gateway VMs, including diagnostic commands, Showtech execution, system shutdown, auditd log exports, TAC shell access, and re-enrollment procedures to help users identify and resolve operational issues.


To access the Troubleshooting menu, select 5 Troubleshooting from the Main Menu.

Troubleshooting menu overview

The Troubleshooting menu provides several options to diagnose and resolve issues with the Crosswork Data Gateway VM.

Note
  1. Some options may be restricted for the dg-oper user. See Table 1.

  2. Crosswork Cloud does not support the Remove All Non-Infra Containers and Reboot option under the Troubleshooting menu.

The Troubleshooting menu provides the options listed here:

Figure 1. Troubleshooting options
The Troubleshooting menu lists options for diagnosing and managing the Crosswork Data Gateway VM, including running diagnostics, rebooting or shutting down the VM, exporting logs, re-enrolling the data gateway, and exiting the menu.

Diagnostic commands

The Run Diagnostics menu provides you these options in the console:

Figure 2. Run diagnostics

Ping a host

The Crosswork Data Gateway provides a ping utility to check the reachability of any IP address.

Procedure

1.

From the Main Menu, select Troubleshooting > Run Diagnostics > Ping.

2.

Enter the required information:

  • Number of pings: Specify how many pings to send.

  • Destination hostname or IP: Enter the target hostname or IP address.

  • Source port: Choose the type (ICMP, ARP, UDP, TCP, or TCP Connect).

  • Destination port: Select the appropriate type (UDP, TCP, or TCP Connect).

3.

Click OK.


Traceroute to a Host

The Crosswork Data Gateway offers the Traceroute option to help troubleshoot latency issues. This tool provides an estimate of the time it takes for the gateway to reach the destination.

Procedure

1.

From the Main Menu, select Troubleshooting > Run Diagnostics > Traceroute.

2.

Specify the destination for the traceroute.

3.

Click OK.


Troubleshoot the commands in Crosswork Data Gateway

The Crosswork Data Gateway provides a set of diagnostic commands to assist with troubleshooting.

Procedure

1.

From the Main Menu, navigate to Troubleshooting > Run Diagnostics.

2.

Choose one of these commands based on your troubleshooting needs:

Figure 3. Troubleshooting options

Apply any relevant filters or options for the selected command.

3.

Click OK.

Crosswork Data Gateway clears the screen and executes the selected command with the specified options.


Download the tcpdump

The tcpdump utility allows you to capture and analyze network traffic on Crosswork Data Gateway.

Note

Only the dg-admin user can run the tcpdump utility.

Procedure

1.

From the Main Menu, navigate to Troubleshooting > Run Diagnostics > tcpdump.

2.

Choose an interface to run tcpdump on. To capture traffic from all interfaces, select the All option.

3.

Select whether to display packet information on screen or save it to a file.

4.

Set the following parameters:

  • Packet count limit

  • Collection time limit

  • File size limit

  • Filter expression

5.

Click OK.

  • When tcpdump reaches the specified limits, Crosswork Data Gateway will:

    • Compress the capture file.

    • Prompt for SCP credentials to transfer the file to a remote host.

  • Once the file transfer is complete (or canceled), the compressed file is deleted.


Run a controller session test

To verify if Crosswork Data Gateway can establish a connection to Crosswork Cloud, use the Controller Session Test. This test also checks whether the VM’s resource allocation matches the deployment profile.

Procedure

From the Main Menu, navigate to Troubleshooting > Run Diagnostics > Run Controller Session Test.

If the connection is successful, a message confirming the connection appears. If the connection fails, the console displays these details to help you troubleshoot:

  • DNS server IP address

  • DNS domain

  • NTP server address

  • NTP status

  • Proxy URL

  • Proxy reachability status

  • Controller URL

  • Controller reachability status

  • Last test date

Figure 4. Run Controller Session Tests menu
Figure 5. Result of the Run Controller Session Tests menu

What to do next

If the session test fails, review the displayed information to determine the probable cause. Follow the corrective actions suggested by the console.


Run the Showtech command

The Showtech command allows you to export logs and vital information from the Crosswork Data Gateway to a user-defined SCP destination.

Typically, the command enables you to collect:

  • Logs from all Crosswork Data Gateway components running on Docker containers

  • VM vitals

When you run the command, it creates a tarball in the directory where it is executed. The tarball is named DG-<CDG version>-<CDG host name>-year-month-day--hour-minute-second.tar.xz.enc.

Procedure

1.

From the Troubleshooting menu, select Show-tech and click OK.

2.

Specify where to save the tarball containing logs and VM vitals.

3.

Enter your SCP passphrase and click OK.

The show-tech file is downloaded in an encrypted format.

Note

The download may take several minutes depending on the system usage.

4.

After the download is complete, run the following command to decrypt it:

openssl enc -d -aes-256-ctr -pbkdf2 -md sha512 -iter 100000 -in <showtech file> -out <decrypted filename> -pass pass:<password>

For example:

openssl enc -d -aes-256-ctr -pbkdf2 -md sha3-512 -iter 100000 -in show-tech-file.tar.xz.enc -out show-tech-file.tar.xz -pass pass: myPassword
Note
  • Use OpenSSL version 1.1.1i to decrypt the file. To check the OpenSSL version on your system, use the command openssl version.

  • The <showtech file> must have a .tar.xz extension.

  • Do not enclose the filenames <showtech file> and <decrypted filename> in quotation marks.

  • To decrypt on a MAC, you need OpenSSL 1.1.1+, as LibreSSL does not support all the necessary switches.


Shutdown the Crosswork Data Gateway VM

From the Troubleshooting menu, select 5 Shutdown VM to power off the Crosswork Data Gateway VM.


Export the auditd logs

Follow the steps to export auditd logs.

Procedure

1.

From Troubleshooting, select Export Audit Logs.

2.

Enter a passphrase to encrypt the auditd log tarball.

3.

Click OK.


Re-enroll Crosswork Data Gateway

To re-enroll Crosswork Data Gateway, complete each step in this task.

Before you begin

Before you re-enroll Crosswork Data Gateway, delete the existing enrollment from the controller.

Procedure

1.

From the Troubleshooting menu, select Re-enroll Data Gateway.

2.

Review the information in the confirmation window and click Yes to proceed.

Figure 6. Re-enroll Data Gateway Confirmation Window

Remove the rotated log files

To remove all rotated log files such as those with the .gz or .xz extension from the /var/log and /opt/dg/log folders, perform these steps:

Procedure

1.

From the Troubleshooting menu, select Remove Rotated Log Files.

2.

In the dialog that appears, select Yes to confirm and proceed with the log removal.


Enable the TAC shell access

The TAC Shell Access function allows a Cisco engineer to log in directly to the Ubuntu shell using multifactor authentication through the dg-tac user.

By default, the dg-tac account is locked and the password is expired to prevent unauthorized access. Once enabled, the dg-tac user is active for less than 24 hours (until midnight UTC [00:00 UTC] the next day).

Before you begin

Confirm that the Cisco engineer you are working with has access to the Secure Web Identity Management Service (SWIMS) Aberto tool. Active communication with the Cisco engineer is required to enable dg-tac access.

  • Enabling this access requires you to communicate actively with the Cisco engineer.

Procedure

1.

Log in to the Crosswork Data Gateway VM as the dg-admin user.

2.

From the Main Menu, select Troubleshooting.

3.

From the Troubleshooting menu, select Enable TAC Shell Access.

A dialog appears, warning you that the dg-tac user login requires a password you set, along with a challenge token from TAC. Choose Yes to continue or No to cancel.

4.

If you proceed, the system prompts you to set a password for the dg-tac user.

5.

Enter a password, and the system displays the expiration date when the account will be disabled.

6.

Log out of Crosswork Data Gateway.

7.

If the Cisco engineer has direct access to the Crosswork Data Gateway VM, share the password you set in Step 3.

  1. Share the password that you had set in Step 5 for the dg-tac user with the Cisco engineer who is working with you.

  2. The engineer logs in via SSH as the dg-tac user with the password you provided.

    The system will then prompt for a challenge token. The engineer signs it using SWIMS Aberto, pastes the signed response into the VM, and logs in successfully.

  3. The Cisco engineer logs in successfully as the dg-tac user and completes the troubleshooting.

    There is a fifteen-minute idle timeout period for the dg-tac user. If the Cisco engineer logs out, they must sign a new challenge to log in again.

  4. After troubleshooting is complete, the Cisco engineer logs out of the TAC shell.

8.

If the Cisco engineer does not have direct access:

  1. Start a meeting with desktop sharing enabled.

  2. Log in as dg-tac using SSH:

    ssh dg-tac@<DG hostname or IP>
  3. Enter the password that you set and obtain the challenge token.

  4. Share the token with the Cisco engineer, who will sign it using SWIMS Aberto and provide the signed response.

  5. Paste the signed response back into the VM to get the shell prompt.

  6. Share your desktop, or follow the engineer’s instructions to troubleshoot.

    There is a fifteen-minute idle timeout period for the dg-tac user. If logged out, the Cisco engineer must sign a new challenge to log in again.

  7. Once troubleshooting is complete, the engineer logs out of the TAC shell.


Audit the TAC shell events

Timestamps of the TAC shell events are logged in the tac_shell.log file and sent to the Crosswork Cloud controller:

  • TAC shell enabled

  • TAC shell disabled

  • dg-tac login

  • dg-tac log out

If the Crosswork Data Gateway cannot connect to the Crosswork Cloud controller, the events are logged in the /opt/dg/data/controller-gateway/audit/pending folder. Once connectivity is restored, these events are sent within 5 minutes.

The tac_shell.log file is available in the Showtech bundle of the Crosswork Data Gateway VM.