Troubleshoot SMAUG FPD "NOT READY" in NCS4K

Introduction

This document describes how to fix SMAUG – Field Programmable Device (FPD) "NOT READY" and the logs that need to be collected for Root Cause Analysis (RCA).

Troubleshoot "SMAUG FPD NOT READY" in NCS4K

Platform: NCS4K

Card: 400G Line card – NCS4K-4H-OPW-QC2

Probable Trigger: FPD upgrade and card reload, RP reload, or switch over

The issue is depicted here:

Problematic state is depicted

Logs to be collected:

• From XR: show tech-support statandshow tech-support slice-manager
• From Admin: show tech-support slice-manager, show tech-support shelf_mgr,show tech-support ccc, show tech-support HBloss
• From the XR: dumpcore running grima_driver location <active LC>
• From the active LC get the traces for grima and dma. Perform the steps from XR Prompt.

#attach location 0/lc0

Fri Jun 7 00:20:00.329 CDT

[xr-vm_node0_LC0_CPU0:~]$ export PS1=#

#lspci -vd:8bec

#grima_ltrace_show -A

#nl_dma_ltrace_show -A

To recover the issue:

• From XR check for the process for grima_driver and slice_manager, then restart them for active LC location.

show processes grima_driver location <0/LC0 or 0/LC1 – active LC>

show processes slice_manager location <0/LC0 or 0/LC1 – active LC>

Process restart grima_driver location <active LC>

Process restart slice_manger location <active LC>

This must fix the issue in most cases.