After upgrading a Cisco Secure Firewall 3130 Threat Defense (FTD) High Availability (HA) pair from version 7.4.3 to 7.6.4, the primary unit appeared to fail at 99% completion on the Firewall Management Center (FMC) and became unresponsive. The secondary unit upgraded successfully and became active in the HA pair. The primary unit was not responding to SSH connections and the FMC could not communicate with it.
Upon investigation, this critical fault was observed on FXOS:
Critical F1000398 2026-08-14T20:52:06.297 58437 [FSM:FAILED]: out-of-band interface configuration(FSM:sam:dme:MgmtIfSwMgmtOobIfConfig)
The FTD upgrade logs showed that the upgrade had actually completed successfully (100% done), but the SFTunnel service was not starting properly, preventing communication between the FTD and FMC. The device could ping itself, its gateway, and the peer FTD unit, but could not reach the FMCs which resided on different subnets.
Cisco Secure Firewall 3130 Threat Defense HA pair
Original software version: FTD 7.4.3
Target software version: FTD 7.6.4 (Build 69)
Cisco Firepower Extensible Operating System (FX-OS) v2.16.1 (build 147)
Management interface configured with static IP addressing
FMC servers located on different subnets from the FTD management network
The issue was resolved by completely reconfiguring the management interface on the affected FTD unit. These steps were performed:
Step 1: Connect to the FTD via console access since SSH was not working.
Step 2: Verify the upgrade completion status by checking the upgrade logs:
root@device:/ngfw/var/log/sf/Cisco_FTD_SSP_FP3K_Upgrade-7.6.4# cat status.log
Step 3: Confirm the management interface fault on FXOS:
device# show fault
Step 4: Attempt basic troubleshooting by reconfiguring the management IP address (this did not resolve the issue).
Step 5: Perform a cold reboot (this did not resolve the issue).
Step 6: Completely disable the management0 interface from FTD CLI:
configure network management-interface disable management0
Step 7: Re-enable the management0 interface:
configure network management-interface enable management0
Step 8: Reconfigure the network settings for the management interface:
configure network ipv4 manual <ip_address> <netmask> <gateway>
Step 9: Verify that FXOS alerts were cleared and the FTD can now ping the FMC:
root@device:/# ping <FMC_IP_address>
Step 10: Restart the SFTunnel service using the manage_procs.pl script:
manage_procs.pl restart sftunnel
Step 11: Verify SFTunnel connectivity:
root@device:/# sftunnel_status.pl
Step 12: Test failover functionality to ensure HA operations are working correctly.
After completing these steps, the FTD was able to properly communicate with the FMC, and all connectivity and failover tests were successful. The hotfix upgrade could then be applied without issues.
The root cause was a failure of the management interface to properly initialize after the FTD upgrade from 7.4.3 to 7.6.4, despite the upgrade process completing successfully. The out-of-band management interface configuration finite state machine (FSM) failed during the upgrade process, preventing the management interface from establishing proper connectivity. This behavior has been documented in Cisco bug ID CSCww08417.
This issue is similar to previously reported defects Cisco bug ID CSCwj04154 and Cisco bug ID CSCwa69303, but occurs under different conditions. The management interface failure created a false impression of upgrade failure at 99% completion, when in reality the upgrade had completed successfully but the management connectivity was broken.
| Revision | Publish Date | Comments |
|---|---|---|
1.0 |
10-Nov-2026
|
Initial Release |