THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.
Revision | Publish Date | Comments |
---|---|---|
1.2 |
23-Feb-22 |
Updated the Problem Description, Background, Problem Symptom, and Workaround/Solution Sections |
1.1 |
22-Dec-21 |
Updated the Products Affected Section |
1.0 |
17-Dec-21 |
Initial Release |
Affected OS Type | Affected Software Product | Affected Release | Affected Release Number | Comments |
---|---|---|---|---|
NON-IOS |
IOSXE |
16 |
16.10.1, 16.10.1a, 16.10.1b, 16.10.1c, 16.10.1d, 16.10.1e, 16.10.1f, 16.10.1g, 16.10.1i, 16.10.1s, 16.11.1, 16.11.1a, 16.11.1b, 16.11.1c, 16.11.1s, 16.12.1, 16.12.1a, 16.12.1c, 16.12.1s, 16.12.1t, 16.12.1w, 16.12.1x, 16.12.1y, 16.12.1z, 16.12.1z1, 16.12.1z2, 16.9.1, 16.9.1a, 16.9.1b, 16.9.1c, 16.9.1d, 16.9.1s |
ESS3x00 Cisco IOS XE Software Releases 17.6.1 or later have the fix. All previous releases need to use the suggested workaround to address the issue. |
NON-IOS |
IOSXE |
17 |
17.1.1, 17.1.1a, 17.1.1s, 17.1.1t, 17.2.1, 17.2.1a, 17.2.1r, 17.2.1v, 17.3.1, 17.3.1a, 17.3.1w, 17.3.1x, 17.3.2, 17.3.2a, 17.3.3, 17.3.3a, 17.4.1, 17.4.1a, 17.4.1b, 17.5.1, 17.5.1a |
ESS3x00 Cisco IOS XE Software Releases 17.6.1 or later have the fix. All previous releases need to use the suggested workaround to address the issue. |
NON-IOS |
IOSXE |
16 |
16.10.1, 16.10.1a, 16.10.1b, 16.10.1c, 16.10.1d, 16.10.1e, 16.10.1f, 16.10.1g, 16.10.1i, 16.10.1s, 16.11.1, 16.11.1a, 16.11.1b, 16.11.1c, 16.11.1s, 16.12.1, 16.12.1a, 16.12.1c, 16.12.1s, 16.12.1t, 16.12.1w, 16.12.1x, 16.12.1y, 16.12.1z, 16.12.1z1, 16.12.1z2, 16.9.1, 16.9.1a, 16.9.1b, 16.9.1c, 16.9.1d, 16.9.1s |
IE3x00 Cisco IOS XE Software Releases 17.6.1 or later have the fix. All previous releases need to use the suggested workaround to address the issue. |
NON-IOS |
IOSXE |
17 |
17.1.1, 17.1.1a, 17.1.1s, 17.1.1t, 17.2.1, 17.2.1a, 17.2.1r, 17.2.1v, 17.3.1, 17.3.1a, 17.3.1w, 17.3.1x, 17.3.2, 17.3.2a, 17.3.3, 17.3.3a, 17.4.1, 17.4.1a, 17.4.1b, 17.5.1, 17.5.1a |
IE3x00 IOS XE Software Releases 17.6.1 or later have the fix. All previous releases need to use the suggested workaround to address the issue. |
NON-IOS |
IOSXE |
17 |
17.4.1, 17.4.1a, 17.4.1b, 17.4.2, 17.4.2a, 17.5.1, 17.5.1a |
ESS9300 IOS XE Software Releases 17.6.1 or later have the fix. All previous releases need to use the suggested workaround to address the issue. |
Defect ID | Headline |
---|---|
CSCvx00521 | QuoVadis root CA decommission impacting Smart Licensing and Smart Call Home Functionality |
For affected IOS XE versions of Cisco Catalyst (IE3x00) Rugged Series Switches, Cisco Catalyst (IE34xx ) Heavy Duty Series Switches, Cisco Embedded Services (ESS3x00) Series and Cisco Embedded Services (ESS9300) Switches, some Secure Sockets Layer (SSL) certificates issued from the QuoVadis root certificate authority (CA) trust chain before March 31, 2021 cannot be renewed from this CA. Once those certificates (issued by QuoVadis) expire on devices or are removed from the Cisco cloud servers, functions such as Smart Licensing and Smart Call Home will fail to establish secure connections to Cisco and might not operate properly.
The QuoVadis Root CA 2 Public Key Infrastructure (PKI) used by Cisco IOS® XE software to issue SSL certificates is subject to an industry-wide issue that affects revocation abilities. Due to this issue, no new QuoVadis Root CA 2 certificates will be issued or renewed by Cisco after March 31, 2021. This affects certificate renewals on devices, Cisco cloud servers, and third-party services.
Certificates issued before the QuoVadis Root CA 2 was decommissioned will continue to be valid. However, the certificates will not renew when they expire on either the device or the Cisco cloud server. This will cause functions such as Smart Licensing and Smart Call Home to fail to establish secure connections to Cisco cloud servers.
This table shows a summary of the QuoVadis Root CA 2 certificate expiration dates for affected Cisco services.
Cisco Cloud Server | QuoVadis Certificate Expiration Date | Affected Services |
---|---|---|
tools.cisco.com | February 5, 2022 |
|
smartreceiver.cisco.com | January 26, 2023 |
|
Expiration of the QuoVadis Root CA 2 certificates affects these services with the associated symptoms.
Affected Services | Symptoms for Affected Services |
---|---|
Smart Licensing | Failure to connect to the server (Details are provided in this section) |
Smart Call Home | Failure to connect to the server and the Call-Home HTTP request fails |
For Cisco IOS XE based devices, affected devices will be unable to connect to the Smart Licensing and Smart Call Home services hosted by Cisco. Smart licenses might fail entitlement and reflect an Out of Compliance status.
The features that use Smart Licensing will continue to function for one year after the last successful secure connection. Some Smart Licensing symptoms are:
Note: Offline licensing, such as Permanent License Reservation (PLR) and Specific License Reservation (SLR), is not affected by the certificate change on the Smart Licensing server.
For additional information, refer to the Cisco Smart Licensing Guide, Smart Licensing using Policy on Catalyst Switching Platforms, and Release Notes for Cisco Catalyst IE3x00 Rugged, IE3400 Heavy Duty, ESS3300, and ESS9300 Series Switches.
Troubleshooting Tech Notes
Cisco Smart Licensing - Troubleshooting Steps and Considerations on Catalyst Platforms
If there is an error in communication with the server due to trust or other reasons, these error messages might be observed:
%CALL_HOME-5-SL_MESSAGE_FAILED: Fail to send out Smart Licensing message to:https://<;ip>/its/service/oddce/services/DDCEService (ERR 205 : Request Aborted)
%SMART_LIC-3-COMM_FAILED:Communications failure with the Cisco Smart Software Manager or satellite: Fail to send out Call Home HTTP message.
%SMART_LIC-3-AUTH_RENEW_FAILED:Authorization renewal with the Cisco Smart Software Manager or satellite: Communication message send error for udi PID:XXX, SN: XXX
%PKI-3-CRL_FETCH_FAIL: CRL fetch for trustpoint SLA-TrustPoint failed Reason : Failed to select socket. Timeout : 5 (Connection timed out)
%PKI-3-CRL_FETCH_FAIL: CRL fetch for trustpoint SLA-TrustPoint failed Reason : Failed to select socket. Timeout : 5 (Connection timed out)
%SMART_LIC-3-OUT_OF_COMPLIANCE: One or more entitlements are out of compliance
%SMART_LIC-3-AUTH_RENEW_FAILED: Authorization renewal with the Cisco Smart Software Manager (CSSM) : Error received from Smart Software Manager: Data and signature do not match for udi PID:IE3300,SN:XXXXXXXXXXX
SAEVT_DEREGISTER_STATUS msgStatus="LS_INVALID_DATA" error="Missing Id cert serial number field; Missing signing cert serial number field; Signed data and certificate does not match
%SMART_LIC-3-COMM_FAILED: Communications failure with the Cisco Smart Software Manager (CSSM) : No detailed information given
Cisco has migrated from the QuoVadis Root CA 2 to the IdenTrust Commercial Root CA 1 for SSL certificates. Cisco recommends these two options to add the new IdenTrust Commercial Root CA 1 certificate to the affected devices.
Software Upgrade
For Cisco IOS XE based products, upgrade the software to the fixed Release 17.6.1 or later in order to resolve the root CA certificate issue for affected platforms.
Manual Certificate Update
In order to resolve the issue without a software upgrade, either of these two workarounds can be implemented.
Workaround 1
Enter this CLI command in order to manually import the IdenTrust Commercial Root CA 1 into the product truststore:
Device(config)# crypto pki trustpool import url http://www.cisco.com/security/pki/trs/ios_core.p7b
Or if it is already copied to the device flash, enter this command:
Device(config)# crypto pki trustpool import url flash:ios_core.p7b
Workaround 2
Enter the CLI commands in the procedure in order to manually import the IdenTrust Commercial Root CA 1 into the product truststore.
The updated IdenTrust Root CA 1 is shown here and complies with sha1WithRSAEncryption signature algorithm requirements.
Manual Update of Trustpool via Terminal
-----BEGIN CERTIFICATE----- MIIFYDCCA0igAwIBAgIQCgFCgAAAAUUjyES1AAAAAjANBgkqhkiG9w0BAQsFADBK MQswCQYDVQQGEwJVUzESMBAGA1UEChMJSWRlblRydXN0MScwJQYDVQQDEx5JZGVu VHJ1c3QgQ29tbWVyY2lhbCBSb290IENBIDEwHhcNMTQwMTE2MTgxMjIzWhcNMzQw MTE2MTgxMjIzWjBKMQswCQYDVQQGEwJVUzESMBAGA1UEChMJSWRlblRydXN0MScw JQYDVQQDEx5JZGVuVHJ1c3QgQ29tbWVyY2lhbCBSb290IENBIDEwggIiMA0GCSqG SIb3DQEBAQUAA4ICDwAwggIKAoICAQCnUBneP5k91DNG8W9RYYKyqU+PZ4ldhNlT 3Qwo2dfw/66VQ3KZ+bVdfIrBQuExUHTRgQ18zZshq0PirK1ehm7zCYofWjK9ouuU +ehcCuz/mNKvcbO0U59Oh++SvL3sTzIwiEsXXlfEU8L2ApeN2WIrvyQfYo3fw7gp S0l4PJNgiCL8mdo2yMKi1CxUAGc1bnO/AljwpN3lsKImesrgNqUZFvX9t++uP0D1 bVoE/c40yiTcdCMbXTMTEl3EASX2MN0CXZ/g1Ue9tOsbobtJSdifWwLziuQkkORi T0/Br4sOdBeo0XKIanoBScy0RnnGF7HamB4HWfp1IYVl3ZBWzvurpWCdxJ35UrCL vYf5jysjCiN2O/cz4ckA82n5S6LgTrx+kzmEB/dEcH7+B1rlsazRGMzyNeVJSQjK Vsk9+w8YfYs7wRPCTY/JTw436R+hDmrfYi7LNQZReSzIJTj0+kuniVyc0uMNOYZK dHzVWYfCP04MXFL0PfdSgvHqo6z9STQaKPNBiDoT7uje/5kdX7rL6B7yuVBgwDHT c+XvvqDtMwt0viAgxGds8AgDelWAf0ZOlqf0Hj7h9tgJ4TNkK2PXMl6f+cB7D3hv l7yTmvmcEpB4eoCHFddydJxVdHixuuFucAS6T6C6aMN7/zHwcz09lCqxC0EOoP5N iGVreTO01wIDAQABo0IwQDAOBgNVHQ8BAf8EBAMCAQYwDwYDVR0TAQH/BAUwAwEB /zAdBgNVHQ4EFgQU7UQZwNPwBovupHu+QucmVMiONnYwDQYJKoZIhvcNAQELBQAD ggIBAA2ukDL2pkt8RHYZYR4nKM1eVO8lvOMIkPkp165oCOGUAFjvLi5+U1KMtlwH 6oi6mYtQlNeCgN9hCQCTrQ0U5s7B8jeUeLBfnLOic7iPBZM4zY0+sLj7wM+x8uwt LRvM7Kqas6pgghstO8OEPVeKlh6cdbjTMM1gCIOQ045U8U1mwF10A0Cj7oV+wh93 nAbowacYXVKV7cndJZ5t+qntozo00Fl72u1Q8zW/7esUTTHHYPTa8Yec4kjixsU3 +wYQ+nVZZjFHKdp2mhzpgq7vmrlR94gjmmmVYjzlVYA211QC//G5Xc7UI2/YRYRK W2XviQzdFKcgyxilJbQN+QHwotL0AMh0jqEqSI5l2xPE4iUXfeu+h1sXIFRRk0pT AwvsXcoz7WL9RccvW9xYoIA55vrX/hMUpu09lEpCdNTDd1lzzY9GvlU47/rokTLq l1gEIt44w8y8bckzOmoKaT+gyOpyj4xjhiO9bTyWnpXgSUyqorkqG5w2gXjtw+hG 4iZZRHUe2XWJUc0QhJ1hYMtd+ZciTY6Y5uN/9lu7rs3KSoFrXgvzUeF0K+l+J6fZ mUlO+KWA2yUPHGNiiskzZ2s8EIPGrd6ozRaOjfAHN3Gf8qv8QfXBi+wAN10J5U6A 7/qxXDgGpRtK4dw4LTzcqx+QGtVKnO7RcGzM7vRX+Bi6hG6H -----END CERTIFICATE-----
Procedure:
config t
command.crypto pki trustpool import terminal
command.exit
.wr mem
command.show crypto pki trustpool
command.See this example:
Device#config t Enter configuration commands, one per line. End with CNTL/Z. Switch(config)#crypto pki trustpool import terminal % Enter PEM-formatted CA certificate. % End with a blank line or "quit" on a line by itself. MIIFYDCCA0igAwIBAgIQCgFCgAAAAUUjyES1AAAAAjANBgkqhkiG9w0BAQsFADBK MQswCQYDVQQGEwJVUzESMBAGA1UEChMJSWRlblRydXN0MScwJQYDVQQDEx5JZGVu VHJ1c3QgQ29tbWVyY2lhbCBSb290IENBIDEwHhcNMTQwMTE2MTgxMjIzWhcNMzQw MTE2MTgxMjIzWjBKMQswCQYDVQQGEwJVUzESMBAGA1UEChMJSWRlblRydXN0MScw JQYDVQQDEx5JZGVuVHJ1c3QgQ29tbWVyY2lhbCBSb290IENBIDEwggIiMA0GCSqG SIb3DQEBAQUAA4ICDwAwggIKAoICAQCnUBneP5k91DNG8W9RYYKyqU+PZ4ldhNlT 3Qwo2dfw/66VQ3KZ+bVdfIrBQuExUHTRgQ18zZshq0PirK1ehm7zCYofWjK9ouuU +ehcCuz/mNKvcbO0U59Oh++SvL3sTzIwiEsXXlfEU8L2ApeN2WIrvyQfYo3fw7gp S0l4PJNgiCL8mdo2yMKi1CxUAGc1bnO/AljwpN3lsKImesrgNqUZFvX9t++uP0D1 bVoE/c40yiTcdCMbXTMTEl3EASX2MN0CXZ/g1Ue9tOsbobtJSdifWwLziuQkkORi T0/Br4sOdBeo0XKIanoBScy0RnnGF7HamB4HWfp1IYVl3ZBWzvurpWCdxJ35UrCL vYf5jysjCiN2O/cz4ckA82n5S6LgTrx+kzmEB/dEcH7+B1rlsazRGMzyNeVJSQjK Vsk9+w8YfYs7wRPCTY/JTw436R+hDmrfYi7LNQZReSzIJTj0+kuniVyc0uMNOYZK dHzVWYfCP04MXFL0PfdSgvHqo6z9STQaKPNBiDoT7uje/5kdX7rL6B7yuVBgwDHT c+XvvqDtMwt0viAgxGds8AgDelWAf0ZOlqf0Hj7h9tgJ4TNkK2PXMl6f+cB7D3hv l7yTmvmcEpB4eoCHFddydJxVdHixuuFucAS6T6C6aMN7/zHwcz09lCqxC0EOoP5N iGVreTO01wIDAQABo0IwQDAOBgNVHQ8BAf8EBAMCAQYwDwYDVR0TAQH/BAUwAwEB /zAdBgNVHQ4EFgQU7UQZwNPwBovupHu+QucmVMiONnYwDQYJKoZIhvcNAQELBQAD ggIBAA2ukDL2pkt8RHYZYR4nKM1eVO8lvOMIkPkp165oCOGUAFjvLi5+U1KMtlwH 6oi6mYtQlNeCgN9hCQCTrQ0U5s7B8jeUeLBfnLOic7iPBZM4zY0+sLj7wM+x8uwt LRvM7Kqas6pgghstO8OEPVeKlh6cdbjTMM1gCIOQ045U8U1mwF10A0Cj7oV+wh93 nAbowacYXVKV7cndJZ5t+qntozo00Fl72u1Q8zW/7esUTTHHYPTa8Yec4kjixsU3 +wYQ+nVZZjFHKdp2mhzpgq7vmrlR94gjmmmVYjzlVYA211QC//G5Xc7UI2/YRYRK W2XviQzdFKcgyxilJbQN+QHwotL0AMh0jqEqSI5l2xPE4iUXfeu+h1sXIFRRk0pT AwvsXcoz7WL9RccvW9xYoIA55vrX/hMUpu09lEpCdNTDd1lzzY9GvlU47/rokTLq l1gEIt44w8y8bckzOmoKaT+gyOpyj4xjhiO9bTyWnpXgSUyqorkqG5w2gXjtw+hG 4iZZRHUe2XWJUc0QhJ1hYMtd+ZciTY6Y5uN/9lu7rs3KSoFrXgvzUeF0K+l+J6fZ mUlO+KWA2yUPHGNiiskzZ2s8EIPGrd6ozRaOjfAHN3Gf8qv8QfXBi+wAN10J5U6A 7/qxXDgGpRtK4dw4LTzcqx+QGtVKnO7RcGzM7vRX+Bi6hG6H % PEM files import succeeded. Device(config)#exit Device#wr mem Destination filename [startup-config]? Building configuration... [OK] Device#show crypto pki trustpool Load for five secs: 30%/2%; one minute: 25%; five minutes: 27% Time source is NTP, 23:40:09.537 CST Sat Mar 6 2021 CA Certificate Status: Available Certificate Serial Number (hex): 0A0142800000014523C844B500000002 Certificate Usage: Signature Issuer: cn=IdenTrust Commercial Root CA 1 o=IdenTrust c=US Subject: cn=IdenTrust Commercial Root CA 1 o=IdenTrust c=US Validity Date: start date: 02:12:23 CST Jan 17 2014 end date: 02:12:23 CST Jan 17 2034 Associated Trustpoints: Trustpool Trustpool: Downloaded CA Certificate Status: Available Certificate Serial Number (hex): 0509 Certificate Usage: Signature Issuer: cn=QuoVadis Root CA 2 o=QuoVadis Limited c=BM Subject: cn=QuoVadis Root CA 2 o=QuoVadis Limited c=BM Validity Date: start date: 02:27:00 CST Nov 25 2006 end date: 02:23:33 CST Nov 25 2031 Associated Trustpoints: Trustpool Trustpool: Built-In <<output snipped>>
Cisco has created a web page to provide customers and partners with additional information on this issue. Consult the QuoVadis Root CA 2 Decommission page for a full list of products affected, associated Field Notices, and frequently asked questions.
If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:
My Notifications—Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.