Configure Certificate for FND to SSM Communication

Introduction

This document describes how to correctly configure communication issues between Field Network Director (FND) and Software Security Module (SSM).

Problem

Since FND 4.4, communication between the FND application server and the SSM service requires mutual authentication.

In case this mutual authentication is not configured correctly or the certificates do not match, the connection from FND to the SSM is refused. 

This can be seen in server.log, if the logging is set to debug, as follows:

7645: SLC-FND: Jun 20 2019 13:22:49.929 +0000: %IOTFND-7-UNSPECIFIED: %[ch=SSMClient][sev=DEBUG][tid=http-/0.0.0.0:443-5]: Sending request to SSM Server. Request :https://127.0.0.1:8445/api/v0/ssmws/loadKeyStore.json
7646: SLC-FND: Jun 20 2019 13:22:49.930 +0000: %IOTFND-7-UNSPECIFIED: %[ch=BasicClientConnectionManager][sev=DEBUG][tid=http-/0.0.0.0:443-5]: Get connection for route {s}->https://127.0.0.1:8445
7647: SLC-FND: Jun 20 2019 13:22:49.931 +0000: %IOTFND-7-UNSPECIFIED: %[ch=DefaultClientConnectionOperator][sev=DEBUG][tid=http-/0.0.0.0:443-5]: Connecting to 127.0.0.1:8445
7648: SLC-FND: Jun 20 2019 13:22:49.938 +0000: %IOTFND-7-UNSPECIFIED: %[ch=DefaultClientConnection][sev=DEBUG][tid=http-/0.0.0.0:443-5]: Connection org.apache.http.impl.conn.DefaultClientConnection@370804ff closed
7649: SLC-FND: Jun 20 2019 13:22:49.938 +0000: %IOTFND-7-UNSPECIFIED: %[ch=DefaultClientConnection][sev=DEBUG][tid=http-/0.0.0.0:443-5]: Connection org.apache.http.impl.conn.DefaultClientConnection@370804ff shut down
7650: SLC-FND: Jun 20 2019 13:22:49.938 +0000: %IOTFND-7-UNSPECIFIED: %[ch=BasicClientConnectionManager][sev=DEBUG][tid=http-/0.0.0.0:443-5]: Releasing connection org.apache.http.impl.conn.ManagedClientConnectionImpl@7bc2e02f
7651: SLC-FND: Jun 20 2019 13:22:49.938 +0000: %IOTFND-7-UNSPECIFIED: %[ch=BasicClientConnectionManager][sev=DEBUG][tid=http-/0.0.0.0:443-5]: Connection can be kept alive for 9223372036854775807 MILLISECONDS
7652: SLC-FND: Jun 20 2019 13:22:49.938 +0000: %IOTFND-7-UNSPECIFIED: %[ch=SSMClient][sev=DEBUG][tid=http-/0.0.0.0:443-5][part=7652.1/114]: Please verify SSM server status. No response received.
7653: SLC-FND: Jun 20 2019 13:22:49.938 +0000: %IOTFND-7-UNSPECIFIED: %[ch=SSMClient][sev=DEBUG][tid=http-/0.0.0.0:443-5][part=7652.2/114]: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated

Solution

The certificate that is used by the FND server to do the client authentication on the SSM server is the FND web certificate from the jbossas_keystore. 

In order for the SSM to trust this certificate, you need to go through these steps:

1. Export the web certificate with the use of the GUI. Navigate to Admin > System Management > Certificates > Certificate for Web and then click on Download (base64) as shown in the image.

2. Copy the text file or create a new file on the FND server with the certificate contents from Step 1. For this example, the file is saved in /opt/cgms/server/cgms/conf/webcert.crt:

[root@fndnms ~]# vi /opt/cgms/server/cgms/conf/webcert.crt
[root@fndnms ~]# cat /opt/cgms/server/cgms/conf/webcert.crt
-----BEGIN CERTIFICATE-----
MIIDbTCCAlWgAwIBAgIEESL+rTANBgkqhkiG9w0BAQsFADBnMQswCQYDVQQGEwJV
UzELMAkGA1UECBMCQ0ExETAPBgNVBAcTCFNhbiBKb3NlMRYwFAYDVQQKEw1DaXNj
byBTeXN0ZW1zMQ8wDQYDVQQLEwZJb1RTU0cxDzANBgNVBAMTBkNHLU5NUzAeFw0x
NTAzMDMyMTU4MTNaFw0yMDAzMDEyMTU4MTNaMGcxCzAJBgNVBAYTAlVTMQswCQYD
VQQIEwJDQTERMA8GA1UEBxMIU2FuIEpvc2UxFjAUBgNVBAoTDUNpc2NvIFN5c3Rl
bXMxDzANBgNVBAsTBklvVFNTRzEPMA0GA1UEAxMGQ0ctTk1TMIIBIjANBgkqhkiG
9w0BAQEFAAOCAQ8AMIIBCgKCAQEAlsgdELNUFi9eXHcb550y0UgbPMgucsKqT1+E
xmwEri517fo+BHdg6AuXpDP4KvLW1/cx8xqWbheKAfPht/HqiFX0ltZdoWaQcaJz
YJOiuL/W3BwQW6UMWPnC1p/Dgnz+qR3JQpR20hC4ymHIIVwKwVfiaJZAnSFNKaZ4
uhOuJDkEC0ZyBbp5Y2Mi9zVRTv/g98p0IqpOjxV0JUtlRkWkjkvCma/Q6dZzSdle
YZzyAS/ud4KVxytKKoxBBDPrtPRbT6lu2VMyWe26cRjPCveZffBABoSvLjptnb7H
mxJMW7EbL+zjTAL/GmHh8J9P16MX7EoePCPCQdwPRdfQ3GkTKwIDAQABoyEwHzAd
BgNVHQ4EFgQUfyFoDj0hJLtUu6ZtKCHuisCQfl4wDQYJKoZIhvcNAQELBQADggEB
AF9fVfEwqbP4BszGHfzTa8pf4zUPJ3Lcz1z6RxwtyGXq8oZK8YQWRpa2NQKLDnve
VjXSdOBvDKRYqPkZeAmTRS0BobeZr2NdHb/FNXMlR6eBm56UrefW+VdQE7syOmGq
Ynlwb/1KF/Fkyp2xVk7nHCtHl+I90l3DlyPmGbQ/TxgA6PXY6V6d571IARNdohYm
qZ/3B+ZK/F4PLOcUwWDtxTBFnlElyq+YjhZiqsCmsxI1GWqleWltUVGMXNM1YLN5
N1KAbOeC0O4n2MqzTWTU9Ss51WfceWsBoSPO+4xyzcRDZmo7IWZiwp4ZAO3eYOz/
4aUEdBZxv29+QQ7dq6ZZOXQ=
-----END CERTIFICATE-----

3. Run this command in order to import the certificate as trusted in the ssm_web_keystore:

[root@fndnms ~]# keytool -import -trustcacerts -alias fnd -keystore /opt/cgms-ssm/conf/ssm_web_keystore -file /opt/cgms/server/cgms/conf/webcert.crt
Enter keystore password:
Owner: CN=CG-NMS, OU=IoTSSG, O=Cisco Systems, L=San Jose, ST=CA, C=US
Issuer: CN=CG-NMS, OU=IoTSSG, O=Cisco Systems, L=San Jose, ST=CA, C=US
Serial number: 1122fead
Valid from: Tue Mar 03 22:58:13 CET 2015 until: Sun Mar 01 22:58:13 CET 2020
Certificate fingerprints:
         MD5:  6D:63:B9:8B:3F:C5:E9:6B:2B:DD:77:30:55:9D:C6:E7
         SHA1: 5F:3B:84:92:06:22:CE:C4:FA:8B:F0:46:65:4B:CE:74:61:AA:3B:AE
         SHA256: 1C:59:50:40:92:09:66:D3:67:E9:AE:CA:6D:C8:25:88:FF:A8:26:F7:62:8A:13:EB:0E:EC:57:32:DB:03:94:31
         Signature algorithm name: SHA256withRSA
         Version: 3

Extensions:

#1: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 7F 21 68 0E 3D 21 24 BB   54 BB A6 6D 28 21 EE 8A  .!h.=!$.T..m(!..
0010: C0 90 7E 5E                                        ...^
]
]

Trust this certificate? [no]:  yes
Certificate was added to keystore

4. Once the certificate has been imported, restart the SSM service:

[root@fndnms ~]# systemctl restart ssm
[root@fndnms ~]# systemctl status ssm
● ssm.service - (null)
   Loaded: loaded (/etc/rc.d/init.d/ssm; bad; vendor preset: disabled)
   Active: active (running) since Thu 2019-06-20 17:44:11 CEST; 5s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 11463 ExecStop=/etc/rc.d/init.d/ssm stop (code=exited, status=0/SUCCESS)
  Process: 11477 ExecStart=/etc/rc.d/init.d/ssm start (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/ssm.service
           └─11485 java -server -Xms128m -Xmx1g -XX:MaxPermSize=256m -server -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/cgms-ssm/log -XX:-OmitStackTraceInFastThrow -Dbase.dir=/opt/cgms-ssm -Dlog4j...

Jun 20 17:44:10 fndnms systemd[1]: Starting (null)...
Jun 20 17:44:11 fndnms ssm[11477]: Starting Software Security Module Server: [  OK  ]
Jun 20 17:44:11 fndnms systemd[1]: Started (null).

You can verify if FND is able to communicate to SSM. Navigate to Admin > Certificates > Certificate for CSMP in the FND GUI.

If all goes well, you should be able to see the CSMP certificate in SSM as shown in the image.