Mobile & Remote Access (MRA) is a deployment solution for Virtual Private Network-less (VPN) Jabber capability. This solution allows end users to connect to internal enterprise resources from anywhere in the world. This guide has been written to give engineers who troubleshoot the Collaboration Edge solution the capability to quickly identify and resolve the most common problems customers face during the deployment phrase.
Cisco recommends that you have knowledge of these topics:
Cisco Unified Communications Manager (CUCM)
Cisco Expressway Core
Cisco Expressway Edge
Cisco IM and Presence (IM&P)
Cisco Jabber for Windows
Cisco Jabber for MAC
Cisco Jabber for Android
Cisco Jabber for iOS
Domain Name System (DNS)
The information in this document is based on these software and hardware versions:
Expressway Version X8.1.1 or later
CUCM Release 9.1(2)SU1 or later and IM&P Version 9.1(1) or later
Cisco Jabber Version 9.7 or later
Log In Issues
Jabber Unable to Sign In Through MRA
This symptom can be caused by a wide range of issues, a few of which are outlined here.
1. Collaboration Edge Service Record (SRV) Not Created and/or Port 8443 Unreachable
For a Jabber client to be able to log in successfully with MRA, a specific collaboration edge SRV record must be created and accessible externally. When a Jabber client is initially started, it makes DNS SRV queries:
_cisco-uds: This SRV record is used in order to determine if a CUCM server is available.
_cuplogin: This SRV record is used in order to determine if an IM&P server is available.
_collab-edge: This SRV record is used in order to determine if MRA is available.
If the Jabber client is started and does not receive an SRV answer for _cisco-uds and _cuplogin and does receive an answer for _collab-edge, then it uses this answer to try to contact the Expressway-E listed in the SRV answer.
The _collab-edge SRV record should point to the Fully Qualified Domain Name (FQDN) of Expressway-E with port 8443. If the _collab-edge SRV is not created, or is not externally available, or if it is available, but port 8443 is not reachable, then the Jabber client fails to log in.
If port 8443 is not reachable, this could be due to a security device (Firewall) blocking the port or a misconfiguration of the Default Gateway (GW) or Static routes in the Exp-E.
2. Unacceptable or No Available Certificate on VCS Expressway
After the Jabber client has received an answer for _collab-edge, it then contacts Expressway with Transport Layer Security (TLS) over port 8443 to try to retrieve the certificate from Expressway to set up TLS for communication between the Jabber client and Expressway.
If Expressway does not have a valid signed certificate that contains either the FQDN or domain of Expressway, then this fails and the Jabber client fails to log in.
If this issue occurs, the customer should use the Certificate Signing Request (CSR) tool on Expressway, which automatically includes the FQDN of Expressway as a Subject Alternative Name (SAN).
Note: MRA requires secure communication between Expressway-C and Expressway-E, and between Expressway-E and external endpoints.
The next table with the Expressway certificate requirements by feature can be found in the MRA Deployment Guide:
3. No UDS Servers Found in Edge Configuration
After the Jabber client successfully establishes a secure connection with Expressway-E, it asks for its edge configuration (get_edge_config). This edge configuration contains the SRV records for _cuplogin and _cisco-uds. If the _cisco-uds SRV records are not returned in the edge configuration, then the Jabber client is not able to proceed with log in.
In order to fix this, make sure that _cisco-uds SRV records are created internally and resolvable by Expressway-C.
This is also a common symptom if you are in a dual domain. If you run in a dual domain and find the Jabber client is not being returned any User Data Service (UDS), you must confirm that the _cisco-uds SRV records are created in the internal DNS with the external domain.
4. Expressway-C Logs Show This Error: XCP_JABBERD Detail="Unable to connect to host '%IP%', port 7400:(111) Connection refused"
If Expressway-E Network Interface Controller (NIC) is incorrectly configured, this can cause the Extensible Communications Platform (XCP) server to not be updated. If Expressway-E meets these criteria, then you will probably encounter this issue:
Uses a single NIC.
Advanced Networking Option Key is installed.
The Use Dual Network Interfaces option is set to Yes.
In order to correct this problem, change the Use Dual Network Interfaces option to No.
The reason this is a problem is because Expressway-E listens for the XCP session on the wrong network interface, which causes the connection to fail/timeout. Expressway-E listens on TCP port 7400 for the XCP session. You can verify this if you use the netstatcommand from the VCS as root.
5. Expressway-E Server Hostname/Domain Name Does Not Match What is Configured in the _collab-edge SRV
If the Expressway-E Server hostname/domain in the DNS page configuration does not match what was received in the _collab-edge SRV answer, the Jabber client is not able to communicate with Expressway-E. The Jabber client uses the xmppEdgeServer/Address element in the get_edge_config response to establish the XMPP connection to Expressway-E.
This is an example of what the xmppEdgeServer/Address looks like in the get_edge_config response from Expressway-E to the Jabber client:
In order to avoid this, make sure that the _collab-edge SRV record matches the Expressway-E hostname/domain name. Cisco bug ID CSCuo83458 has been filed for this and partial support was added on Cisco bug ID CSCuo82526.
6. Unable to Log In Because of an Existing WebEx Connect Subscription
7. The Expressway-C Server Displays the Error Message: "Configured but with errors. Provisioning server: Waiting for traversal server info."
If you navigate to Status > Unified Communications and see the error message, "Configured but with errors. Provisioning server: Waiting for traversal server info." for Unified CM registrations and IM&P Service, the internal DNS server(s) configured on the Expressway-C have two DNS A records for the Expressway-E. The reason behind multiple DNS A records for the Expressway-E could be the affected user moved from single NIC with static NAT enabled on the Expressway-E to dual-NIC with static NAT enabled, or vice versa, and forgot to delete the appropriate DNS A record in the internal DNS server(s). Therefore, when you use the DNS lookup utility in the Expressway-C and resolve the Expressway-E FQDN, you will notice two DNS A records.
If the Expressway-E NIC is configured for a single NIC with static NAT:
Delete the DNS A record for the Expressway-E internal IP address in the DNS server(s) configured in the Expressway-C.
Flush the DNS cache in the Expressway-C and the user's PC via CMD (ipconfig /flushdns).
Reboot the Expressway-C server.
If the Expressway-E NIC is configured for dual NIC with static NAT enabled:
Delete the DNS A record for the Expressway-E external IP address in the DNS server(s) configured in the Expressway-C.
Flush the DNS cache in the Expressway-C and user's PC via CMD (ipconfig /flushdns).
Reboot the Expressway-C server.
8. Microsoft DirectAccess Installed
The customer might use Microsoft DirectAccess on the same PC as the Jabber Client. When you attempt to log in remotely, this can break MRA. DirectAccess will force DNS queries to be tunnelled in to the internal network as though the PC was using a VPN.
Note: Microsoft DirectAccess is not supported with Jabber over MRA. Any troubleshooting is best effort. Configuration of DirectAccess is the responsibility of the network administrator.
Some customers have had success by blocking all DNS records in the Microsoft DirectAccess Name Resolution Policy Table. These records should not be processed by DirectAccess (Jabber needs to be able to resolve these via public DNS when using MRA):
SRV record for _cisco-uds
SRV record for _cuplogin
SRV record for _collab-edge
A record for all Expressway Es
9. Expressway Reverse DNS Lookups Fails
Beginning in Version X8.8, Expressway/VCS requires forward and reverse DNS entries to be created for ExpE, ExpC, and all CUCM nodes.
Softphone is Not Able to Register, SIP/2.0 405 Method Not Allowed
A diagnostic log from Expressway-C shows a SIP/2.0 405 Method Not Allowed message in response to the Registration request sent by the Jabber client. This is likely due to an existing Session Initiation Protocol (SIP) trunk between Expressway-C and CUCM using port 5060/5061.
In order to correct this issue, change the SIP port on the SIP Trunk Security Profile that is applied to the existing SIP trunk configured in CUCM and the Expressway-C neighbor zone for CUCM to a different port such as 5065. This is explained further in this video. Here is a configuration summary:
Create a new SIP Trunk security profile with a listening port other than 5060 (5065).
Create a SIP Trunk associated to the SIP Trunk Security Profile and destination set to the Expressway-C IP address, port 5060.
Create a neighbor zone to CUCM(s) with a target port other than 5060 (5065) to match the CUCM configuration.
In Expressway-C Settings > Protocols > SIP, make sure Expressway-C still listens on 5060 for SIP.
Softphone is Not Able to Register, Reason="Unknown domain"
A diagnostic log from Expressway-C shows Event="Registration Rejected" Reason="Unknown domain" Service="SIP" Src-ip="XXX.XXX.XXX.XXX" Src-port="51601" Protocol="TCP" AOR="sip:XXX.XXX.XXX.XXX".
In order to correct this issue, check these points:
Does the Jabber client use a Secure Device Security Profile in CUCM when the intention is not to use a non-secure Device Security Profile?
If the Jabber clients should use a secured Device Security Profile, is the name of the security profile in FQDN format and is that FQDN name configured on the Expressway-C's certificate as a SAN?
If the Jabber clients should use a secured Device Security Profile, navigate to System > Enterprise Parameters > Security Parameters > Cluster Security Mode and check that the Cluster Security Mode is set to 1 in order to verify that the CUCM cluster has been secured. If the value is 0, the administrator must go through the documented procedure to secure the cluster.
Softphone is Not Able to Register, Reason "Idle countdown expired"
When you review the Expressway-E logs during the timeframe that the Jabber client sends in a REGISTER message, you might encounter an Idle countdown expired error as indicated in the code snippet here.
This snippet indicates that the firewall does have port 5061 open; however, there is no application-layer traffic that is passed over in a sufficent amount of time so the TCP connection closes.
If you encounter this situation, there is a high degree of probability that the firewall in front of Expressway-E has SIP Inspection/Application Layer Gateway (ALG) functionality turned on. In order to remediate this issue, you must diable this functionality. If you are unsure of how to do this, you should reference your firewall vendor's product documentation.
The packet capture from Jabber shows a SSL negotiation with the Expressway E IP; however the certificate sent does not come from this server:
The FW has Phone Proxy configured.
Confirm that the FW runs Phone Proxy. In order to check that, enter the show run policy-map command and it will show you something similar to:
class sec_sip inspect sip phone-proxy ASA-phone-proxy
Disable Phone Proxy for phone services to connect successfully.
No Media When You Call Through MRA
These are some of the missing and incorrect configurations that can cause this problem in Single and Dual NIC deployments:
Static NAT is not configured in the Expressway-E under System > Network Interfaces > IP. NAT at the network layer still needs to be done in the firewall, but this setting will translate the IP at the application layer.
More information on this can be found in Appendix 4 of the Cisco Expressway-E and Expressway-C Basic Configuration Deployment Guide.
No Ringback When Call Over MRA to PSTN
This issue is due to a limitation on Expressways prior to Version X8.5. Cisco bug ID CSCua72781 describes how Expressway-C does not forward early media in 183 Session Progress or 180 Ringing across the traversal zone. If you run Versions X8.1.x or X8.2.x, you can upgrade to Version X8.5 or alternatively perform the workaround listed here.
It is possible to use a workaround on the Cisco Unified Border Element (CUBE) if you make a SIP profile that turns the 183 into a 180 and applies it on the incoming dial-peer. For example:
Afterwards they would disable 180 Early Media on either the SIP profile of the CUCM > CUBE or the CUBE itself within sip-ua configuration mode.
CUCM and IM&P Issues
ASCII Error That Prevents CUCM From Being Added
When you add CUCM to Expressway-C, you encounter an ASCII error that prevents CUCM from being added.
When Expressway-C adds CUCM to its database, it runs through a series of AXL queries that relate to get and list functions. Examples of these include getCallManager, listCallManager, listProcessNode, listProcessNodeService, and getCCMVersion. After the getCallManager process is run, it is succeeded by an ExecuteSQLQuery set to retrieve all CUCM Call Manager-trust or tomcat-trusts.
Once CUCM receives the query and executes on it, CUCM then reports back all of its certificates. If one of the certificates contains a non-ASCII character, Expressway generates an error in the web interface similar to ascii codec can't decode byte 0xc3 in position 42487: ordinal not in range(128).
This issue is tracked with Cisco bug ID CSCuo54489 and is resolved in Version X8.2.
Outbound TLS Failures on 5061 from Expressway-C to CUCM in Secure Deployments
This issue occurs when you use self-signed certificates on CUCM and Tomcat.pem/CallManager.pem have the same subject. The issue is addressed with Cisco bug ID CSCun30200. The workaround to correct the issue is to delete the tomcat.pem and disable TLS Verify from the CUCM configuration on Expressway-C.
IM&P Server Not Added and Errors Encountered
When you add an IM&P Server, Expressway-C reports "This server is not an IM and Presence Server" or "Unable to communicate with .AXL query HTTP error ''HTTPError:500'", which results in the IM&P Server not being added.
As part of the addition of an IM&P server, Expressway-C uses an AXL query to look for the IM&P certificates in an explicit directory. Due to Cisco bug ID CSCul05131, the certificates are not in that store; therefore, you encounter the false error.
Voicemail Status on Jabber Client Shows “Not connected”
In order to make the Jabber client Voicemail status successfully connect, you must configure the Cisco Unity Connection IP address or hostname within the HTTP allow list on Expressway-C.
In order to complete this from Expressway-C, perform the relevant procedure:
Jabber Clients Are Prompted to Accept the Expressway-E Certificate During Login
This error message can be related with the Expressway Edge certificate not signed by a public CA that is trusted by the client's device or that the domain is missing as a SAN in the server certificate.
In order to stop the Jabber client from being prompted to accept the Expressway certificate, you must meet the two criteria listed below:
The device/machine that runs the Jabber client must have the signer of the Expressway-E certificate listed within its certificate trust store.
Note: This is easily accomplished if you use a public certificate authority because mobile devices contain a large certificate trust store.
The Unified CM registration domain used for the collab-edge record must be present within the SAN of the Expressway-E certificate. CSR tool in the Expressway server will give you the option to add the Unified CM registration domain as a SAN, it will be preloaded if the domain is configured for MRA. If the CA signing the certificate does not accept a domain as a SAN, you can also use the "CollabEdgeDNS" option, which will prefix "collab-edge" to the domain: