This article describes how to identify TLS negotiation failure when STARTTLS is available within the EHLO SMTP commands and the server not conforming to rfc1869.
TLS is enabled on Email Security Appliance (ESA) with a valid certificate.
TLS is enabled on the destination server and STARTTLS is seen when an SMTP connection is established.
ESA tries to connect to destination server using TLS, however TLS negotiation fails with following error on the ESA's mail_logs/Message Tracking.
Info: DCID xxxxxx STARTTLS command not supported.
As per RFC rfc1869, the first response to EHLO should be ehlo-ok-rsp and ehlo-ok-rsp has following syntax and order:
ehlo-ok-rsp ::= "250" domain [ SP greeting ] CR LF
/ ( "250-" domain [ SP greeting ] CR LF
*( "250-" ehlo-line CR LF )
"250" SP ehlo-line CR LF )
Incorrect RFC Syntax SMTP Conversation example
220 mail.domain1.com ESMTP Service ready
EHLO ESA.com
250-STARTTLS <--- 250-STARTTLS is before the server greeting.
250-mail.domain1.com <--- This is the 250 destination server greeting.
250-8BITMIME
250-PIPELINING
250-HELP
250-DELIVERBY 300
250 SIZE 30000000
Which means everything before ehlo-line (250-mail.domain1.com in this example) is considered as greeting, so the ESA will not consider 250-STARTTLS command available and report STARTTLS command not supported. Refer https://tools.ietf.org/html/rfc1869 for more details.
Correct RFC Syntax SMTP Conversation example
220 mail-esa.com ESMTP
EHLO connecting.server.com
250-mail-esa.com <--- This is the 250 destination server greeting.
250-8BITMIME
250-SIZE 33554432
250 STARTTLS <--- STARTTLS is available after the greeting, it's not considered a greeting as per RFC.