k. rodman mannix

Fun with SMTP and TLS

2020-05-22

A few weeks ago, I went down a rabbit hole about SMTP and SSL and learned more than I thought I would need to know about SMTP. I have definitely taken for granted how abstracted email delivery is in most of the applications I’ve worked on. Email, and message delivery in general, is such common functionality that it’s easy to forget how incredibly complex it is to deal with legacy systems, unstable networks, and various formats and protocols.

It all began with wanting to guarantee that emails were delivered over SSL/TLS for a specific project¹ (by the way, I’ll just use TLS to refer to secure communication for the rest of this post). At first glance, this seems like it should be a minor configuration change. Unfortunately, enforcing TLS over SMTP is much less straightforward than, say, enforcing TLS over HTTP.

There are two main ways to use SMTP over TLS, which I briefly go into below.

Explicit/Opportunistic TLS:

By default, most SMTP communication that hopes to use a secure connection (read: TLS) uses STARTTLS². During the handshake to establish communication between the client and the server, the client can issue the STARTTLS command to the server to request that the connection be upgraded to TLS. This is called Explicit TLS, and is also sometimes referred to as Opportunistic TLS.

Using STARTTLS does not require TLS. If the server supports TLS, great - the connection will switch to TLS. If the server does not support TLS, the communication can still proceed over plaintext. Thus, using TLS is not required with Explicit TLS - even if a client requests it, TLS may not be available and the upgrade to TLS may not happen.

The reason STARTTLS exists (as opposed to just forcing communication over TLS) is to support secure communication while also being flexible for older mail clients that may not support TLS. It allows a client to prefer secure communication but still work unabated if the server does not support it. Explicit TLS allows a single port to serve both plaintext communication and communication over TLS.

A weakness of Explicit TLS is that if a malicious agent is in control of the network, the agent can modify the server messages to prevent the upgrade of the connection to TLS. This type of man-in-the-middle attack has been coined as a STRIPTLS attack³.

If you want to guarantee the use of TLS, this isn’t great: it prefers and attempts TLS but doesn’t require it.

Implicit/Forced TLS

Some SMTP servers have open ports that expect communication over TLS. Different than Explicit TLS, communication with these ports won’t do the STARTTLS dance and will assume communication over TLS. Because of this, it’s less flexible, but allows a client to be more sure about how communication is proceeding. This allows one to require TLS, but it’s not a standard so SMTP servers aren’t guaranteed to support it.

It seems that Implicit TLS does not play well with Explicit TLS, i.e. one cannot just connect to an SMTP server set up for Explicit TLS with Implicit TLS, because the SMTP server is expecting to first handshake in plaintext and then use STARTTLS (here’s a not-so-well written StackOverflow answer, but I thought it explained it kind of well).

Implicit TLS is weird because there isn’t ubiquitous acceptance or a current RFC supporting it - STARTTLS is sort of the current standard. So SMTP servers that implement a port that works with Implicit TLS isn’t necessarily a standard, although it is standardized (does that make sense?). The standardized port to set up TLS-only SMTP is 465 (some history here).

Wrap Up

Luckily, the email delivery provider I was using had a port available for Implicit TLS. I was able to update the SMTP configs to connect over that port and force SSL connections.

P.S.: Do you need to force Implicit SSL in a Ruby on Rails project? The SSL/TLS options in the mail gem (used by ActionMailer) appear to force Implicit TLS. If you’d like to set a specific version of TLS to use for SMTP connections, you can monkeypath Net::SMTP.default_ssl_context, which is used by which is used by the mail gem when making a connection (Inspired by this StackOverflow question).