Nginx to DMS fails with SSL failed error:0A00010B:SSL routines::wrong version number

[UhCyR9]

hi, help I've been sitting on this quite a bit and have no idea where the error lies

I have nginx and mail server in my docker container and now I want to accept the connection to nginx and redirect to mail, but I get an error about the wrong version of SSL and I do not know what I should do next

Here are my configs

Nginx

server {
  listen 465 ssl;

  # SSL
  ssl_certificate /etc/nginx/server.crt;
  ssl_certificate_key /etc/nginx/server.key;
  ssl_protocols TLSv1.2 TLSv1.3;

  proxy_pass mailserver:465;
  proxy_ssl_protocols TLSv1.2 TLSv1.3;
}
server {
  listen 993 ssl;

  # SSL
  ssl_certificate /etc/nginx/server.crt;
  ssl_certificate_key /etc/nginx/server.key;

  proxy_pass mailserver:993;
  proxy_ssl_protocols TLSv1.2 TLSv1.3;
}

and i got

mail dovecot: imap-login: Disconnected: Connection closed: SSL_accept() failed: error:0A00010B:SSL routines::wrong version number (no auth attempts in 0 secs): user=<>, rip=192.168.32.2, lip=192.168.32.3, TLS handshaking: SSL_accept() failed: error:0A00010B:SSL routines::wrong version number,

[polarathene]

I am not that familiar with nginx configuration, but can you confirm you understand this?:

• TLS cannot be terminated by a reverse proxy, there is no equivalent HTTP vs HTTPS ports here for mail protocols.
• You must proxy layer 7 TCP traffic directly to DMS ports, you will lose the original client IP due to the proxy layer and thus you should also configure PROXY protocol (v2). PROXY protocol will keep the IP correct, it requires configuration on both reverse proxy and DMS (see our docs with Traefik example). Not doing this will allow abuse with Fail2Ban (if enabled), as well as potentially allowing for open relay depending on your config.

It is usually better to just expose the ports directly, without reverse proxy due to the above. It's rare that these ports are shared by any other mail server, unlike with web services where you need to route the common ports 80 and 443 to the different services HTTP port.

The only other reason you'd have then is for certificate management. Nginx doesn't provision certificates like Caddy and Traefik, unless you're using something like nginx-proxy, but again we provide docs for that sort of integration.

Just to clarify, DMS has two types of TLS:

• Implicit TLS (ports 465, 995, 993), much like HTTPS. Since there is no unencrypted alternative like HTTP, this should be terminated by DMS.
• Explicit TLS (ports 25, 587, 110, 143), these rely on STARTTLS. It is rare that there is proxy support for this protocol. Nginx does have it to some extent AFAIK, but it might be via custom module / config. TLS is delayed via STARTTLS, the client must first start with unencrypted connection and then negotiate to upgrade to TLS.

---

I assume from the config snippet you provided, that nginx is managing the cert and terminating TLS, after the connection is forwarded to DMS ports that expect TLS. That is why you get an error about failed SSL.

[UhCyR9]

First point, thank you very much for your answer.

I understand, however, I have an application with UI, to create configs for deploy to the server, to automatically refresh and create certificates from an external provider, etc. and the application is just based on nginx. That's why I wanted to create such a scheme, that nginx accepts requests which it redirects to DMS and DMS gets manually certificates which I created.

So I have two questions if I understood your answer correctly:

1. I understand that nginx terminates the TLS connection and the connection to DMS is no longer encrypted and hence this error?
2. Do the ports that use STARTTLS need to be exposed or is it enough if I expose those ports with Implicit TLS?

Is what I want to achieve even possible?

[polarathene]

1. I understand that nginx terminates the TLS connection and the connection to DMS is no longer encrypted and hence this error?

Correct:

• STARTTLS ports => nginx would fail by default as this is not TLS to begin with, it must understand STARTTLS protocol to upgrade to TLS.
• Regardless when TLS is terminated at nginx, DMS is expecting to terminate TLS with STARTTLS or implicit TLS itself, there is no unencrypted option except for port 25.

If you do proxy to DMS, it should be at layer 7 only, and as mentioned you would need to configure PROXY protocol. This is more effort than it is worth for you, you shouldn't need to proxy connections from nginx to DMS.

---

2. Do the ports that use STARTTLS need to be exposed or is it enough if I expose those ports with Implicit TLS?

You can use implicit TLS for mail submission (SMTP over port 465), mail retrieval (IMAP over port 993, you likely don't need port 995 for POP3).

However if you expect to have mail delivered to DMS (you have DNS MX record that points to DMS), you will need port 25. Port 25 is only STARTTLS and it is expected to allow accepting mail without TLS.

Since nginx doesn't understand / support STARTTLS out of the box, if you attempt to proxy as implicit TLS it will fail to work correctly, no mail server or client will attempt to use implicit TLS with port 25, so if you enforce TLS this way through nginx all delivery to DMS would fail.

---

Is what I want to achieve even possible?

I'll answer from the two different statements you provided:

I have an application with UI, to create configs for deploy to the server, to automatically refresh and create certificates from an external provider, etc. and the application is just based on nginx.

You can provision certificates externally, but if are not like the LetsEncrypt structure that CertBot uses, you will want to avoid SSL_TYPE=letsencrypt and instead choose SSL_TYPE=manual. See the linked docs, there are additional ENV required to configure the path to public cert and private key.

When the file is updated, DMS should detect it (presently this will not happen with ACCOUNT_PROVISIONER=LDAP), and copy the updated files internally for Postfix and Dovecot to reload with.

That's why I wanted to create such a scheme, that nginx accepts requests which it redirects to DMS and DMS gets manually certificates which I created.

This does not make much sense. There is no need for nginx to manage DMS traffic like this.

You can have nginx (or any other service) manage the certificates, but DMS expects to terminate TLS.

[UhCyR9]

Thanks a lot, in that case I think I'll give up nginx and just expose those ports straight from DMS.

Also I want to use LDAP for user accounts. Is there any way to refresh these certificates in DMS when LDAP is enabled, without doing docker compose down && docker compose up?

[polarathene]

Is there any way to refresh these certificates in DMS when LDAP is enabled, without doing docker compose down && docker compose up?

You would need to have some script triggered like check-for-changes.sh does. You might be able to just patch that to remove the LDAP restriction, it's been a while since I checked if it would cause any potential issues, removing the restriction has been on my TODO backlog.

Alternatively, you might be able to just use postfix reload and dovecot reload commands. IIRC, SSL_TYPE=letsencrypt has both Postfix and Dovecot directly configured to the certificate files, it might only be the other types like SSL_TYPE=manual that presently make an internal copy, so if you use manual you just need to ensure the updated certs are at the expected internal path and then reload the services.