[WARN] tls handshake with 127.0.0.1:XXXXX failed: received corrupt message of type InvalidContentType

[rwjack]

Subject of the issue

Container shows as unhealthy on portainer, even though everything is working. Unsure why I'm getting these WARN logs, the curl healthcheck works within the container.

[2023-11-28 13:04:14.301][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:39154 failed: received corrupt message of type InvalidContentType
[2023-11-28 13:05:14.348][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:51004 failed: received corrupt message of type InvalidContentType
[2023-11-28 13:05:30.237][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:34490 failed: cannot decrypt peer's message
[2023-11-28 13:06:14.389][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:43980 failed: received corrupt message of type InvalidContentType
[2023-11-28 13:07:14.432][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:51626 failed: received corrupt message of type InvalidContentType

Deployment environment

• vaultwarden version:
  1.30.1

• Install method:
  Docker

• Clients used:
  Irellevant

• Reverse proxy and version:
  Irellevant

• Other relevant details:

version: "3"

volumes:
  data:

services:
  bitwarden:
    image: vaultwarden/server:latest
    restart: unless-stopped

    container_name: bitwarden
    hostname: bitwarden

    environment:
      - TZ=[redacted]

    ports:
      - "[redacted]:80"
      #- "[redacted]:3012" WSS disabled due to bw extension data leakage via GET request

    volumes:
      - data:/data/
      - ./certs/:/etc/ssl/custom/
      - ./.env:/.env:ro

Relevant .env changes:

DOMAIN=https://[redacted]

## Rocket specific settings
## See https://rocket.rs/v0.4/guide/configuration/ for more details.
# ROCKET_ADDRESS=0.0.0.0
# ROCKET_PORT=80  # Defaults to 80 in the Docker images, or 8000 otherwise.
# ROCKET_WORKERS=10
ROCKET_TLS={certs="/etc/ssl/custom/[redacted].pem",key="/etc/ssl/custom/[redacted]-key.pem"}

Steps to reproduce

Expected behaviour

Container to show as healthy

Actual behaviour

Container shows as unhealthy, even though everything is working.

Troubleshooting data

root@bitwarden:/# curl --insecure --fail --silent --show-error https://localhost:80/alive || exit 1
"2023-11-28T12:06:47.277011Z"root@bitwarden:/# echo $?
0

[BlackDex]

It seems to work fine for me.
Try setting the LOG_LEVEL to debug and see if you get some more information.
I tested it with curl versions, 7.68.0, 7.81.0, 7.88.1 and 8.4.0, they all worked fine.

Tested both the FQDN and localhost nothing breaks.

It could be your host platform which causes some (Open)SSL client settings to be different host-wide, and causes this to happen.
The certs might be unsupported in some way, or not have a full chain available?

Also, i see you use port 80, while it looks like you do a port-proxy via Docker. Not sure if you use a reverse proxy in-front of Vaultwarden, if so, it could also be your reverse proxy which is causes an issue.

I'm not able to reproduce this on my side.

[rwjack]

Try setting the LOG_LEVEL to debug and see if you get some more information.

Let me give it a shot.

---

It could be your host platform which causes some (Open)SSL client settings to be different host-wide, and causes this to happen.

Debian 11 with the latest version of Docker, I doubt that's the issue.

---

The certs might be unsupported in some way, or not have a full chain available?

I don't think they do have a full chain, it's just a regular certificate generated by mkcert

---

Also, i see you use port 80, while it looks like you do a port-proxy via Docker. Not sure if you use a reverse proxy in-front of Vaultwarden, if so, it could also be your reverse proxy which is causes an issue.

Port 80 is the default for the container. I expose another port on the Debian host, which is open only to the reverse proxy, but I don't see how that's related, as the healthcheck should curl "localhost", since ROCKET_ADDRESS (by default I hope) is configured to 0.0.0.0.

addr="${ROCKET_ADDRESS}"
if [ -z "${addr}" ] || [ "${addr}" = '0.0.0.0' ] || [ "${addr}" = '::' ]; then
    addr='localhost'
fi

[rwjack]

@BlackDex Though I didn't just try to run the healthcheck manually.

This is the error:

root@bitwarden:/# ./healthcheck.sh
curl: (1) Received HTTP/0.9 when not allowed

And the timestamp matches a corresponding error in the container logs:

[2023-12-01 12:41:08.865][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:34488 failed: received corrupt message of type InvalidContentType

Relevant issue: curl/curl#12183

[BlackDex]

Ah! Looks like ROCKET_TLS is not seen as a env variable to the healthcheck.sh script.
Probably because you are using a .env file there which isn't seen by the script.

That scenario isn't covered it looks like.

[rwjack]

Okay, definitely a minor bug, I fixed it by adding this in my docker-compose:

    env_file:
      - .env

Not sure why I didn't set it like that in the first place, that's how I do it everywhere. Anyways, all good now.

This is probably the reason:

## By default, Vaultwarden expects for this file to be named ".env" and located
## in the current working directory.

[BlackDex]

That is if not running via Docker. But that could be cleared up. Since there also is a ENV variable which can relocate the .env file location ;). Which doesn't make it more clear haha.

[rwjack]

Gotcha, thanks for the help!

Even though my issue is resolved, do you still want to keep this open as a reminder?

[BlackDex]

Yes please, this is something we need to fix i think.