How to set DOCKER_HOST on 4.13.0 without /var/run/docker.sock

[joshua-at-aws]

Expected behavior

• DOCKER_HOST working in Docker Desktop 4.13.0

Actual behavior

• Docker Desktop for Mac removed /var/run/docker.sock per 4.13.0 Release Notes: https://docs.docker.com/desktop/release-notes/#docker-desktop-4130 "By default Docker will not create the /var/run/docker.sock symlink on the host and use the docker-desktop CLI context instead."

• I use Docker Desktop along with other tools that require DOCKER_HOST. The docs say to set this to the /var/run/docker.sock https://docs.docker.com/desktop/faqs/general/#how-do-i-connect-to-the-remote-docker-engine-api

Is there a different setting for DOCKER_HOST that will work in 4.13.0, or a preference to create the socket?
I poked around a bit but did not find anything in the UI. Thanks in advance!

Information

• macOS Version: 12.6 Monterey
• Intel chip or Apple chip: Intel
• Docker Desktop Version: 4.13.0

Steps to reproduce the behavior

1. Install 4.13.0 on Mac
2. DOCKER_HOST not working

[nicks]

Thanks for reporting!

We're trying to make Docker Desktop more userspace-friendly (e.g., fewer root privileges). As part of that, we're migrating the sockets to user-owned directories rather than root-owned directories (like /var/run).

Everything should work correctly if you unset DOCKER_HOST and let the default socket-picking behavior kick in?

There's supposed to be some shim code that fixes this automatically, let me check in with some people and get back to you.

[joshua-at-aws]

Hi, thanks for the reply!

If I do unset DOCKER_HOST then the other tools, e.g. Maven or AWS SAM CLI do not work Error: Running AWS SAM projects locally requires Docker.

Could the new user-owned directory for the socket be added to the docs?

[nicks]

Can you post a diagnostic id and the output of docker context list?

I think it's currently at ~/.docker/run/docker.sock, though I've been told that may change again. we're trying to get away from people hard-coding it.

[joshua-at-aws]

• Diagnostics ID: 8E63E195-B162-440D-826A-09E31E28D3A6/20221021210358

docker context list 
NAME                TYPE                DESCRIPTION                               DOCKER ENDPOINT                                KUBERNETES ENDPOINT   ORCHESTRATOR
default *           moby                Current DOCKER_HOST based configuration   unix:///Users/joshuf/.docker/run/docker.sock                         swarm
desktop-linux       moby                                                          unix:///Users/joshuf/.docker/run/docker.sock     

Setting DOCKER_HOST to the default socket works! I see docker context list has a JSON output format, which I can use in a script if that's the new recommended way, for example with a Python one-liner:

docker context list --format json | python3 -c "import sys, json; j=json.load(sys.stdin); [print(x['DockerEndpoint']) if x['Current'] else None for x in j]"

unix:///Users/joshuf/.docker/run/docker.sock

[nicks]

Our current plan is to roll this change back for now and figure out a more orderly upgrade path with docs for client library authors.

[hopeseekr]

Here is the workaround for OSX:

sudo ln -s "$HOME/.docker/run/docker.sock" /var/run/docker.sock

[hopeseekr]

I'd like to add that many thirdparty utilities, particularly IntelliJ and PhpStorm, expect there to be a /var/run/docker.sock. This change broke a wide swath of our toolchain.

[johndutchover]

Here is the workaround for OSX:

sudo ln -s "$HOME/.docker/run/docker.sock" /var/run/docker.sock

@hopeseekr I needed to force -f
sudo ln -sf "$HOME/.docker/run/docker.sock" /var/run/docker.sock

[tbeauvais-imagine]

This wasted 4 hours of my life thinking it was a configuration problem that I had.

As no valid configuration was found, execution cannot continue
Root cause NoSuchFileException (/var/run/docker.sock)

I appreciate the intent of this change, yet the intent should have been captured in the release notes. Also, with such changes, you should have a workaround available BEFORE release to ensure people can continue as necessary.

Still though, updated the DOCKER_HOST doesn't work since a lot of 3rd parties rely on the location of the sock descriptor since it has been there for a decade. To change it all of a sudden is really a really bold and strange call.

Hopefully, people find this (though I didn't) and don't waste hours like I did.

[johndutchover]

This wasted 4 hours of my life thinking it was a configuration problem that I had.

As no valid configuration was found, execution cannot continue Root cause NoSuchFileException (/var/run/docker.sock)

I appreciate the intent of this change, yet the intent should have been captured in the release notes. Also, with such changes, you should have a workaround available BEFORE release to ensure people can continue as necessary.

Still though, updated the DOCKER_HOST doesn't work since a lot of 3rd parties rely on the location of the sock descriptor since it has been there for a decade. To change it all of a sudden is really a really bold and strange call.

Hopefully, people find this (though I didn't) and don't waste hours like I did.

updated post

Desktop 4.13.0 (89412) no longer hangs with "workaround", however I am not able to connect to Docker daemon . See steps taken below. Yes, previous release 4.12 does work on this setup.

My env:

• macOS Version: 13.0 Ventura (previously didn't work in 12.6 Monterey)
• Intel chip or Apple chip: Apple M1 Pro
• Docker Desktop Version: 4.13.0,89412

Steps taken (zsh)

• sudo ln -sf "$HOME/.docker/run/docker.sock" /var/run/docker.sock
• add export DOCKER_HOST=unix://"$HOME/.docker/run/docker.sock" (note only two forward slashes after colon) to .zprofile
• restart

Output after restart and login:
zsh ➜ docker version
Cannot connect to the Docker daemon at unix:///Users/username/.docker/run/docker.sock. Is the docker daemon running?
Client:
Cloud integration: v1.0.29
Version: 20.10.20
API version: 1.41
Go version: go1.18.7
Git commit: 9fdeb9c
Built: Tue Oct 18 18:20:35 2022
OS/Arch: darwin/arm64
Context: default
Experimental: true

unclear why "true" since set to false in $HOME/.docker/daemon.json

~
zsh ❯ docker context list
NAME TYPE DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR
default * moby Current DOCKER_HOST based configuration unix:///Users/username/.docker/run/docker.sock swarm
desktop-linux moby unix:///Users/username/.docker/run/docker.sock

[johndutchover]

I found an issue when tailing file DockerAppStderr.txt in ~/Library/Group\ Containers/group.com.docker/DockerAppStderr.txt after launching 4.13.0 (89412):

time="2022-10-28T12:59:28Z" level="info" msg="GoBackend: notifying bugsnag: setting overrides for proxies: computing HTTPIP of 192.168.65.0/28: prefix of 28 does not accommodate a host numbered 128

I interpreted this to mean that the default Docker subnet (/28) subnet is too small.

Steps taken (in addition to #6529 (comment))

1. Launch Docker Desktop 4.12.0 (85629)
2. Navigate: Preferences -> Resources -> Network and changed subnet to /24

3. Updated in-app to 4.13.0 (89412)

And it's working!

[chaizeg]

Closing this issue as a fix has been pushed with the new release of Docker Desktop 4.13.1. See release notes for more details.

[kiwifellows]

• export DOCKER_HOST=unix://"$HOME/.docker/run/docker.sock"

Amen is all I can say.... and when I read the "4 hours" yeap same here!! Almost
Updated my ~/.docker/init-zsh.sh with export DOCKER_HOST=unix://"$HOME/.docker/run/docker.sock"
Works

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

/lifecycle locked

[niklasschaeffer]

Thanks for reporting!

We're trying to make Docker Desktop more userspace-friendly (e.g., fewer root privileges). As part of that, we're migrating the sockets to user-owned directories rather than root-owned directories (like /var/run).

Everything should work correctly if you unset DOCKER_HOST and let the default socket-picking behavior kick in?

There's supposed to be some shim code that fixes this automatically, let me check in with some people and get back to you.

If someone find this in 2023

Hi, Docker-Desktop for Linux User here.
You can't just remove that without any backwards-compatibility flag or fix. PPL depend on /var/run/docker.sock in different scenarios. And literally running a link to this file after every restart is not the way. At least not for me.

this was the fix for me but literally took me hours to fiddle out.

/usr/lib/systemd/user/docker-desktop.service

[Service]
ExecStart=/opt/docker-desktop/bin/com.docker.backend && sudo ln -sf "/home/youruser/.docker/desktop/docker.sock" "/var/run/docker.sock" && sudo chmod 666 /var/run/docker.sock

[tsal]

This has returned in Docker 4.17.0; please provide clear documentation about the environment variable. Using sudo to create a symlink to a home dir for a user-land socket is not a safe option.

[fharper]

I'm also having this issue on a new laptop with Docker Desktop 4.18.0 (104112). My workaround was to create a symlink sudo ln -s "$HOME/.docker/run/docker.sock" /var/run/docker.sock

[Darkslide77]

Issue is still occurring, @fharper solution worked for me.

[niklasschaeffer]

@Darkslide77 @fharper

I literally found another working solution to this and this was by simply setting the DOCKER_HOST variable like so.

export DOCKER_HOST="unix://$HOME/.docker/desktop/docker.sock" of for your case probably
export DOCKER_HOST="unix://$HOME/.docker/run/docker.sock"

[fharper]

@niklasschaeffer: nice, but it assume that apps connecting to the socket don't use default path, but use this environment variable no?

[niklasschaeffer]

@fharper As long as they respect the DOCKER_HOST Enviroment variable i think you won't have any problems at all

[fharper]

@niklasschaeffer yeah, that's why I prefer to symlink. I would assume most people use the default path as it shouldn't NOT be created by Docker, instead of checking an environment variable that needs to be set by the user. Also, in my case, the app didn't check it :)

[adler-sudo]

This solves the issue without the need for root access

export DOCKER_HOST=unix://"$HOME/.docker/run/docker.sock"

[ataylormays-duetto]

I'm having this issue using docker desktop version 4.22.1. Running sudo ln -s "$HOME/.docker/run/docker.sock" /var/run/docker.sock fixes the issue, but as mentioned above this is not a great long-term solution, nor one that should require digging through closed issues to find a resolution for.

[nicks]

Here's a good doc that explains sockets on Docker Desktop on Mac, and why we're moving away from the root-owned /var/run/docker.sock:
https://docs.docker.com/desktop/mac/permission-requirements/