Allow to automatically set podman connection for other WSL2 instances

[gbraad]

For WSL2 a new instance is created and a connection is added for interaction from the host using the podman-remote client, however people might like to use podman from other WSL installed distros on the same machine. For this, a new connection would need to be added to those instances. Enabling this in an easy way using a command?

Connections are composed as follows:

podman-machine-default       ssh://user@localhost:59971/run/user/1000/podman/podman.sock  C:\Users\gbraad\.ssh\podman-machine-default  false
podman-machine-default-root  ssh://root@localhost:59971/run/podman/podman.sock            C:\Users\gbraad\.ssh\podman-machine-default  false

the port, eg 59971 in this example is randomly generated.

ref: #15107

[Luap99]

How can this work? There are no ssh keys setup, we have no way of knowing how to start the podman system service (systemd is not available usually).

If users need this they can always manually create the connection.

[gbraad]

there are ssh keys set up, and would have to be copied to the WSL distro.

graph TD;
    WSL-Ubuntu-->WSL-Podman;

Loading

This is something Docker also allows to do.

@n1hility WDYS?

[kyriakos]

I am not sure how docker-desktop sets tis up behind the scenes but I can access docker commands from WSL2 existing distributions. Getting this to work with podman would be amazing since it means we'll have transparent switch from docker to podman without any workflow changes.

[n1hility]

We could theoretically add a podman machine command to provision a client config on a WSL instance. For example, something like (On Windows prompt):

podman machine install-wsl-client user@Ubuntu

This would also work with alias docker=podman

There are two challenges though:

1. Everyone else's (Ubuntu etc) package of podman is lagging (in particular the version on WSL). It's not fully compatible talking to 4.2. We probably would need to uninstall distro packages and install our own client.
2. To support native docker clients and docker API usage, we would need to install an ssh proxy to forward docker.sock (e.g. our win-sshproxy compiled to static linux) and launch it from, perhaps, a .bashrc. Win 11 added a single boot value but that also has the chance to conflict.

[n1hility]

Actually it might also be possible to share a mounted VHD and symlink docker.sock there instead of an ssh forward.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

Any update on this?

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[mancaus]

FYI, this, along with #15338 look like barriers to using devcontainers hosted in \\wsl$ from VS Code.

VS Code executes docker commands within WSL itself when they are contained within the wsl filesystem. This works in Docker Desktop, which makes docker.sock available in installed WSL distributions.

The other option is to configure VS Code to run the docker commands in the podman distribution. But this does not work due to #15338.

We don't typically use devcontainers hosted within the Windows filesystem, for performance and compatibility reasons.

An install-wsl-client command would be ideal for helping with this.

[gbraad]

A similar question was opened as a discussion: #16130
/cc: @n1hility @benoitf

[gbraad]

@n1hility This might be an option we wanna enable WSL wide or targeted per distribution? The issue is that we also need to 'clean' this after the stop of podman machine.

[n1hility]

Closed dup which contained the following:

Currently, if you want to do this, manually creating the bind is the recommended approach.

For example, for rootless:

touch /mnt/wsl/podman.sock
sudo mount --bind /var/run/user/1000/podman/podman.sock /mnt/wsl/podman.sock
For rootful:

touch /mnt/wsl/podman.sock
sudo mount --bind /var/run/podman/podman.sock /mnt/wsl/podman.sock
In the future, podman machine start can help by automatically binding into /mnt/wsl in a way that is compatbile with multiple machines, like so:

/mnt/wsl/podman/podman-machine-default-rootless.sock
/mnt/wsl/podman/podman-machine-default-rootful.sock
/mnt/wsl/podman/other-rootless.sock
/mnt/wsl/podman/other-rootful.sock

[gbraad]

According to the instructions from @n1hility, this can be done as follows:

PS> podman machine start
PS> podman machine ssh
$ touch /mnt/wsl/podman.sock
$ sudo mount --bind /var/run/user/1000/podman/podman.sock /mnt/wsl/podman.sock
$ exit

After this, you can use the following to use the remote Podman machine from your 'default' WSL:

PS> wsl
$ podman --remote --url unix:///mnt/wsl/podman.sock ps -a

[frenchesco]

@gbraad Thanks for these instructions. Is there a way to permanently set --remote --url unix:///mnt/wsl/podman.sock without having to specify it on every command?

[porjo]

@frenchesco you can set CONTAINER_HOST environment variable to point to the socket e.g. put this in ~/.bashrc:

export CONTAINER_HOST=unix:///mnt/wsl/podman.sock

[n1hility]

You can also do podman system connection add

(If there is more than one you can set the default with podman system connection default). Then you could edit your system containers conf under the engine section and set remote=true.

[gbraad]

Otherwise an alias could work if you use multiple podman instances at the same time.

The parameters for system connection add are in one of my previous comments.

[crruizb]

How does this solution work with Test Containers?
Do I need to export DOCKER_HOST to unix:///mnt/wsl/podman.sock ?
And what should be the value of TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE and TESTCONTAINERS_HOST_OVERRIDE ?

This env vars are coming from: https://java.testcontainers.org/features/configuration/ at the last section.

[frenchesco]

Thanks all! I have it mostly working except when I specify volumes in WSL via something like -v /home/marc/data:/data I get an error:
Error: statfs /home/marc/data: no such file or directory

How can I map folders from my WSL home directory to podman remote?

[crruizb]

I also have same error as @frenchesco and related to my question with testcontainers I always get:

Status 500: {"cause":"permission denied","message":"make cli opts(): making volume mountpoint for volume /home/crruiz/development/project/kotlin/repo/services/backend/testresources/db/init: mkdir /home/crruiz: permission denied","response":500}

And also when trying to run docker-compose:

Error response from daemon: make cli opts(): making volume mountpoint for volume /home/crruiz: mkdir /home/crruiz: permission denied

So looks like all these could be related