No connection could be made because the target machine actively refused it #19554

bwgjoseph · 2023-02-18T05:54:57Z

Bug description

note: realize this could be the same issue as podman-desktop/podman-desktop#1173, but I have tried to provide more information and am not sure if it's really the same. Ok to close as duplicate if deem so.

I have saved an image from the internet machine and bring into an air-gapped machine to load it in.

I encounter the following error

Error: failed to connect: dial tcp [::1]:49675: connectex: No connection could be made because the target machine actively refused it.

Operating system

Windows 10

Version

0.12.0

Steps to reproduce

// save image on internet machine
docker save busybox:latest > busybox.tar

// load image on air-gapped machine
podman load < busybox.tar

Docker was installed but not running. I did start Docker afterwards, but the same error occur

Initially, I was thinking if this has something to do with DOCKER_HOST, so I followed the guide but when I ran the command

podman machine inspect

The output

[
     {
          "ConfigPath": {
               "Path": "C:\\Users\\bwgjoseph\\.config\\containers\\podman\\machine\\wsl\\podman-machine-default.json"
          },
          "ConnectionInfo": {
               "PodmanSocket": null
          },
          "Created": "2023-02-18T11:25:40.0285425-08:00",
          "Image": {
               "IgnitionFilePath": {
                    "Path": ""
               },
               "ImageStream": "custom",
               "ImagePath": {
                    "Path": "C:\\Users\\bwgjoseph\\.local\\share\\containers\\podman\\machine\\wsl\\podman-image-x64.tar.xz"
               }
          },
          "LastUp": "2023-02-18T13:43:18.4028289-08:00",
          "Name": "podman-machine-default",
          "Resources": {
               "CPUs": 2,
               "DiskSize": 550502400,
               "Memory": 325951488
          },
          "SSHConfig": {
               "IdentityPath": "C:\\Users\\bwgjoseph\\.ssh\\podman-machine-default",
               "Port": 49675,
               "RemoteUsername": "user"
          },
          "State": "running"
     }
]

Relevant log output

> podman system connection list


Name                         URI                                                          Identity                                        Default
podman-machine-default       ssh://user@localhost:49675/run/user/1000/podman/podman.sock  C:\Users\bwgjoseph\.ssh\podman-machine-default  true
podman-machine-default-root  ssh://root@localhost:49675/run/podman/podman.sock            C:\Users\bwgjoseph\.ssh\podman-machine-default  false

podman machine info

Host:
  Arch: amd64
  CurrentMachine: podman-machine-default
  DefaultMachine: podman-machine-default
  EventsDir: C:\Users\bwgjoseph\.local\share\containers\podman\podman
  MachineConfigDir: C:\Users\bwgjoseph\.config\containers\podman\machine\wsl
  MachineImageDir: C:\Users\bwgjoseph\.local\share\containers\podman\machine\wsl
  MachineState: Running
  NumberOfMachines: 1
  OS: windows
  VMType: wsl
Version:
  APIVersion: 4.4.1
  Built: 1675890486
  BuiltTime: Wed Feb  8 13:08:06 2023
  GitCommit: 34e8f3933242f2e566bbbbf343cf69b7d506c1cf
  GoVersion: go1.19.5
  Os: windows
  OsArch: windows/amd64
  Version: 4.4.1

Additional context

It looks a little related to
I also tried rebooted my machine. This is running on a Virtual Machine if this helps

The text was updated successfully, but these errors were encountered:

benoitf · 2023-02-21T06:54:18Z

cc @lstocchi @jeffmaury do you have faced this issue on Windows ?

@bwgjoseph are you using a firewall or connected to a VPN ?

jeffmaury · 2023-02-21T11:13:25Z

cc @lstocchi @jeffmaury do you have faced this issue on Windows ?

@bwgjoseph are you using a firewall or connected to a VPN ?

Yes generally happens when podman machine state and wsl are not in sync.
Step to reproduce:

podman machine start
podman version ==> OK
wsl --shutdown
podman version ==> error message

benoitf · 2023-02-21T11:29:23Z

and podman machine inspect command ? It reports running I guess

bwgjoseph · 2023-02-21T12:57:16Z

cc @lstocchi @jeffmaury do you have faced this issue on Windows ?

@bwgjoseph are you using a firewall or connected to a VPN ?

Don't think there's firewall but I can double check. No VPN.

bwgjoseph · 2023-02-21T12:58:40Z

cc @lstocchi @jeffmaury do you have faced this issue on Windows ?

@bwgjoseph are you using a firewall or connected to a VPN ?

Yes generally happens when podman machine state and wsl are not in sync.
Step to reproduce:

podman machine start

podman version ==> OK

wsl --shutdown

podman version ==> error message

Not sure what does not in sync means. Is there something I can run to check?

I'm pretty sure my wsl is running when I'm testing on podman.

Anything else for me to check?

jeffmaury · 2023-02-21T15:36:22Z

After I run wsl --shutdown, podman machine inspect reports:

$ podman machine inspect
[
     {
          "ConfigPath": {
               "Path": "C:\\Users\\Jeff\\.config\\containers\\podman\\machine\\wsl\\podman-machine-default.json"
          },
          "ConnectionInfo": {
               "PodmanSocket": null
          },
          "Created": "2023-02-17T08:16:04.8648643+01:00",
          "Image": {
               "IgnitionFilePath": {
                    "Path": ""
               },
               "ImageStream": "35",
               "ImagePath": {
                    "Path": "C:\\Users\\Jeff\\.local\\share\\containers\\podman\\machine\\wsl\\podman-machine-default_fedora-podman-amd64-v36.0.127.tar"
               }
          },
          "LastUp": "2023-02-21T16:25:26.035611+01:00",
          "Name": "podman-machine-default",
          "Resources": {
               "CPUs": 0,
               "DiskSize": 3591372800,
               "Memory": 0
          },
          "SSHConfig": {
               "IdentityPath": "C:\\Users\\Jeff\\.ssh\\podman-machine-default",
               "Port": 54249,
               "RemoteUsername": "user"
          },
          "State": "stopped"
     }
]

jeffmaury · 2023-02-21T15:38:16Z

cc @lstocchi @jeffmaury do you have faced this issue on Windows ?
@bwgjoseph are you using a firewall or connected to a VPN ?

Yes generally happens when podman machine state and wsl are not in sync.
Step to reproduce:

podman machine start

podman version ==> OK

wsl --shutdown

podman version ==> error message

Not sure what does not in sync means. Is there something I can run to check?

I'm pretty sure my wsl is running when I'm testing on podman.

Anything else for me to check?

Yes when you're in this start, try to enter the wsl machine by running wsl if you have a single wsl machine. But as that may restart the wsl machine you need to check if it correctly started so run ps -ef to get the list of processed and you should see at least sshd processes.

bwgjoseph · 2023-02-21T16:46:54Z

cc @lstocchi @jeffmaury do you have faced this issue on Windows ?
@bwgjoseph are you using a firewall or connected to a VPN ?

Yes generally happens when podman machine state and wsl are not in sync.
Step to reproduce:

podman machine start

podman version ==> OK

wsl --shutdown

podman version ==> error message

Not sure what does not in sync means. Is there something I can run to check?
I'm pretty sure my wsl is running when I'm testing on podman.
Anything else for me to check?

Yes when you're in this start, try to enter the wsl machine by running wsl if you have a single wsl machine. But as that may restart the wsl machine you need to check if it correctly started so run ps -ef to get the list of processed and you should see at least sshd processes.

This is windows machine, so there isn't ps -ef command

Anyway, my wsl is alive, but the connection is still null

benoitf · 2023-02-21T19:56:50Z

could you list all wsl instances ? AFAIK podman brings a fedora wsl instance and here we're seeing Ubuntu so I guess there is something that happened

bwgjoseph · 2023-02-22T13:40:21Z

I don't see any fedora instance

I have ubuntu installed previously

jeffmaury · 2023-02-22T13:42:28Z

I don't see any fedora instance

I have ubuntu installed previously

the podman-machine-default should be fedora based

jeffmaury · 2023-02-22T13:52:26Z

cc @lstocchi @jeffmaury do you have faced this issue on Windows ?
@bwgjoseph are you using a firewall or connected to a VPN ?

Yes generally happens when podman machine state and wsl are not in sync.
Step to reproduce:

podman machine start

podman version ==> OK

wsl --shutdown

podman version ==> error message

Not sure what does not in sync means. Is there something I can run to check?
I'm pretty sure my wsl is running when I'm testing on podman.
Anything else for me to check?

Yes when you're in this start, try to enter the wsl machine by running wsl if you have a single wsl machine. But as that may restart the wsl machine you need to check if it correctly started so run ps -ef to get the list of processed and you should see at least sshd processes.

This is windows machine, so there isn't ps -ef command

Anyway, my wsl is alive, but the connection is still null

The ps -ef is to run inside the wsl machine. So list of instructions is:

1. wsl -d podman-machine-default
2. ps -ef

bwgjoseph · 2023-02-22T14:05:58Z

Alright. It is now the default. I rebooted my machine after I set podman-machine-default as the default wsl instance

Not sure if relevant but podman machine inspect still return null

This is the output for ps -ef in podman-machine-default

[user@DESKTOP-8A256M6 ~]$ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 22:02 ?        00:00:00 /lib/systemd/systemd
root        18     1  0 22:02 ?        00:00:00 /usr/lib/systemd/systemd-journald
root        25     1  0 22:02 ?        00:00:00 sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups
dbus        28     1  0 22:02 ?        00:00:00 /usr/bin/dbus-broker-launch --scope system --audit
root        29     1  0 22:02 ?        00:00:00 /usr/lib/systemd/systemd-userdbd
root        30    29  0 22:02 ?        00:00:00 systemd-userwork
root        31    29  0 22:02 ?        00:00:00 systemd-userwork
root        32    29  0 22:02 ?        00:00:00 systemd-userwork
dbus        33    28  0 22:02 ?        00:00:00 dbus-broker --log 4 --controller 9 --machine-id 43c24020ff5e49838ebd4554
user        35     1  0 22:02 ?        00:00:00 /usr/lib/systemd/systemd --user
user        37    35  0 22:02 ?        00:00:00 (sd-pam)
user        44    35  0 22:02 ?        00:00:00 /usr/bin/sleep infinity
root        45     1  2 22:04 ?        00:00:00 /usr/lib/systemd/systemd-hostnamed
root        46     0  0 22:04 pts/0    00:00:00 su -l user
root        51     1  1 22:04 ?        00:00:00 /usr/lib/systemd/systemd-logind
user        52    46  0 22:04 pts/0    00:00:00 -bash
user        70    52  0 22:04 pts/0    00:00:00 ps -ef

jeffmaury · 2023-02-22T14:10:28Z

Alright. It is now the default. I rebooted my machine after I set podman-machine-default as the default wsl instance

Not sure if relevant but podman machine inspect still return null

This is the output for ps -ef in podman-machine-default

[user@DESKTOP-8A256M6 ~]$ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 22:02 ?        00:00:00 /lib/systemd/systemd
root        18     1  0 22:02 ?        00:00:00 /usr/lib/systemd/systemd-journald
root        25     1  0 22:02 ?        00:00:00 sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups
dbus        28     1  0 22:02 ?        00:00:00 /usr/bin/dbus-broker-launch --scope system --audit
root        29     1  0 22:02 ?        00:00:00 /usr/lib/systemd/systemd-userdbd
root        30    29  0 22:02 ?        00:00:00 systemd-userwork
root        31    29  0 22:02 ?        00:00:00 systemd-userwork
root        32    29  0 22:02 ?        00:00:00 systemd-userwork
dbus        33    28  0 22:02 ?        00:00:00 dbus-broker --log 4 --controller 9 --machine-id 43c24020ff5e49838ebd4554
user        35     1  0 22:02 ?        00:00:00 /usr/lib/systemd/systemd --user
user        37    35  0 22:02 ?        00:00:00 (sd-pam)
user        44    35  0 22:02 ?        00:00:00 /usr/bin/sleep infinity
root        45     1  2 22:04 ?        00:00:00 /usr/lib/systemd/systemd-hostnamed
root        46     0  0 22:04 pts/0    00:00:00 su -l user
root        51     1  1 22:04 ?        00:00:00 /usr/lib/systemd/systemd-logind
user        52    46  0 22:04 pts/0    00:00:00 -bash
user        70    52  0 22:04 pts/0    00:00:00 ps -ef

Seems your WSL instance is fine now. What's about podman ps ?

bwgjoseph · 2023-02-22T14:11:33Z

Forgot about that. It's the same error though.

jeffmaury · 2023-02-22T14:34:15Z

Forgot about that. It's the same error though.

Seems the SSH daemon is not initialized correctly. Can you run journalctl --system --unit sshd in the WSL machine ?

bwgjoseph · 2023-02-22T14:38:58Z

Here's the log. I tried to run podman machine inspect, and restarting podman but it's still the same issue.

Feb 18 11:34:38 DESKTOP-8A256M6 systemd[1]: Starting sshd.service - OpenSSH server daemon...
Feb 18 11:34:38 DESKTOP-8A256M6 sshd[65]: Server listening on 0.0.0.0 port 49675.
Feb 18 11:34:38 DESKTOP-8A256M6 sshd[65]: Server listening on :: port 49675.
Feb 18 11:34:38 DESKTOP-8A256M6 systemd[1]: Started sshd.service - OpenSSH server daemon.
Feb 18 11:34:38 DESKTOP-8A256M6 sshd[68]: PAM unable to dlopen(/usr/lib64/security/pam_sss.so):
/usr/lib64/security/pam_sss.so: cannot open shared object file: No such file or directory
Feb 18 11:34:38 DESKTOP-8A256M6 sshd[68]: PAM adding faulty module: /usr/lib64/security/pam_sss.
so
Feb 18 11:34:38 DESKTOP-8A256M6 sshd[67]: PAM unable to dlopen(/usr/lib64/security/pam_sss.so):
/usr/lib64/security/pam_sss.so: cannot open shared object file: No such file or directory
Feb 18 11:34:38 DESKTOP-8A256M6 sshd[67]: PAM adding faulty module: /usr/lib64/security/pam_sss.
so
Feb 18 11:34:38 DESKTOP-8A256M6 sshd[67]: Accepted publickey for user from ::1 port 34468 ssh2: ED25519 SHA256:RzJAGMQVvbKMy3mY5kKOywth1IauyfpdcYvK0U1iXrE
Feb 18 11:34:38 DESKTOP-8A256M6 sshd[68]: Accepted publickey for user from ::1 port 34470 ssh2: ED25519 SHA256:RzJAGMQVvbKMy3mY5kKOywth1IauyfpdcYvK0U1iXrE
Feb 18 11:34:38 DESKTOP-8A256M6 sshd[68]: pam_unix(sshd:session): session opened for user user(uid=1000) by (uid=0)
Feb 18 11:34:38 DESKTOP-8A256M6 sshd[67]: pam_unix(sshd:session): session opened for user user(uid=1000) by (uid=0)
-- Boot d7483dd70b9f4e78864dc69ddd909545 --
Feb 18 12:31:16 DESKTOP-8A256M6 systemd[1]: Starting sshd.service - OpenSSH server daemon...
Feb 18 12:31:17 DESKTOP-8A256M6 sshd[25]: Server listening on 0.0.0.0 port 49675.
Feb 18 12:31:17 DESKTOP-8A256M6 sshd[25]: Server listening on :: port 49675.
Feb 18 12:31:17 DESKTOP-8A256M6 systemd[1]: Started sshd.service - OpenSSH server daemon.
-- Boot 6ca7a04f859a45e7bb6548ba432f8f50 --
Feb 18 13:47:20 DESKTOP-8A256M6 systemd[1]: Starting sshd.service - OpenSSH server daemon...
Feb 18 13:47:20 DESKTOP-8A256M6 sshd[25]: Server listening on 0.0.0.0 port 49675.
Feb 18 13:47:20 DESKTOP-8A256M6 sshd[25]: Server listening on :: port 49675.
Feb 18 13:47:20 DESKTOP-8A256M6 systemd[1]: Started sshd.service - OpenSSH server daemon.
-- Boot 1fd421016f944fce9172a3e2f63ba5c3 --
Feb 22 00:12:56 DESKTOP-8A256M6 systemd[1]: Starting sshd.service - OpenSSH server daemon...
Feb 22 00:12:57 DESKTOP-8A256M6 sshd[25]: Server listening on 0.0.0.0 port 49675.
Feb 22 00:12:57 DESKTOP-8A256M6 sshd[25]: Server listening on :: port 49675.
Feb 22 00:12:57 DESKTOP-8A256M6 systemd[1]: Started sshd.service - OpenSSH server daemon.
-- Boot fa78c5db2725454aa10ccfcf3e59f047 --
Feb 22 21:39:13 DESKTOP-8A256M6 systemd[1]: Starting sshd.service - OpenSSH server daemon...
Feb 22 21:39:14 DESKTOP-8A256M6 sshd[25]: Server listening on 0.0.0.0 port 49675.
Feb 22 21:39:14 DESKTOP-8A256M6 sshd[25]: Server listening on :: port 49675.
Feb 22 21:39:14 DESKTOP-8A256M6 systemd[1]: Started sshd.service - OpenSSH server daemon.
-- Boot 366a048caec7416abfb4e69a1c430e33 --
Feb 22 21:58:29 DESKTOP-8A256M6 systemd[1]: Starting sshd.service - OpenSSH server daemon...
Feb 22 21:58:30 DESKTOP-8A256M6 sshd[25]: Server listening on 0.0.0.0 port 49675.
Feb 22 21:58:30 DESKTOP-8A256M6 sshd[25]: Server listening on :: port 49675.
Feb 22 21:58:30 DESKTOP-8A256M6 systemd[1]: Started sshd.service - OpenSSH server daemon.
-- Boot bfe7f9e258024e59bc5f42d940ac0f9d --
Feb 22 22:02:31 DESKTOP-8A256M6 systemd[1]: Starting sshd.service - OpenSSH server daemon...
Feb 22 22:02:31 DESKTOP-8A256M6 sshd[25]: Server listening on 0.0.0.0 port 49675.
Feb 22 22:02:31 DESKTOP-8A256M6 systemd[1]: Started sshd.service - OpenSSH server daemon.
Feb 22 22:02:31 DESKTOP-8A256M6 sshd[25]: Server listening on :: port 49675.
Feb 22 22:36:44 DESKTOP-8A256M6 sshd[25]: Received signal 15; terminating.
Feb 22 22:36:44 DESKTOP-8A256M6 systemd[1]: Stopping sshd.service - OpenSSH server daemon...
Feb 22 22:36:44 DESKTOP-8A256M6 systemd[1]: sshd.service: Deactivated successfully.
Feb 22 22:36:44 DESKTOP-8A256M6 systemd[1]: Stopped sshd.service - OpenSSH server daemon.
Feb 22 22:36:53 DESKTOP-8A256M6 systemd[1]: Starting sshd.service - OpenSSH server daemon...
Feb 22 22:36:53 DESKTOP-8A256M6 sshd[25]: Server listening on 0.0.0.0 port 49675.
Feb 22 22:36:53 DESKTOP-8A256M6 sshd[25]: Server listening on :: port 49675.
Feb 22 22:36:53 DESKTOP-8A256M6 systemd[1]: Started sshd.service - OpenSSH server daemon.

ps -ef after running the above command

UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 22:36 ?        00:00:00 /lib/systemd/systemd
root        18     1  0 22:36 ?        00:00:00 /usr/lib/systemd/systemd-journald
root        25     1  0 22:36 ?        00:00:00 sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups
dbus        28     1  0 22:36 ?        00:00:00 /usr/bin/dbus-broker-launch --scope system --audit
root        29     1  0 22:36 ?        00:00:00 /usr/lib/systemd/systemd-userdbd
dbus        30    28  0 22:36 ?        00:00:00 dbus-broker --log 4 --controller 9 --machine-id 43c24020ff5e49838ebd4554ff633d1d --max-bytes 536870912 --max-fds 4096 --max-matches 16384 --audit
root        31    29  0 22:36 ?        00:00:00 systemd-userwork
root        32    29  0 22:36 ?        00:00:00 systemd-userwork
root        33    29  0 22:36 ?        00:00:00 systemd-userwork
user        36     1  0 22:36 ?        00:00:00 /usr/lib/systemd/systemd --user
user        37    36  0 22:36 ?        00:00:00 (sd-pam)
user        44    36  0 22:36 ?        00:00:00 /usr/bin/sleep infinity
root        46     0  0 22:37 pts/1    00:00:00 su -l user
root        51     1  0 22:37 ?        00:00:00 /usr/lib/systemd/systemd-logind
user        52    46  0 22:37 pts/1    00:00:00 -bash
user        74    52  0 22:38 pts/1    00:00:00 ps -ef

jeffmaury · 2023-02-22T16:53:34Z

Seems the sshd service stopped around 22:36:44 so maybe checking why it stopped

bwgjoseph · 2023-02-22T17:41:23Z

Sorry, I have no idea. I'm not familiar with this, but if there's something for me to run, I can do that.

If it helps, I am running within a VM. Pretty much a fresh VM with WSL and podman installed only.

mvromer · 2023-03-01T16:39:08Z

For what it's worth, I've been running into this as well from time to time on my Windows 10 machine running WSL 2. Like others, occasionally it seems to resolve itself, but today I haven't been able to figure out how to coax my podman machine back to a working state. That being said, this is what I'm seeing on my end. Not sure if it helps to find a lasting solution or not, but maybe it can help narrow out some other debugging paths.

In my case, a podman system connection list shows that podman is listening for ssh connections on port 49312. I can enter into my machine using wsl -d podman-machine-default and see that sshd is, in fact, listening on that port. I can also run journalctl and see that the sshd service is still running normally.

On the Windows side, I run TCPView and can't see anything listening on port 49312. That's enough to explain the connection refused errors I get from any client on the Windows side. It's almost like WSL isn't exposing to Windows the ports services running inside the Linux distributions are listening on.

Interestingly enough, when I launch my Ubuntu WSL distro with wsl -d Ubuntu, I can run ss -l -t and see the service listening on port 49312. If I try to run ssh user@localhost -p 49312, it does try to connect to the sshd running in my podman machine. I can even run journalctl from the podman machine and see the connection attempt being made.

On top of that, if I start netcat inside my Ubuntu machine using nc -l 8787, I do see it show up in the list of listening ports inside TCPView, which says the wslhost.exe process is doing the listening. I can run the Windows distro of ncat from the NMap 7.92 distro, connect using ncat 127.0.0.1 8787, and successfully echo commands from the Windows side to the Ubuntu side.

At this point I'm not sure why some ports get exposed through WSL vs. not. I'm continuing to try things on my end, and I'd be happy to try to collect any debug output I can on my end. Hopefully though that this helps narrow down where to look for the underlying fault.

isaacsgi · 2023-03-08T19:39:10Z

I am having this same issue.

Have been a longtime Docker user, but now working for a new firm, they asked that I switch over to Podman.

Note, this is a NEW PC: Windows 11 Pro; Version22H2; OS Build: 22621.1265

I had Podman working fine, shutdown and restarted machine after some of the latest Windows Updates and now can't get Podman to work.

Please let me know what logs or diagnostics I can provide to help resolve.

jeffmaury · 2023-03-09T13:19:58Z

What is the output of podman system connection ls and podman machine ls

isaacsgi · 2023-03-09T20:11:50Z

PS C:\Users\GilIsaacs> podman system connection ls
Name                         URI                                                          Identity
                  Default
podman-machine-default       ssh://user@localhost:63320/run/user/1000/podman/podman.sock  C:\Users\GilIsaacs\.ssh\podman-machine-default  false
podman-machine-default-root  ssh://root@localhost:63320/run/podman/podman.sock            C:\Users\GilIsaacs\.ssh\podman-machine-default  true
PS C:\Users\GilIsaacs> podman machine ls
NAME                    VM TYPE     CREATED       LAST UP            CPUS        MEMORY      DISK SIZE
podman-machine-default  wsl         25 hours ago  Currently running  20          914.7MB     2.294GB

jeffmaury · 2023-03-10T08:33:39Z

So what kind of error do you have. What happen when you run podman ps or podman images

isaacsgi · 2023-03-10T14:18:26Z

PS C:\Users\GilIsaacs> podman info
Error: failed to connect: dial tcp [::1]:63320: connectex: No connection could be made because the target machine actively refused it.
PS C:\Users\GilIsaacs> podman ps
Error: failed to connect: dial tcp [::1]:63320: connectex: No connection could be made because the target machine actively refused it.

isaacsgi · 2023-03-10T14:35:00Z

Seems that my base Windows terminal doesn't connect to the machine, but when I go into the running machine it can run?

PS C:\Users\GilIsaacs> podman run hello-world
Error: failed to connect: dial tcp [::1]:63320: connectex: No connection could be made because the target machine actively refused it.
PS C:\Users\GilIsaacs> wsl -d podman-machine-default

You will be automatically entered into a nested process namespace where
systemd is running. If you need to access the parent namespace, hit ctrl-d
or type exit. This also means to log out you need to exit twice.

[user@gilisaacs ~]$ podman run hello-world
!... Hello Podman World ...!

         .--"--.
       / -     - \
      / (O)   (O) \
   ~~~| -=(,Y,)=- |
    .---. /`  \   |~~
 ~/  o  o \~~~~.----. ~~
  | =(X)= |~  / (O (O) \
   ~~~~~~~  ~| =(Y_)=-  |
  ~~~~    ~~~|   U      |~~

Project:   https://github.com/containers/podman
Website:   https://podman.io
Documents: https://docs.podman.io
Twitter:   @Podman_io
[user@gilisaacs ~]$ podman ps
CONTAINER ID  IMAGE       COMMAND     CREATED     STATUS      PORTS       NAMES
[user@gilisaacs ~]$ podman images
REPOSITORY                               TAG         IMAGE ID      CREATED       SIZE
quay.io/podman/hello                     latest      e66ef1031ffc  44 hours ago  82.3 kB
registry.access.redhat.com/ubi8-micro    latest      89765c93bd06  3 weeks ago   28.5 MB

jeffmaury · 2023-03-10T14:36:55Z

From the Windows shell can you try:

podman machine stop
podman machine start
podman ps

bwgjoseph · 2023-03-10T14:38:32Z

Hi, was wondering if there's anything I can try with my issue. Thanks

isaacsgi · 2023-03-10T15:26:30Z

PS C:\Users\GilIsaacs> podman machine stop
Machine "podman-machine-default" stopped successfully

PS C:\Users\GilIsaacs> podman machine start
Starting machine "podman-machine-default"
API forwarding listening on: npipe:////./pipe/docker_engine

Docker API clients default to this address. You do not need to set DOCKER_HOST.
Machine "podman-machine-default" started successfully
PS C:\Users\GilIsaacs> podman ps
Error: failed to connect: dial tcp [::1]:63320: connectex: No connection could be made because the target machine actively refused it.
PS C:\Users\GilIsaacs>

jeffmaury · 2023-03-10T15:57:51Z

Can you run netstat -n -q | find "LISTENING"

n1hility · 2023-05-31T16:10:22Z

@tppalani So when that happens that means you have the previous generation of WSL that was a fixed / static component of the OS. In the older generation wsl --update will at least update the Linux kernel in use, so you could try that first, and instead of wsl --version, you can use wsl --status on that one to verify the version. Microsoft recently moved to a model where there is an independent update stream of WSL, allowing all recent variants and versions of Windows to stay current. However you need to be on 21H1, 21H2, or 22H2 of Windows 10, or 21H1 on Windows 11. If you run Windows update it should present you with the option to install one of these. Once that is done (and you have rebooted), wsl --install should install the current stream.

tppalani · 2023-05-31T17:12:42Z

Hi @n1hility here the output of wsl --status

Default Distribution: podman-machine-default
Default Version: 2

Windows Subsystem for Linux was last updated on 4/21/2023
Kernel version: 5.10.16

jaxor24 · 2023-06-04T03:33:08Z

I ran into this same issue after a Windows 10 update this week. Moving from 4.5.0 to 4.5.1 then running the following fixed it:
podman machine rm
podman machine init
podman machine start

tppalani · 2023-06-06T16:44:52Z

Hi @n1hility

Can you please help me about this

 wsl -l -v
  NAME                                   STATE           VERSION
* podman-machine-default    Running         2
  wsl-vpnkit                              Running         2


podman system connection ls
Name                         URI                                                          Identity                                      Default
podman-machine-default       ssh://[email protected]:64117/run/user/1000/podman/podman.sock  C:\Users\user\.ssh\podman-machine-default  false
podman-machine-default-root  ssh://[email protected]:64117/run/podman/podman.sock            C:\Users\user\.ssh\podman-machine-default  true


Get-NetTCPConnection -LocalPort 64117 -State Listen

LocalAddress                        LocalPort RemoteAddress                       RemotePort State       AppliedSetting OwningProcess
------------                        --------- -------------                       ---------- -----       -------------- -------------
::1                                      64117     ::                                  0          Listen                     40484
127.0.0.1                           64117     0.0.0.0                             0          Listen                     40484

when I'm doing kubectl get pod i'm getting error connection refused with different port number

$ kubectl get pod
E0606 22:02:52.983865   40012 memcache.go:238] couldn't get current server API group list: Get "https://127.0.0.1:51916/api?timeout=32s": dial tcp 127.0.0.1:51916: connectex: No connection could be made because the target machine actively refused it

TroySchmidt · 2023-06-20T10:04:22Z

I have tried everything I can. Uninstalled, reinstalled, uninstalled WSL and let Podman reinstall that. The connection refused error is still sporadic even with the latest podman install 4.5.1. I got the podman info to connect for me before I did the complete uninstall / reinstall this last time, but port forwarding to sql server container was still failing to connect.

So this last time I uninstalled WSL and now back to the failed to connect connectex errors.

techert · 2023-06-20T18:59:40Z

I have the same issue. It worked fine until Windows had crashed and restarted.
Windows Version 22H2 Build 19045.3086

podman.exe version 4.5.1
Machine had been created with this version.

wsl -l -v
NAME STATE VERSION

podman-machine-default Running 2
Ubuntu-20.04 Stopped 2
Ubuntu-18.04 Stopped 1

podman system connection ls
Name URI Identity Default
podman-machine-default ssh://[email protected]:11627/run/user/1000/podman/podman.sock C:\Users\me.ssh\podman-machine-default true
podman-machine-default-root ssh://[email protected]:11627/run/podman/podman.sock C:\Users\me.ssh\podman-machine-default false

Get-NetTCPConnection -LocalPort 11627 -State Listen
Get-NetTCPConnection : No matching MSFT_NetTCPConnection objects found by CIM query for instances of the ROOT/StandardCimv2/MSFT_NetTCPConnection class on the CIM server: SELECT * FROM MSFT_NetTCPConnection WHERE ((LocalPort = 11627)) AND ((State =
2)). Verify query parameters and retry.
At line:1 char:1

Get-NetTCPConnection -LocalPort 11627 -State Listen

  + CategoryInfo          : ObjectNotFound: (MSFT_NetTCPConnection:String) [Get-NetTCPConnection], CimJobException
  + FullyQualifiedErrorId : CmdletizationQuery_NotFound,Get-NetTCPConnection

Windows event log shows:
Error listening on socket: npipe:////./pipe/docker_engine: open \.\pipe\docker_engine: Access is denied.

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
  <Provider Name=".NET Runtime" /> 
  <EventID Qualifiers="0">1000</EventID> 
  <Version>0</Version> 
  <Level>2</Level> 
  <Task>0</Task> 
  <Opcode>0</Opcode> 
  <Keywords>0x80000000000000</Keywords> 
  <TimeCreated SystemTime="2023-06-20T16:52:29.2467589Z" /> 
  <EventRecordID>212456</EventRecordID> 
  <Correlation /> 
  <Execution ProcessID="0" ThreadID="0" /> 
  <Channel>Application</Channel> 
  <Computer>mydesktop</Computer> 
  <Security /> 
  </System>
<EventData>
  <Data>[error] podman-machine-default: Error occured in execution group: Error listening on socket: npipe:////./pipe/docker_engine: open \\.\pipe\docker_engine: Access is denied.</Data> 
  </EventData>
  </Event>

What helped me fix it was to disable the "Start on login" in the settings.
Rebooted.
Then started podman desktop as administrator.

TroySchmidt · 2023-06-20T20:14:51Z

I did get mine resolved by completely going thru the uninstall process again. I also uninstalled WSL as well this time and rebooted. I got it all to install and podman info worked. Then it looked like the sql server container I couldn't connect. But once I changed it from localhost over to 127.0.0.1 then things started to work without an issue.
So, I don't know if that is a port forwarding podman problem, a WSL relay problem, or some other configuration on my machine.

caaromerogi · 2023-06-21T16:53:06Z

I ran into this same issue after a Windows 10 update this week. Moving from 4.5.0 to 4.5.1 then running the following fixed it: podman machine rm podman machine init podman machine start

This solved the issue for me. Podman v4.5.1 in W10

fangpenlin · 2023-07-05T22:24:35Z

I encountered the same problem, and I think I found a solution, at least for my case.
tl;dr, run the following commands with admin privilege powershell fix the problem for me

net stop winnat
net start winnat

And here's what I have tried.

I tried to run SSH to localhost in podman-machine-default, and it worked, so seems like the sshd is working correctly
Also checked the sshd service and log, didn't see anything looks like it's not working
I checked my wslconfig, and it seems like localhostForwarding should be enabled by default
I ran simple http server in a WSL instance with port like 8080, and localhostForwarding is working correctly by running a wslrelay.exe for that port
Then I wondered, what about the port 60516 assigned by podman for the ssh connection? So I tried to shutdown sshd and run a simple http server instead, somehow the localhostForwarding feature of WSL is not working for that port (no wslrelay.exe started)

After some research, I finally found this WSL issues:

microsoft/WSL#5306

Not sure how WSL works for making the reserved ports, but somehow the port assigned by podman got reserved by WSL. I ran

netsh int ipv4 show excludedportrange protocol=tcp

and the result:

Protocol tcp Port Exclusion Ranges

Start Port    End Port
----------    --------
      5357        5357
     50000       50059     *
     56547       56646
     56647       56746
     59963       60062
     60063       60162
     60263       60362
     60363       60462
     60463       60562
     60563       60662
     60663       60762
     60763       60862
     60866       60965
     61283       61382
     61514       61613
     61614       61713
     61714       61813
     61814       61913
     61914       62013
     62014       62113
     62114       62213
     62214       62313
     62314       62413
     62414       62513
     62514       62613

* - Administered port exclusions.

As you can see my assigned port 60516 is covered by the excluded range 60463 to 60562, and probably that's now wslrelay.exe is not started for the port.

After restarting the service:

net stop winnat
net start winnat

And see the excluded port ranges again:

netsh int ipv4 show excludedportrange protocol=tcp

Protocol tcp Port Exclusion Ranges

Start Port    End Port
----------    --------
      5357        5357
     50000       50059     *

* - Administered port exclusions.

The podman port is not in the excluded list and thus working as expected now. I didn't dig too deep into why WSL is excluding tons of ports like that and somehow those ports covered podman's SSH port, could be a bug. People we are interested can dig deeper into the issue I found. But I guess podman desktop might need to find a way to workaround that issue by selecting a port less likely to be excluded by WSL or maybe come up with a way to dynamically select one not from the excluded range.

n1hility · 2023-07-06T00:12:53Z

@fangpenlin great observation! We should ensure the port selected is just below the dynamic range, since these can be tied up by WSL / hyperv for internal usage. Will get a fix in for that.

recodingLife · 2023-08-22T21:01:18Z

For me the root cause was a DNS configuration issue in WSL, which sometimes arises after a Windows reboot.

To address this:

Open WSL terminal.
Edit the DNS configuration: sudo vim /etc/resolv.conf.
Set the nameserver to Google's DNS: nameserver 8.8.8.8.
Save and exit the file.

This change redirects DNS requests to Google's nameserver and allowed podman to connect again.

Hope it can help others...

n1hility · 2023-08-23T15:42:48Z

@recodingLife you were most likely experiencing a different connectivity issue than the primary issue described here since DNS should not be a factor. If you are using a VPN that drops VM traffic you might also be interested in the recently introduced user-mode-networking feature.

n1hility · 2023-08-23T15:49:53Z

FYI to those impacted by the issue described by the OP: #19557 (just merged) should address this problem.

The new behavior will detect port conflicts on startup and automatically switch to a new available port, which will then be bindable by the wsl relay. You won't need to take any action other than updating to the new release when it's available.

kid1412621 · 2023-11-01T07:21:03Z

after enabled User mode networking, this issue had been resolved.

lstocchi mentioned this issue Jun 14, 2023

Podman Desktop not able to detect the list of images present in local computer podman-desktop/podman-desktop#2806

Closed

themr0c mentioned this issue Jul 3, 2023

Update the troubleshooting guide podman-desktop/podman-desktop#3045

Closed

7 tasks

n1hility self-assigned this Jul 6, 2023

github-actions bot added the windows issue/bug on Windows label Aug 8, 2023

n1hility transferred this issue from podman-desktop/podman-desktop Aug 8, 2023

n1hility mentioned this issue Aug 8, 2023

Implement automatic port reassignment on Windows #19557

Merged

themr0c added this to Podman Desktop docs and troubleshooting Aug 16, 2023

themr0c moved this to Todo in Podman Desktop docs and troubleshooting Aug 17, 2023

themr0c moved this from Todo to In Progress in Podman Desktop docs and troubleshooting Aug 17, 2023

themr0c mentioned this issue Aug 17, 2023

docs: added _Older WSL versions might lead to networking issues_ - fi… podman-desktop/podman-desktop#3583

Merged

themr0c moved this from In Progress to In Review in Podman Desktop docs and troubleshooting Aug 17, 2023

openshift-merge-robot closed this as completed in #19557 Aug 23, 2023

github-project-automation bot moved this from In Review to Done in Podman Desktop docs and troubleshooting Aug 23, 2023

github-project-automation bot moved this from 📋 Backlog to ✅ Done in Podman Desktop Planning Aug 23, 2023

github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Jan 31, 2024

github-actions bot locked as resolved and limited conversation to collaborators Jan 31, 2024

No connection could be made because the target machine actively refused it #19554

No connection could be made because the target machine actively refused it #19554

Comments

bwgjoseph commented Feb 18, 2023 • edited Loading

Bug description

Operating system

Version

Steps to reproduce

Relevant log output

Additional context

benoitf commented Feb 21, 2023

jeffmaury commented Feb 21, 2023

benoitf commented Feb 21, 2023

bwgjoseph commented Feb 21, 2023

bwgjoseph commented Feb 21, 2023

jeffmaury commented Feb 21, 2023

jeffmaury commented Feb 21, 2023 • edited Loading

bwgjoseph commented Feb 21, 2023

benoitf commented Feb 21, 2023

bwgjoseph commented Feb 22, 2023

jeffmaury commented Feb 22, 2023

jeffmaury commented Feb 22, 2023

bwgjoseph commented Feb 22, 2023

jeffmaury commented Feb 22, 2023

bwgjoseph commented Feb 22, 2023

jeffmaury commented Feb 22, 2023

bwgjoseph commented Feb 22, 2023

jeffmaury commented Feb 22, 2023

bwgjoseph commented Feb 22, 2023

mvromer commented Mar 1, 2023

isaacsgi commented Mar 8, 2023

jeffmaury commented Mar 9, 2023

isaacsgi commented Mar 9, 2023

jeffmaury commented Mar 10, 2023

isaacsgi commented Mar 10, 2023

isaacsgi commented Mar 10, 2023

jeffmaury commented Mar 10, 2023

bwgjoseph commented Mar 10, 2023

isaacsgi commented Mar 10, 2023

jeffmaury commented Mar 10, 2023

n1hility commented May 31, 2023 • edited Loading

tppalani commented May 31, 2023

jaxor24 commented Jun 4, 2023 • edited Loading

tppalani commented Jun 6, 2023

TroySchmidt commented Jun 20, 2023

techert commented Jun 20, 2023 • edited Loading

TroySchmidt commented Jun 20, 2023

caaromerogi commented Jun 21, 2023 • edited Loading

fangpenlin commented Jul 5, 2023

n1hility commented Jul 6, 2023

recodingLife commented Aug 22, 2023

n1hility commented Aug 23, 2023

n1hility commented Aug 23, 2023

kid1412621 commented Nov 1, 2023

bwgjoseph commented Feb 18, 2023 •

edited

Loading

jeffmaury commented Feb 21, 2023 •

edited

Loading

n1hility commented May 31, 2023 •

edited

Loading

jaxor24 commented Jun 4, 2023 •

edited

Loading

techert commented Jun 20, 2023 •

edited

Loading

caaromerogi commented Jun 21, 2023 •

edited

Loading