Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to restart a podman machine when proxy is set in containers.conf file #14636

Closed
benoitf opened this issue Jun 17, 2022 · 5 comments · Fixed by #14893
Closed

Unable to restart a podman machine when proxy is set in containers.conf file #14636

benoitf opened this issue Jun 17, 2022 · 5 comments · Fixed by #14893
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. machine

Comments

@benoitf
Copy link
Contributor

benoitf commented Jun 17, 2022

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

Sets proxy settings in containers.conf

Creates and start a podman machine.
Then stop the machine and restart it
machine is not starting

Steps to reproduce the issue:

  1. Add proxy settings in containers.conf file
[containers]

[engine]
  env = ["https_proxy=http://10.0.0.123:9191"]

[machine]

[network]

[secrets]

[configmaps]
  1. Create/start a machine
$ podman machine init --now                                                                                                                                                                                       
Extracting compressed file
Image resized.
Machine init complete
Waiting for VM ...
Mounting volume... /Users/benoitf:/Users/benoitf

This machine is currently configured in rootless mode. If your containers
require root permissions (e.g. ports < 1024), or if you run into compatibility
issues with non-podman clients, you can switch using the following command:

	podman machine set --rootful

API forwarding listening on: /Users/benoitf/.local/share/containers/podman/machine/podman-machine-default/podman.sock

The system helper service is not installed; the default Docker API socket
address can't be used by podman. If you would like to install it run the
following commands:

	sudo /usr/local/Cellar/podman/4.1.1/bin/podman-mac-helper install
	podman machine stop; podman machine start

You can still connect Docker API clients by setting DOCKER_HOST using the
following command in your terminal session:

	export DOCKER_HOST='unix:///Users/benoitf/.local/share/containers/podman/machine/podman-machine-default/podman.sock'

Machine "podman-machine-default" started successfully
  1. run a container
$ podman run --rm quay.io/podman/hello                                                                                                                                                                   
Trying to pull quay.io/podman/hello:latest...
Getting image source signatures
Copying blob sha256:818f2864c3fe9a15b67f4f1b4140d55497670d3ba20387e309ea60e434fc792c
Copying blob sha256:818f2864c3fe9a15b67f4f1b4140d55497670d3ba20387e309ea60e434fc792c
Copying config sha256:291a4d05648352c2ec59057b4cdca71cb2e7a6d11e204633fd11ab4cf82a318d
Writing manifest to image destination
Storing signatures
!... Hello Podman World ...!

         .--"--.
       / -     - \
      / (O)   (O) \
   ~~~| -=(,Y,)=- |
    .---. /`  \   |~~
 ~/  o  o \~~~~.----. ~~
  | =(X)= |~  / (O (O) \
   ~~~~~~~  ~| =(Y_)=-  |
  ~~~~    ~~~|   U      |~~

Project:   https://github.com/containers/podman
Website:   https://podman.io
Documents: https://docs.podman.io
Twitter:   @Podman_io
  1. stop machine
$ podman machine stop                                                                                                                                                                                        Machine "podman-machine-default" stopped successfully
  1. restart the machine
podman machine start  --log-level=debug                                                                                                                                                                         
INFO[0000] podman filtering at log level debug
DEBU[0000] [/usr/local/bin/qemu-system-x86_64 -m 2048 -smp 1 -fw_cfg name=opt/com.coreos/config,file=/Users/benoitf/.config/containers/podman/machine/qemu/podman-machine-default.ign -qmp unix://var/folders/tg/_5rxbhmj4xncz4szvpgswrmc0000gn/T/podman/qmp_podman-machine-default.sock,server=on,wait=off -netdev socket,id=vlan,fd=3 -device virtio-net-pci,netdev=vlan,mac=5a:94:ef:e4:0c:ee -device virtio-serial -chardev socket,path=/var/folders/tg/_5rxbhmj4xncz4szvpgswrmc0000gn/T/podman/podman-machine-default_ready.sock,server=on,wait=off,id=podman-machine-default_ready -device virtserialport,chardev=podman-machine-default_ready,name=org.fedoraproject.port.0 -machine q35,accel=hvf:tcg -cpu host -virtfs local,path=/Users/benoitf,mount_tag=vol0,security_model=mapped-xattr -drive if=virtio,file=/Users/benoitf/.local/share/containers/podman/machine/qemu/podman-machine-default_fedora-coreos-36.20220605.2.0-qemu.x86_64.qcow2 -fw_cfg name=opt/com.coreos/environment,string=aHR0cHNfcHJveHk9Imh0dHA6Ly8xMC4wLjAuMTIzOjkxOTEifA== -fw_cfg name=opt/com.coreos/environment,string=aHR0cHNfcHJveHk9Imh0dHA6Ly8xMC4wLjAuMTIzOjkxOTEifA==]
Starting machine "podman-machine-default"
[/usr/local/opt/podman/libexec/gvproxy -listen-qemu unix:///var/folders/tg/_5rxbhmj4xncz4szvpgswrmc0000gn/T/podman/qmp_podman-machine-default.sock -pid-file /var/folders/tg/_5rxbhmj4xncz4szvpgswrmc0000gn/T/podman/podman-machine-default.pid -ssh-port 64205 -forward-sock /Users/benoitf/.local/share/containers/podman/machine/podman-machine-default/podman.sock -forward-dest /run/user/501/podman/podman.sock -forward-user core -forward-identity /Users/benoitf/.ssh/podman-machine-default --debug]
DEBU[0000] [/usr/local/bin/qemu-system-x86_64 -m 2048 -smp 1 -fw_cfg name=opt/com.coreos/config,file=/Users/benoitf/.config/containers/podman/machine/qemu/podman-machine-default.ign -qmp unix://var/folders/tg/_5rxbhmj4xncz4szvpgswrmc0000gn/T/podman/qmp_podman-machine-default.sock,server=on,wait=off -netdev socket,id=vlan,fd=3 -device virtio-net-pci,netdev=vlan,mac=5a:94:ef:e4:0c:ee -device virtio-serial -chardev socket,path=/var/folders/tg/_5rxbhmj4xncz4szvpgswrmc0000gn/T/podman/podman-machine-default_ready.sock,server=on,wait=off,id=podman-machine-default_ready -device virtserialport,chardev=podman-machine-default_ready,name=org.fedoraproject.port.0 -machine q35,accel=hvf:tcg -cpu host -virtfs local,path=/Users/benoitf,mount_tag=vol0,security_model=mapped-xattr -drive if=virtio,file=/Users/benoitf/.local/share/containers/podman/machine/qemu/podman-machine-default_fedora-coreos-36.20220605.2.0-qemu.x86_64.qcow2 -fw_cfg name=opt/com.coreos/environment,string=aHR0cHNfcHJveHk9Imh0dHA6Ly8xMC4wLjAuMTIzOjkxOTEifA== -fw_cfg name=opt/com.coreos/environment,string=aHR0cHNfcHJveHk9Imh0dHA6Ly8xMC4wLjAuMTIzOjkxOTEifA==]
Waiting for VM ...
Error: dial unix /var/folders/tg/_5rxbhmj4xncz4szvpgswrmc0000gn/T/podman/podman-machine-default_ready.sock: connect: connection refused

Describe the results you received:
Error: dial unix /var/folders/tg/_5rxbhmj4xncz4szvpgswrmc0000gn/T/podman/podman-machine-default_ready.sock: connect: connection refused

Describe the results you expected:
machine should restart

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

podman version 4.1.1

Output of podman info --debug:

(paste your output here)

Package info (e.g. output of rpm -q podman or apt list podman):

(paste your output here)

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes/No

Additional environment details (AWS, VirtualBox, physical, etc.):

@openshift-ci openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Jun 17, 2022
@Luap99
Copy link
Member

Luap99 commented Jun 17, 2022

This only happens with the custom proxy env in containers.conf?

@benoitf
Copy link
Contributor Author

benoitf commented Jun 17, 2022

yes if I remove the proxy setting in the containers.conf file it works fine

$ podman machine init  --now                                                                                                                                                                          Extracting compressed file
Image resized.
Machine init complete
Waiting for VM ...
Mounting volume... /Users/benoitf:/Users/benoitf

This machine is currently configured in rootless mode. If your containers
require root permissions (e.g. ports < 1024), or if you run into compatibility
issues with non-podman clients, you can switch using the following command:

	podman machine set --rootful

API forwarding listening on: /Users/benoitf/.local/share/containers/podman/machine/podman-machine-default/podman.sock

The system helper service is not installed; the default Docker API socket
address can't be used by podman. If you would like to install it run the
following commands:

	sudo /usr/local/Cellar/podman/4.1.1/bin/podman-mac-helper install
	podman machine stop; podman machine start

You can still connect Docker API clients by setting DOCKER_HOST using the
following command in your terminal session:

	export DOCKER_HOST='unix:///Users/benoitf/.local/share/containers/podman/machine/podman-machine-default/podman.sock'

Machine "podman-machine-default" started successfully
$  podman run --rm quay.io/podman/hello                                                                                                                                                                    Trying to pull quay.io/podman/hello:latest...
Getting image source signatures
Copying blob sha256:818f2864c3fe9a15b67f4f1b4140d55497670d3ba20387e309ea60e434fc792c
Copying blob sha256:818f2864c3fe9a15b67f4f1b4140d55497670d3ba20387e309ea60e434fc792c
Copying config sha256:291a4d05648352c2ec59057b4cdca71cb2e7a6d11e204633fd11ab4cf82a318d
Writing manifest to image destination
Storing signatures
!... Hello Podman World ...!

         .--"--.
       / -     - \
      / (O)   (O) \
   ~~~| -=(,Y,)=- |
    .---. /`  \   |~~
 ~/  o  o \~~~~.----. ~~
  | =(X)= |~  / (O (O) \
   ~~~~~~~  ~| =(Y_)=-  |
  ~~~~    ~~~|   U      |~~

Project:   https://github.com/containers/podman
Website:   https://podman.io
Documents: https://docs.podman.io
Twitter:   @Podman_io

$ podman machine stop                                                                                                                                                                                        Machine "podman-machine-default" stopped successfully

$ podman machine start                                                                                                                                                                                            
Starting machine "podman-machine-default"
Waiting for VM ...
Mounting volume... /Users/benoitf:/Users/benoitf

This machine is currently configured in rootless mode. If your containers
require root permissions (e.g. ports < 1024), or if you run into compatibility
issues with non-podman clients, you can switch using the following command:

	podman machine set --rootful

API forwarding listening on: /Users/benoitf/.local/share/containers/podman/machine/podman-machine-default/podman.sock

The system helper service is not installed; the default Docker API socket
address can't be used by podman. If you would like to install it run the
following commands:

	sudo /usr/local/Cellar/podman/4.1.1/bin/podman-mac-helper install
	podman machine stop; podman machine start

You can still connect Docker API clients by setting DOCKER_HOST using the
following command in your terminal session:

	export DOCKER_HOST='unix:///Users/benoitf/.local/share/containers/podman/machine/podman-machine-default/podman.sock'

Machine "podman-machine-default" started successfully

@ashley-cui
Copy link
Member

Closed via: #14665

@Luap99
Copy link
Member

Luap99 commented Jul 5, 2022

No I just linked this issue by accident, this was not fixed in #14665

@Luap99 Luap99 reopened this Jul 5, 2022
@Luap99
Copy link
Member

Luap99 commented Jul 11, 2022

Can you try with podman from latest main, this will return a proper qemu error. I guess this is the same as #14837 (comment)

Luap99 added a commit to Luap99/libpod that referenced this issue Jul 11, 2022
qemu fails when the same `fw_cfg` options is used more than once.
Since the current logic always adds a new option on each machine load
this will fail on the second start.

We can fix this by checking if the option is already set and replace but
I think it is easier to just not commit the option in the config and add
it dynamically on start. User that hit this bug have to recreate the
machine.

[NO NEW TESTS NEEDED]

Fixes containers#14636
Fixes containers#14837

Signed-off-by: Paul Holzinger <[email protected]>
mheon pushed a commit to mheon/libpod that referenced this issue Jul 26, 2022
qemu fails when the same `fw_cfg` options is used more than once.
Since the current logic always adds a new option on each machine load
this will fail on the second start.

We can fix this by checking if the option is already set and replace but
I think it is easier to just not commit the option in the config and add
it dynamically on start. User that hit this bug have to recreate the
machine.

[NO NEW TESTS NEEDED]

Fixes containers#14636
Fixes containers#14837

Signed-off-by: Paul Holzinger <[email protected]>
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 20, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 20, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. machine
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants