Leak the HTTP*PROXY environment variables into the VM when doing a podman machine start.

[rhatdan]

Leak the following environment variables

var ProxyEnv = []string{
	"http_proxy",
	"https_proxy",
	"ftp_proxy",
	"no_proxy",
	"HTTP_PROXY",
	"HTTPS_PROXY",
	"FTP_PROXY",
	"NO_PROXY",
}

Into the VM at start time.

<@cyberpear> systemd will set them if you pass on the kernel CMDLINE systemd.setenv=VARIABLE=VALUE at boot

[rhatdan]

The goal here is if you are sitting on a MAC and have these settings set, you want your Podman within the VM to follow them also.

[esendjer]

Hey @rhatdan
May I take this?

[rhatdan]

Go for it.

[rhatdan]

@baude thinks this could be done via qemu.

[esendjer]

@baude thinks this could be done via qemu.

SGTM. Unfortunately, I have no ideas how this could be done with qemu. I would thankful if @baude or somebody else share their thoughts or any useful links to clarify this.
Thank you

[flouthoc]

@rhatdan @esendjer I think this is already handled here: #12748

[flouthoc]

Please use the latest podman-remote and podman-server all proxy variables must be already set.

Edit: podman-server on mac must be updated so this works.

[flouthoc]

I see we would also need them to be in /etc/systemd/system.conf.d/proxy.conf so that entire system can use it. @esendjer you can use the above PR to extend variable to /etc/systemd/system.conf.d/proxy.conf via ignition in the similar manner.

[esendjer]

I see we would also need them to be in /etc/systemd/system.conf.d/proxy.conf so that entire system can use it. @esendjer you can use the above PR to extend variable to /etc/systemd/system.conf.d/proxy.conf via ignition in the similar manner.

Got it. Thanks! Will do it

[flouthoc]

@esendjer Something similar on the lines of following diff should be enough.

diff --git a/pkg/machine/ignition.go b/pkg/machine/ignition.go
index 206c9144f..89c5f515a 100644
--- a/pkg/machine/ignition.go
+++ b/pkg/machine/ignition.go
@@ -363,7 +363,7 @@ Delegate=memory pids cpu io
 		},
 	})
 
-	setProxyOpts := getProxyVariables()
+	setProxyOpts := getProxyVariables(false)
 	if setProxyOpts != "" {
 		files = append(files, File{
 			Node: Node{
@@ -381,9 +381,26 @@ Delegate=memory pids cpu io
 		})
 	}
 
+	setProxyOpts = getProxyVariables(true)
+	if setProxyOpts != "" {
+		files = append(files, File{
+			Node: Node{
+				Group: getNodeGrp("root"),
+				Path:  "/etc/systemd/system/system.conf.d/http-proxy.conf",
+				User:  getNodeUsr("root"),
+			},
+			FileEmbedded1: FileEmbedded1{
+				Append: nil,
+				Contents: Resource{
+					Source: encodeDataURLPtr(setProxyOpts),
+				},
+				Mode: intToPtr(0644),
+			},
+		})
+	}
+
 	setDockerHost := `export DOCKER_HOST="unix://$(podman info -f "{{.Host.RemoteSocket.Path}}")"
 `
-
 	files = append(files, File{
 		Node: Node{
 			Group: getNodeGrp("root"),
@@ -506,11 +523,18 @@ func prepareCertFile(path string, name string) (File, error) {
 	return file, nil
 }
 
-func getProxyVariables() string {
+func getProxyVariables(systemd bool) string {
 	proxyOpts := ""
+	if systemd {
+		proxyOpts = "[Service]\n"
+	}
 	for _, variable := range config.ProxyEnv {
 		if value, ok := os.LookupEnv(variable); ok {
-			proxyOpts += fmt.Sprintf("\n export %s=%s", variable, value)
+			if !systemd {
+				proxyOpts += fmt.Sprintf("\n export %s=%s", variable, value)
+			} else {
+				proxyOpts += fmt.Sprintf("Environment=\"%s=%s\"\n", variable, value)
+			}
 		}
 	}
 	return proxyOpts

[esendjer]

Hey folks!
Thank you for your advice and don't worry, I'm researching the question about how these env vars could be provided into VM in the best way and spread for the whole system.
For instance: what is better usage /etc/environment or /etc/profile.d/proxy-opts.sh, or another instance: is it a good way to use the ignition with providing a .conf file for systemd or to use passed options to the kernel via cmdline...

[esendjer]

The research has been done. Smal outcomes are below.

• environment.d and its files contain lists of environment variable assignments for services started by the systemd user instance. So, if I understand this right it doesn't set envs for the whole system services.
• qemu and cmdline couldn't be applied here because qemu allows adding or overriding kernel options only when an external kernel is used.
• In my point of view, a better way is using system and session service manager configuration files such as /etc/systemd/system.conf.d/*.conf file, like @flouthoc mentioned before.

So it will be done with the last option

UPD
podman is run under the core user in VM under systems user manager, for this reason, it gets envs from environment.d files. Because of that, adding envs to a /etc/environment.d/*.conf file in a VM was added in the related PR

[rhatdan]

If this is done in ignition, then it will only happen once correct, not every time the machine is started?

[esendjer]

Unfortunately, it's true for now, but at the same time, the current state doesn't allow propagating anything on a start too, because there are no mechanisms for this here now.

Let me dive deeper into the question about how it might be done on a start.

Also, I would ask you about embedding bash/python code in ignition is ok? I'm not sure that this way will be used, but it's important to me to know this before.

[rhatdan]

I have no idea, best idea is to try it out and see if it works.

[flouthoc]

@rhatdan @esendjer I think it will happen everytime when we do machine start cause it did persist config in filesystem. We don't need to anything special @esendjer have you tried machine init and then machine start followed by a machine stop and then machine start again. It should still persist variables.

The only drawback is that it will keep the environment variable exactly as it was on host when user did machine init.

[esendjer]

@flouthoc as far as I see the main goal of this issue and @rhatdan's question is in providing actual envs from a host into guests system avoiding a need of recreating the last one, for instance, when somebody overrides/update their envs to new, a VM has to be recreated to have a fresh envs.

[rhatdan]

Correct, I am fine with this improvement but this is not really fixed if I do

podman macine init
podman machine start
podman machine start
export HTTP_PROXY=1.2.3.4
podman machine start

Will not work.

[esendjer]

I hope, in the last commit I found a good way to provide and propagate environment variables related to proxy settings on a start

[esendjer]

Sorry, but unfortunately, it's still is not good enough
it needs to do start twice for applying new settings

WIP

[baude]

This would be an excellent candidate for a cabal, where all the community experts can contribute. What say you?

[esendjer]

This would be an excellent candidate for a cabal, where all the community experts can contribute. What say you?

@baude sorry, I don't understand what for a cabal means. Is it an offer to attract the attention of the community experts and ask them for help and advice?

So, maybe it is so. But I've managed to deal with the issue, just take a look at the latest commit. In that commit, all settings are propagated on a start successfully without a need to re-init and restart VM twice.