Qemu windows machine #16872

arixmkii · 2022-12-17T11:25:57Z

Adds QEMU podman machine provider on Windows hosts.

How to test it:

Build Windows installer from this branch and install it as normally
Install latest QEMU from https://qemu.weilnetz.de/w64/
Run commands in console to test it:
3.1. Set environment variable to select provider QEMU
3.2. Set PATH to include qemu binaries (their default installer seems doesn't update path)
3.3. Initialize new machine overriding username and image-path, because defaults are different from WSL and there is no way to have defaults per provider as of now
3.4. Start the machine
3.5. Run some network enabled workload to verify machine and networking

Example for bullet point 3:

set CONTAINERS_MACHINE_PROVIDER=QEMU
set PATH=%PROGRAMFILES%\qemu;%PATH%
podman machine init --cpus 4 --memory 8192 --username core --image-path testing
podman machine start
podman run -d -p 8080:80 nginx
curl http://localhost:8080/

Alternatively to setting environment variables in console session, one could opt for setting them globally for the host (restart is recommended to make sure that they would be propagated everywhere).

The PR will require rebase later as it includes commit from #16807 If needed the former one may be closed and squashed with this one, but for me it looks like #16807 has the value on its own and is pretty complete.

Note worthy technical details:

stub settings for Windows on Arm were added to allow compilation to pass for this target, but QEMU doesn't exist for it yet (mingw64 doesn't have this build target (at least in stable form);
usage of Environment Variable to select provider is a hack, because there is no feature to select provider from the CLI
it is impossible to operate with multiple providers within the same process (like list all QEMU and WSL2 machines)
ssh keys collision happens for same named WSL2 and QEMU machines - issue created Consider adding machine provider prefix to ssh keys #16710
vlan socket name is derived from qmp socket name, which works, but doesn't feel right - issue registered Use dedicated configured path for QEMU and gvproxy vlan socket #15852
named pipe collision with same named WSL2 machine - issue registered WSL machine - consider adding pipe address to machine inspect output #16860

Missing features:

9pfs support on Windows hosts is not yet merged into QEMU. So, folder mounts are unsupported.

Known issues:

~~sending Ctrl-C to the console or closing console session, where the machine was started will terminate QEMU and gvproxy.~~ (fixed)

Does this PR introduce a user-facing change?

QEMU podman machine provider on Windows hosts

Signed-off-by: Arthur Sengileyev <[email protected]>

openshift-ci · 2022-12-17T11:26:03Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: arixmkii
Once this PR has been reviewed and has the lgtm label, please assign baude for approval by writing /assign @baude in a comment. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

arixmkii

Some notes about specific code changes.

arixmkii · 2022-12-17T11:27:56Z

pkg/machine/qemu/machine.go

@@ -1749,6 +1798,10 @@ func (p *Provider) VMType() string {
 	return vmtype
 }

+func toVlanSockPath(path string) string {


Will not be needed, when #15852 is implemented. Now there is nowhere to save this, so, we rely on a different saved value and derive from it.

arixmkii · 2022-12-17T11:29:21Z

pkg/machine/qemu/machine.go

@@ -1289,6 +1330,14 @@ func (v *MachineVM) setupAPIForwarding(cmd []string) ([]string, string, apiForwa
 	cmd = append(cmd, []string{"-forward-user", forwardUser}...)
 	cmd = append(cmd, []string{"-forward-identity", v.IdentityPath}...)

+	// Workaround for Podman Desktop compatibility
+	if runtime.GOOS == "windows" {


Workaround to make Podman Desktop happy. Will not be needed when #16860 is implemented and Podman Desktop will be updated to use the CLI instead of building the address internally.

arixmkii · 2022-12-17T11:30:21Z

pkg/machine/qemu/machine.go

@@ -1229,6 +1261,10 @@ func (v *MachineVM) startHostNetworking() (string, apiForwardingState, error) {
 		return "", noForwarding, err
 	}
 	binary, err := cfg.FindHelperBinary(machine.ForwarderBinaryName, false)
+	// workaround for lookup on Windows


Will not be needed, when containers/common#1123 is fixed.

arixmkii · 2022-12-17T11:32:18Z

pkg/machine/qemu/machine.go

+		cmd = append(cmd, []string{"-listen-qemu", fmt.Sprintf("unix://%s", v.QMPMonitor.Address.GetPath()), "-pid-file", v.PidFilePath.GetPath()}...)
+	} else {
+		vlanSocket := machine.VMFile{Path: toVlanSockPath(v.QMPMonitor.Address.GetPath())}
+		cmd = append(cmd, []string{"-listen-qemu", fmt.Sprintf("unix://%s", strings.Replace(vlanSocket.GetPath(), "\\", "/", -1)), "-pid-file", v.PidFilePath.GetPath()}...)


This is the only place, which is tricky and OS specific, because GO URL parser seems to have troubles handling urls like unix://C:\temp\some.sock.

arixmkii · 2022-12-17T11:35:05Z

pkg/machine/qemu/machine.go

 		return err
 	}
-	defer qemuSocketConn.Close()
+	if runtime.GOOS != "windows" {


Don't remember what didn't work well on Windows here. Will need to re-check.

arixmkii · 2022-12-17T11:37:37Z

pkg/machine/qemu/machine.go

@@ -128,7 +129,12 @@ func (p *Provider) NewMachine(opts machine.InitOptions) (machine.VM, error) {
 	// Add network
 	// Right now the mac address is hardcoded so that the host networking gives it a specific IP address.  This is
 	// why we can only run one vm at a time right now
-	cmd = append(cmd, []string{"-netdev", "socket,id=vlan,fd=3", "-device", "virtio-net-pci,netdev=vlan,mac=5a:94:ef:e4:0c:ee"}...)
+	if runtime.GOOS != "windows" {


Technically this could be not OS specific, but QEMU version specific switch - it should be possible to use podman machine w/o passing FDs on QEMU versions equal or newer than 7.2.0 on all platforms (no switch necessary at all). But I guess it is a no go yet as there are some older RHEL linuxes, which doesn't have latest QEMU, but still need to be supported, so, it is easier to keep it this way than replace it with a switch based on QEMU version.

arixmkii · 2022-12-17T11:42:27Z

Also, I believe this should require some new tests, but I have no idea how to write e2e tests for this sort of changes (and if it is possible to get accelerated QEMU running in the CI machines).

arixmkii · 2022-12-17T12:43:15Z

sending Ctrl-C to the console or closing console session, where the machine was started will terminate QEMU and gvproxy.

The fix for this will be to use "w" versions of QEMU binaries ("qemu-system-x86_64w.exe" etc.) and build gvproxy on Windows with -ldflags -H=windowsgui

Fix for QEMU side could be applied by just modifying machine config json. Will push the code fix after gvproxy part is ready.

Update: gvproxy changes requested in containers/gvisor-tap-vsock#167

Signed-off-by: Arthur Sengileyev <[email protected]>

openshift-merge-robot · 2023-01-07T22:40:33Z

@arixmkii: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

arixmkii · 2023-01-22T18:54:11Z

Closing this as prototype has evolved since this PR and potential feedback on this prototype will not be as useful as it could be.

Add gvproxy to Windows packages

b8e4e7c

Signed-off-by: Arthur Sengileyev <[email protected]>

openshift-ci bot added the release-note label Dec 17, 2022

arixmkii marked this pull request as draft December 17, 2022 11:26

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 17, 2022

arixmkii commented Dec 17, 2022

View reviewed changes

arixmkii mentioned this pull request Dec 17, 2022

Support Qemu accelerated podman machine on Windows hosts #13006

Open

arixmkii mentioned this pull request Dec 17, 2022

Option to build gvproxy as Windows GUI (background) app containers/gvisor-tap-vsock#167

Merged

Windows QEMU machine support

5063d1a

Signed-off-by: Arthur Sengileyev <[email protected]>

arixmkii force-pushed the qemu-windows-machine branch from c307283 to 5063d1a Compare December 22, 2022 19:47

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 7, 2023

hgkamath mentioned this pull request Jan 8, 2023

On windows support qemu with accel whpx podman-desktop/podman-desktop#1134

Closed

arixmkii mentioned this pull request Jan 8, 2023

Updated options for QEMU on Windows hosts #17029

Merged

arixmkii closed this Jan 22, 2023

github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 5, 2023

github-actions bot locked as resolved and limited conversation to collaborators Sep 5, 2023

arixmkii deleted the qemu-windows-machine branch March 1, 2024 13:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Qemu windows machine #16872

Qemu windows machine #16872

arixmkii commented Dec 17, 2022 •

edited

Loading

openshift-ci bot commented Dec 17, 2022

arixmkii left a comment

arixmkii Dec 17, 2022

arixmkii Dec 17, 2022

arixmkii Dec 17, 2022

arixmkii Dec 17, 2022

arixmkii Dec 17, 2022

arixmkii Dec 17, 2022

arixmkii commented Dec 17, 2022

arixmkii commented Dec 17, 2022 •

edited

Loading

openshift-merge-robot commented Jan 7, 2023

arixmkii commented Jan 22, 2023

Qemu windows machine #16872

Qemu windows machine #16872

Conversation

arixmkii commented Dec 17, 2022 • edited Loading

Does this PR introduce a user-facing change?

openshift-ci bot commented Dec 17, 2022

arixmkii left a comment

Choose a reason for hiding this comment

arixmkii Dec 17, 2022

Choose a reason for hiding this comment

arixmkii Dec 17, 2022

Choose a reason for hiding this comment

arixmkii Dec 17, 2022

Choose a reason for hiding this comment

arixmkii Dec 17, 2022

Choose a reason for hiding this comment

arixmkii Dec 17, 2022

Choose a reason for hiding this comment

arixmkii Dec 17, 2022

Choose a reason for hiding this comment

arixmkii commented Dec 17, 2022

arixmkii commented Dec 17, 2022 • edited Loading

openshift-merge-robot commented Jan 7, 2023

arixmkii commented Jan 22, 2023

arixmkii commented Dec 17, 2022 •

edited

Loading

arixmkii commented Dec 17, 2022 •

edited

Loading