Add support for building macOS pkg installer

[anjannath]

it installs podman and supporting binaries along with
qemu to have a functioning podman install using a pkg

podman and podman-mac-helper is compiled from source

gvproxy binary is downloaded from its github releases
and qemu from github release of containers-contribs/qemu-macos-build

[NO NEW TESTS NEEDED]

Signed-off-by: Anjan Nath [email protected]

Does this PR introduce a user-facing change?

NONE

Needs #14324

[rhatdan]

@anjannath Looks like the validate stage does not like the format of your bash scripts.

[anjannath]

@anjannath Looks like the validate stage does not like the format of your bash scripts.

I think it was a missing new line after the shebang, re-pushed lets see

[TomSweeneyRedHat]

@gbraad Thoughts?

[gbraad]

This is the PR we propose from crc.
I already approved it from my end, but have no rights here.

[rhatdan]

LGTM
@baude @mheon PTAL

[anjannath]

Hi @rhatdan CI is green now, thanks.

I also wanted to get your views about the qemu build it uses from (http://github.com/containers-contribs/qemu-macos-build).

We'd like to move that repo under the containers org and have it hooked to the CI so that we can have automated builds of qemu and releases that we can use with the pkg, since that will also need to be maintained.

Or is it preferred to not install qemu as part of this installer and leave it to the user to setup, but probably that means the users who can't do brew install on their machines for some reason, will not have a full podman setup ootb after using this installer.

[gbraad]

Qemu is either part of the installer, or there is a a separate qemu installer? For us CRC we are not planning to use qemu as vfkit and virtiofs look very promising at this point. Perhaps it is possible for the moment to include a qemu binary and over time move to vfkit?

…

On Sat, Jun 11, 2022, 19:49 Anjan Nath ***@***.***> wrote: Hi @rhatdan <https://github.com/rhatdan> CI is green now, thanks. I also wanted to get your views about the qemu build it uses from ( http://github.com/containers-contribs/qemu-macos-build <https://github.com/containers-contribs/qemu-macos-build>). We'd like to move that repo under the containers org and have it hooked to the CI so that we can have automated builds of qemu and releases that we can use with the pkg, since that will also need to be maintained. Or is it preferred to not install qemu as part of this installer and leave it to the user to setup, but probably that means the users who can't do brew install on their machines for some reason, will not have a full podman setup ootb after using this installer. — Reply to this email directly, view it on GitHub <#14540 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAOZVP67KQ22XW4WHBC2LVOR4K7ANCNFSM5YHYTM2A> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[rhatdan]

@anjannath I am fine with move that into containers.

[gbraad]

@anjannath I am fine with move that into containers.

Lets make that a dependency of this pr. Who can move/accept this? We will do so tomorrow:
containers/podman-machine-qemu ?

---

Done

[anjannath]

@gbraad Also updated the PR to use the qemu build from https://github.com/containers/podman-machine-qemu/releases/tag/6.2.0-1

[rhatdan]

/approve
LGTM
@baude PTAL

[rhatdan]

@vrothberg @mheon @giuseppe PTAL

[vrothberg]

I can't test without a Mac.

[rhatdan]

@ashley-cui PTAL

[ashley-cui]

I was able to build the installer on my m1, but am unable to run it without the "install rosetta prompt" which shouldn't be the case. Looks like the install script is using aarch64 and arm arches, so this is strange that I'm getting this error. Anyone have any ideas?

[evidolob]

I was able to build and run installer on may intel mac.

[anjannath]

I was able to build the installer on my m1, but am unable to run it without the "install rosetta prompt" which shouldn't be the case. Looks like the install script is using aarch64 and arm arches, so this is strange that I'm getting this error. Anyone have any ideas?

looks like the gvproxy binary we are using from https://github.com/containers/gvisor-tap-vsock/releases/download/v0.3.0/gvproxy-darwin is x86 we need to publish arm64 binaries of gvproxy

[rhatdan]

Podman has no way to update containers.conf, but we can read multiple. Not sure if it supports dropin files yet.

[anjannath]

updated to set the helper binary location as suggested in #14540 (comment)

tested locally and everything is working as expected

[gbraad]

@ashley-cui @benoitf feedback needed

[benoitf]

hi,

I've tried with 9bcb56b commit

it works but it seems it's using the brew qemu binary instead of the one from /opt/podman

here are some processes:

benoitf          73002 102.0  0.9 37319460 309232 s001  R+   10:06AM   0:06.53 /usr/local/bin/qemu-system-x86_64 -m 2048 -smp 1 -fw_cfg name=opt/com.coreos/config,file=/Users/benoitf/.config/containers/podman/machine/qemu/podman-machine-default.ign -qmp unix://var/folders/tg/_5rxbhmj4xncz4szvpgswrmc0000gn/T/podman/qmp_podman-machine-default.sock,server=on,wait=off -netdev socket,id=vlan,fd=3 -device virtio-net-pci,netdev=vlan,mac=5a:94:ef:e4:0c:ee -device virtio-serial -chardev socket,path=/var/folders/tg/_5rxbhmj4xncz4szvpgswrmc0000gn/T/podman/podman-machine-default_ready.sock,server=on,wait=off,id=apodman-machine-default_ready -device virtserialport,chardev=apodman-machine-default_ready,name=org.fedoraproject.port.0 -pidfile /var/folders/tg/_5rxbhmj4xncz4szvpgswrmc0000gn/T/podman/podman-machine-default_vm.pid -machine q35,accel=hvf:tcg -cpu host -virtfs local,path=/Users/benoitf,mount_tag=vol0,security_model=mapped-xattr -drive if=virtio,file=/Users/benoitf/.local/share/containers/podman/machine/qemu/podman-machine-default_fedora-coreos-36.20220716.2.0-qemu.x86_64.qcow2 -display none
benoitf          73034   0.0  0.0 34262068    920 s002  R+   10:06AM   0:00.00 grep --color=auto --exclude-dir=.bzr --exclude-dir=CVS --exclude-dir=.git --exclude-dir=.hg --exclude-dir=.svn --exclude-dir=.idea --exclude-dir=.tox qemu
benoitf          73001   0.0  0.0 34848392  14576 s001  S+   10:06AM   0:00.02 /opt/podman/qemu/bin/gvproxy -listen-qemu unix:///var/folders/tg/_5rxbhmj4xncz4szvpgswrmc0000gn/T/podman/qmp_podman-machine-default.sock -pid-file /var/folders/tg/_5rxbhmj4xncz4szvpgswrmc0000gn/T/podman/podman-machine-default_proxy.pid -ssh-port 60357 -forward-sock /Users/benoitf/.local/share/containers/podman/machine/podman-machine-default/podman.sock -forward-dest /run/user/501/podman/podman.sock -forward-user core -forward-identity /Users/benoitf/.ssh/podman-machine-default

as we can see gvproxy is executed from /opt/podman/qemu/bin but qemu-system-x86_64 is coming from /usr/local/bin/qemu-system-x86_64

ls -la /opt/podman/qemu/bin/qemu-system-x86_64                                                                                                                                 -rwxr-xr-x@ 1 root  wheel  17101120 Jul 26 09:59 /opt/podman/qemu/bin/qemu-system-x86_64

but it's there

[benoitf]

And I've used make ARCH=amd64 NO_CODESIGN=1 pkginstaller to build the installer. Maybe my command is not correct

[anjannath]

can you share your PATH value echo $PATH i think the homebrew binaries dir is before the /opt/podman/qemu/bin one.

my thinking is that podman finds the qemu binary by searching at the PATH first and then in the configured helper binaries dir, but not sure about this

but even if it uses the qemu from brew its fine, as normal brew install podman depends on that version of qemu, the thing to test would be if you uninstall it brew uninstall qemu does it use the one installed by the installer in /opt/podman/qemu then

[benoitf]

echo $PATH
/opt/podman/bin:/Users/benoitf/.nvm/versions/node/v16.14.0/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin:/usr/local/munki:/opt/X11/bin:/Library/Apple/usr/bin

[benoitf]

but even if it uses the qemu from brew its fine, as normal brew install podman depends on that version of qemu

maybe I can have an older qemu version installed by brew

[benoitf]

uninstall qemu brew and trying again

benoitf          77320  27.9  0.1 37004376  37444 s001  R+   10:28AM   0:00.18 /opt/podman/qemu/bin/qemu-system-x86_64 -m 2048 -smp 1 -fw_cfg name=opt/com.coreos/config,file=/Users/benoitf/.config/containers/podman/machine/qemu/podman-machine-default.ign -qmp unix://var/folders/tg/_5rxbhmj4xncz4szvpgswrmc0000gn/T/podman/qmp_podman-machine-default.sock,server=on,wait=off -netdev socket,id=vlan,fd=3 -device virtio-net-pci,netdev=vlan,mac=5a:94:ef:e4:0c:ee -device virtio-serial -chardev socket,path=/var/folders/tg/_5rxbhmj4xncz4szvpgswrmc0000gn/T/podman/podman-machine-default_ready.sock,server=on,wait=off,id=apodman-machine-default_ready -device virtserialport,chardev=apodman-machine-default_ready,name=org.fedoraproject.port.0 -pidfile /var/folders/tg/_5rxbhmj4xncz4szvpgswrmc0000gn/T/podman/podman-machine-default_vm.pid -machine q35,accel=hvf:tcg -cpu host -virtfs local,path=/Users/benoitf,mount_tag=vol0,security_model=mapped-xattr -drive if=virtio,file=/Users/benoitf/.local/share/containers/podman/machine/qemu/podman-machine-default_fedora-coreos-36.20220716.2.0-qemu.x86_64.qcow2 -display none

it's using the /opt/podman/qemu/bin/qemu-system-x86_64 binary

[vrothberg]

@ashley-cui can you take a final look?

[ashley-cui]

I can build the package, but am still running into the rosetta prompt when running on my m1 using make ARCH=aarch64 NO_CODESIGN=1 pkginstaller. Could anyone else with a M1 and no rosetta installed confirm?

[benoitf]

I've only mac/intel so I haven't faced this issue

[ashley-cui]

@rhatdan @baude Maybe you could try it out?

make ARCH=aarch64 NO_CODESIGN=1 pkginstaller
cd out
open .

then double click the .pkg file from the opened finder window?

[ashley-cui]

Found a solution to the Rosetta problem! However, even after that, podman isn't installed into my path and going to /opt/podman and running ./podman results in zsh: permission denied: ./podman

[ashley-cui]

Suggested change

	<options customize="never" />
	<options customize="always" hostArchitectures="x86_64,arm64"/>

[anjannath]

only adding hostArchitectures="x86_64,arm64" solves the rosetta problem, the customize="always" attribute shows a component selection page with only one option to select so keeping it as customize="never"

[anjannath]

Found a solution to the Rosetta problem! However, even after that, podman isn't installed into my path and going to /opt/podman

yes it installs in /opt/podman and adds /opt/podman/bin to PATH, by appending PATH=/opt/podman/bin:$PATH to the shell config file in case of zsh ~/.zshrc and/or ~/.zshenv

and running ./podman results in zsh: permission denied: ./podman

is it missing executable permissions? maybe we need to explicitly make it executable in the postinstall script.

[ashley-cui]

Tried it again, it all worked!
/lgtm
/hold

[TomSweeneyRedHat]

@benoitf TY!
/hold cancel

[ashley-cui]

/cherry-pick v4.2

[openshift-cherrypick-robot]

@ashley-cui: new pull request created: #15193

In response to this:

/cherry-pick v4.2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.