Fix platform handling for empty os/arch values #3698

Luap99 · 2022-01-10T14:32:29Z

What type of PR is this?

/kind bug

What this PR does / why we need it:

If you run buildah bud --platform windows/ the os is ignored at the
moment and it uses the host os. We should still use the os from the
platform in this case. This fixes an issue with podman-remote build
where it is possible that only the os is set in the platform string.

How to verify it

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Luap99 · 2022-01-10T14:57:08Z

@nalind @rhatdan @flouthoc PTAL
This should fix this failing test in podman: https://storage.googleapis.com/cirrus-ci-6707778565701632-fcae48/artifacts/containers/podman/5501967194652672/html/int-remote-ubuntu-2110-root-host.log.html#t--podman-build-os-windows-flag--1

nalind · 2022-01-10T16:06:20Z

Why are we passing an empty value for one (but not the other) to this API?

Luap99 · 2022-01-10T16:14:54Z

podman-remote build converts --os and --arch into a platform, if you only set os the arch is left empty.
The problem is that buildah completely ignores the OS part in this case. IMO it should either error or accept it. I decided to accept it with this PR.

imagebuildah/build.go

rhatdan · 2022-01-10T16:50:15Z

/approve

openshift-ci · 2022-01-10T16:50:18Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Luap99, rhatdan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [rhatdan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Luap99 · 2022-01-11T15:37:24Z

It looks like I have a legitimate test failure:

[+0628s] not ok 177 buildah-bud-policy
[+0628s] # (from function `die' in file ./helpers.bash, line 249,
[+0628s] #  from function `run_buildah' in file ./helpers.bash, line 236,
[+0628s] #  in test file ./bud.bats, line 1883)
[+0628s] #   `run_buildah build --signature-policy ${TESTSDIR}/deny.json -t ${target} -v ${TESTSDIR}:/testdir ${TESTSDIR}/bud/mount' failed with status 125
[+0628s] # /var/tmp/go/src/github.com/containers/buildah/tests /var/tmp/go/src/github.com/containers/buildah/tests
[+0628s] # $ /var/tmp/go/src/github.com/containers/buildah/tests/./../bin/buildah build --signature-policy /var/tmp/go/src/github.com/containers/buildah/tests/./deny.json -t foo -v /var/tmp/go/src/github.com/containers/buildah/tests/.:/testdir /var/tmp/go/src/github.com/containers/buildah/tests/./bud/mount
[+0628s] # STEP 1/2: FROM alpine
[+0628s] # Resolved "alpine" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
[+0628s] # Trying to pull docker.io/library/alpine:latest...
[+0628s] # error creating build container: Source image rejected: Running image docker://alpine:latest is rejected by policy.
[+0628s] # [ rc=125 (expected) ]
[+0628s] # $ /var/tmp/go/src/github.com/containers/buildah/tests/./../bin/buildah build --signature-policy /var/tmp/go/src/github.com/containers/buildah/tests/./docker.json -t foo -v /var/tmp/go/src/github.com/containers/buildah/tests/.:/testdir /var/tmp/go/src/github.com/containers/buildah/tests/./bud/mount
[+0628s] # STEP 1/2: FROM alpine
[+0628s] # Resolved "alpine" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
[+0628s] # Trying to pull docker.io/library/alpine:latest...
[+0628s] # Getting image source signatures
[+0628s] # Copying blob sha256:9d16cba9fb961d1aafec9542f2bf7cb64acfc55245f9e4eb5abecd4cdc38d749
[+0628s] # Copying blob sha256:9d16cba9fb961d1aafec9542f2bf7cb64acfc55245f9e4eb5abecd4cdc38d749
[+0628s] # Copying config sha256:961769676411f082461f9ef46626dd7a2d1e2b2a38e6a44364bcbecf51e66dd4
[+0628s] # Writing manifest to image destination
[+0628s] # Storing signatures
[+0628s] # STEP 2/2: RUN mount
[+0628s] # overlay on / type overlay (rw,context="system_u:object_r:container_file_t:s0:c168,c767",relatime,lowerdir=/var/tmp/buildah_tests.zwx98x/root/overlay/l/FLHC2LMLZ5PTO7KVOQGCJ2K44I,upperdir=/var/tmp/buildah_tests.zwx98x/root/overlay/7c34c030e726c1f964cbdb6eedc9b2c73a5e4a86a36a1bdf3821eeb37aecbb97/diff,workdir=/var/tmp/buildah_tests.zwx98x/root/overlay/7c34c030e726c1f964cbdb6eedc9b2c73a5e4a86a36a1bdf3821eeb37aecbb97/work,volatile)
[+0628s] # proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
[+0628s] # /dev/sda1 on /testdir type ext4 (rw,seclabel,relatime)
[+0628s] # sysfs on /sys type sysfs (ro,seclabel,nosuid,nodev,noexec,relatime)
[+0628s] # tmpfs on /dev type tmpfs (rw,context="system_u:object_r:container_file_t:s0:c168,c767",nosuid,size=65536k,mode=755,inode64)
[+0628s] # /dev/sda1 on /run/.containerenv type ext4 (rw,seclabel,relatime)
[+0628s] # /dev/sda1 on /etc/resolv.conf type ext4 (rw,seclabel,relatime)
[+0628s] # /dev/sda1 on /etc/hosts type ext4 (rw,seclabel,relatime)
[+0628s] # devpts on /dev/pts type devpts (rw,context="system_u:object_r:container_file_t:s0:c168,c767",nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666)
[+0628s] # shm on /dev/shm type tmpfs (rw,context="system_u:object_r:container_file_t:s0:c168,c767",nosuid,nodev,noexec,relatime,size=200k,inode64)
[+0628s] # mqueue on /dev/mqueue type mqueue (rw,seclabel,nosuid,nodev,noexec,relatime)
[+0628s] # /dev/sda1 on /run/secrets type ext4 (rw,seclabel,relatime)
[+0628s] # cgroup on /sys/fs/cgroup type tmpfs (rw,context="system_u:object_r:container_file_t:s0:c168,c767",nosuid,nodev,noexec,relatime,size=1024k,inode64)
[+0628s] # cgroup on /sys/fs/cgroup/pids type cgroup (ro,seclabel,nosuid,nodev,noexec,relatime,pids)
[+0628s] # cgroup on /sys/fs/cgroup/cpuset type cgroup (ro,seclabel,nosuid,nodev,noexec,relatime,cpuset)
[+0628s] # cgroup on /sys/fs/cgroup/misc type cgroup (ro,seclabel,nosuid,nodev,noexec,relatime,misc)
[+0628s] # cgroup on /sys/fs/cgroup/memory type cgroup (ro,seclabel,nosuid,nodev,noexec,relatime,memory)
[+0628s] # cgroup on /sys/fs/cgroup/hugetlb type cgroup (ro,seclabel,nosuid,nodev,noexec,relatime,hugetlb)
[+0628s] # cgroup on /sys/fs/cgroup/freezer type cgroup (ro,seclabel,nosuid,nodev,noexec,relatime,freezer)
[+0628s] # cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (ro,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio)
[+0628s] # cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (ro,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct)
[+0628s] # cgroup on /sys/fs/cgroup/blkio type cgroup (ro,seclabel,nosuid,nodev,noexec,relatime,blkio)
[+0628s] # cgroup on /sys/fs/cgroup/devices type cgroup (ro,seclabel,nosuid,nodev,noexec,relatime,devices)
[+0628s] # cgroup on /sys/fs/cgroup/perf_event type cgroup (ro,seclabel,nosuid,nodev,noexec,relatime,perf_event)
[+0628s] # cgroup on /sys/fs/cgroup/systemd type cgroup (ro,seclabel,nosuid,nodev,noexec,relatime,xattr,name=systemd)
[+0628s] # tmpfs on /proc/acpi type tmpfs (ro,context="system_u:object_r:container_file_t:s0:c168,c767",relatime,size=0k,inode64)
[+0628s] # devtmpfs on /proc/kcore type devtmpfs (rw,seclabel,size=1988532k,nr_inodes=497133,mode=755,inode64)
[+0628s] # devtmpfs on /proc/keys type devtmpfs (rw,seclabel,size=1988532k,nr_inodes=497133,mode=755,inode64)
[+0628s] # devtmpfs on /proc/latency_stats type devtmpfs (rw,seclabel,size=1988532k,nr_inodes=497133,mode=755,inode64)
[+0628s] # devtmpfs on /proc/timer_list type devtmpfs (rw,seclabel,size=1988532k,nr_inodes=497133,mode=755,inode64)
[+0628s] # tmpfs on /proc/scsi type tmpfs (ro,context="system_u:object_r:container_file_t:s0:c168,c767",relatime,size=0k,inode64)
[+0628s] # tmpfs on /sys/firmware type tmpfs (ro,context="system_u:object_r:container_file_t:s0:c168,c767",relatime,size=0k,inode64)
[+0628s] # tmpfs on /sys/fs/selinux type tmpfs (ro,context="system_u:object_r:container_file_t:s0:c168,c767",relatime,size=0k,inode64)
[+0628s] # tmpfs on /sys/dev type tmpfs (ro,context="system_u:object_r:container_file_t:s0:c168,c767",relatime,size=0k,inode64)
[+0628s] # proc on /proc/bus type proc (ro,relatime)
[+0628s] # proc on /proc/fs type proc (ro,relatime)
[+0628s] # proc on /proc/irq type proc (ro,relatime)
[+0628s] # proc on /proc/sys type proc (ro,relatime)
[+0628s] # proc on /proc/sysrq-trigger type proc (ro,relatime)
[+0628s] # COMMIT foo
[+0628s] # Getting image source signatures
[+0628s] # Copying blob sha256:03901b4a2ea88eeaad62dbe59b072b28b6efa00491962b8741081c5df50c65e0
[+0628s] # Copying blob sha256:8ec9bf60fb4945f8898dd87c96524ee98261deed256d394ce6e9cf033968978d
[+0628s] # Copying config sha256:de1b0153c180830569125f4d65492f70e3cad0e4517709c5f9387b275337d1e1
[+0628s] # Writing manifest to image destination
[+0628s] # Storing signatures
[+0628s] # --> de1b0153c18
[+0628s] # Successfully tagged localhost/foo:latest
[+0628s] # de1b0153c180830569125f4d65492f70e3cad0e4517709c5f9387b275337d1e1
[+0628s] # $ /var/tmp/go/src/github.com/containers/buildah/tests/./../bin/buildah push --signature-policy /var/tmp/go/src/github.com/containers/buildah/tests/./deny.json foo dir:/var/tmp/buildah_tests.zwx98x/mount
[+0628s] # Getting image source signatures
[+0628s] # Copying blob sha256:03901b4a2ea88eeaad62dbe59b072b28b6efa00491962b8741081c5df50c65e0
[+0628s] # Copying blob sha256:8ec9bf60fb4945f8898dd87c96524ee98261deed256d394ce6e9cf033968978d
[+0628s] # Copying config sha256:de1b0153c180830569125f4d65492f70e3cad0e4517709c5f9387b275337d1e1
[+0628s] # Writing manifest to image destination
[+0628s] # Storing signatures
[+0628s] # $ /var/tmp/go/src/github.com/containers/buildah/tests/./../bin/buildah rmi foo
[+0628s] # untagged: localhost/foo:latest
[+0628s] # de1b0153c180830569125f4d65492f70e3cad0e4517709c5f9387b275337d1e1
[+0628s] # $ /var/tmp/go/src/github.com/containers/buildah/tests/./../bin/buildah pull --signature-policy /var/tmp/go/src/github.com/containers/buildah/tests/./docker.json alpine
[+0628s] # 961769676411f082461f9ef46626dd7a2d1e2b2a38e6a44364bcbecf51e66dd4
[+0628s] # $ /var/tmp/go/src/github.com/containers/buildah/tests/./../bin/buildah build --signature-policy /var/tmp/go/src/github.com/containers/buildah/tests/./deny.json -t foo -v /var/tmp/go/src/github.com/containers/buildah/tests/.:/testdir /var/tmp/go/src/github.com/containers/buildah/tests/./bud/mount
[+0628s] # STEP 1/2: FROM alpine
[+0628s] # Resolved "alpine" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
[+0628s] # Trying to pull docker.io/library/alpine:latest...
[+0628s] # error creating build container: Source image rejected: Running image docker://alpine:latest is rejected by policy.
[+0628s] # [ rc=125 (** EXPECTED 0 **) ]
[+0628s] # #/vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
[+0628s] # #| FAIL: exit code is 125; expected 0
[+0628s] # #\^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

@nalind Any idea how this patch could result in the image being pulled?

nalind · 2022-01-11T17:12:15Z

The pull logic in common's libimage bases some of its behavior on the contents of the SystemContext that we pass to it, though that's changed a couple of times at this point.

Luap99 · 2022-01-11T17:22:11Z

I guess at this point I will just fix the issue in podman to get things done but we should still consider fixing this in buildah, so either this approach or we make it error for an invalid platform string (e.g. windows/) WDYT?

Luap99 · 2022-01-11T17:48:40Z

Ahh I cannot really fix this in podman so I still need this. I looked at c/common and it looks like containers/common#880 fixes the test issue (at least locally).

TomSweeneyRedHat · 2022-01-12T00:55:11Z

@Luap99 looks like you have file conflicts with this. Rebase fun I'm sure.

If you run `buildah bud --platform windows/` the os is ignored at the moment and it uses the host os. We should still use the os from the platform in this case. This fixes an issue with podman-remote build where it is possible that only the os is set in the platform string. Signed-off-by: Paul Holzinger <[email protected]>

Luap99 · 2022-01-12T12:35:33Z

Is this a flake?

[+0903s] # error building at STEP "COPY --from=0 /root/Dockerfile.no-run /root/": checking on sources under "/var/tmp/buildah_tests.mia0b9/root/overlay/2d7b6d4f8db751f19e9b291c3e11f46495b290f63d29e8d49b52cd849d9e134f/merged": error in copier subprocess: error changing to intended-new-root directory "/var/tmp/buildah_tests.mia0b9/root/overlay/2d7b6d4f8db751f19e9b291c3e11f46495b290f63d29e8d49b52cd849d9e134f/merged": chdir /var/tmp/buildah_tests.mia0b9/root/overlay/2d7b6d4f8db751f19e9b291c3e11f46495b290f63d29e8d49b52cd849d9e134f/merged: no such file or directory
[+0903s] # [ rc=125 (** EXPECTED 0 **) ]
[+0903s] # #/vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
[+0903s] # #| FAIL: exit code is 125; expected 0
[+0903s] # #\^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Would be great if someone could restart the test.

vrothberg · 2022-01-12T12:38:14Z

Restarted. Yes, that's a flake :(

Luap99 · 2022-01-12T13:34:50Z

The test failed again :/

Luap99 · 2022-01-12T15:56:07Z

Tests are finally green.
@nalind @rhatdan @vrothberg @flouthoc PTAL

nalind · 2022-01-12T16:01:48Z

LGTM

rhatdan · 2022-01-12T16:02:11Z

/lgtm

openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Jan 10, 2022

nalind reviewed Jan 10, 2022

View reviewed changes

imagebuildah/build.go Outdated Show resolved Hide resolved

Luap99 force-pushed the os-arch branch from fa292ef to e1af41d Compare January 10, 2022 16:49

openshift-ci bot added the approved label Jan 10, 2022

Luap99 force-pushed the os-arch branch from e1af41d to d5043a8 Compare January 11, 2022 13:56

Luap99 force-pushed the os-arch branch from f0d8f79 to f1969e1 Compare January 11, 2022 19:54

openshift-ci bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 11, 2022

Luap99 mentioned this pull request Jan 12, 2022

api: remove use of deprecated/removed constants for CNI from buildah containers/podman#12822

Closed

Luap99 force-pushed the os-arch branch from f1969e1 to 42f1333 Compare January 12, 2022 10:31

openshift-ci bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 12, 2022

Luap99 mentioned this pull request Jan 12, 2022

Update vendoring in buildah,common, image containers/podman#12794

Closed

openshift-ci bot assigned rhatdan Jan 12, 2022

openshift-ci bot added the lgtm label Jan 12, 2022

openshift-merge-robot merged commit d744ebc into containers:main Jan 12, 2022

Luap99 deleted the os-arch branch January 12, 2022 16:05

edsantiago mentioned this pull request Jan 13, 2022

CI-Flake: bud-multiple-platform-no-run fails: error changing to intended-new-root directory #3710

Closed

github-actions bot added the locked - please file new issue/PR label Sep 13, 2023

github-actions bot locked as resolved and limited conversation to collaborators Sep 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix platform handling for empty os/arch values #3698

Fix platform handling for empty os/arch values #3698

Luap99 commented Jan 10, 2022

Luap99 commented Jan 10, 2022

nalind commented Jan 10, 2022

Luap99 commented Jan 10, 2022

rhatdan commented Jan 10, 2022

openshift-ci bot commented Jan 10, 2022

Luap99 commented Jan 11, 2022

nalind commented Jan 11, 2022

Luap99 commented Jan 11, 2022

Luap99 commented Jan 11, 2022

TomSweeneyRedHat commented Jan 12, 2022

Luap99 commented Jan 12, 2022

vrothberg commented Jan 12, 2022

Luap99 commented Jan 12, 2022

Luap99 commented Jan 12, 2022

nalind commented Jan 12, 2022

rhatdan commented Jan 12, 2022

Fix platform handling for empty os/arch values #3698

Fix platform handling for empty os/arch values #3698

Conversation

Luap99 commented Jan 10, 2022

What type of PR is this?

What this PR does / why we need it:

How to verify it

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Luap99 commented Jan 10, 2022

nalind commented Jan 10, 2022

Luap99 commented Jan 10, 2022

rhatdan commented Jan 10, 2022

openshift-ci bot commented Jan 10, 2022

Luap99 commented Jan 11, 2022

nalind commented Jan 11, 2022

Luap99 commented Jan 11, 2022

Luap99 commented Jan 11, 2022

TomSweeneyRedHat commented Jan 12, 2022

Luap99 commented Jan 12, 2022

vrothberg commented Jan 12, 2022

Luap99 commented Jan 12, 2022

Luap99 commented Jan 12, 2022

nalind commented Jan 12, 2022

rhatdan commented Jan 12, 2022