ARG values truncated at '=' when used to set ENV values #2424

srcshelton · 2020-06-25T22:38:47Z

Description

This is an apparent regression between buildah-1.15.0-dev (libpod-2.0.0_rc6) and buildah-1.15.0 (libpod-2.0.0_rc7):

When an ARG is used to initialise an ENV variable, the ENV value in the resulting image is truncated at the first equals sign.

This doesn't appear to be the same as issue #2345 as the ARG and ENV variable names are different and the value is partially coming through.

Steps to reproduce the issue:

Create a Docker Dockerfile file with the following content:

FROM scratch
ARG multivalarg="a=1 b=2 c=3 d=4"
ENV multival="${multivalarg}"

N.B. This works:

FROM scratch
ARG multivalarg="a b c d"
ENV multival="${multivalarg}"

Build the container (podman build -t test .);
Inspect the image variables (podman inspect -f '{{ .Config.Env }}' test):

Describe the results you received:

podman-2.0.0_rc6 and previous:

[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin multival=a=1 b=2 c=3 d=4]

podman-2.0.0_rc7 and later:

[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin multival=a]

Describe the results you expected:

multival=a=1 b=2 c=3 d=4

Output of rpm -q buildah or apt list buildah:

n/a (buildah installed as a libpod dependency)

Output of buildah version:

bash: buildah: command not found

Output of podman version if reporting a podman build issue:

Working:

podman version 2.0.0-rc6

Broken:

podman version 2.0.0-rc7 & podman version 2.0.0

Output of cat /etc/*release:

Gentoo Base System release 2.6
DISTRIB_ID="Gentoo"
NAME=Gentoo
ID=gentoo
PRETTY_NAME="Gentoo/Linux"
ANSI_COLOR="1;32"
HOME_URL="https://www.gentoo.org/"
SUPPORT_URL="https://www.gentoo.org/support/"
BUG_REPORT_URL="https://bugs.gentoo.org/"

Output of uname -a:

Linux dellr330 5.4.38-gentoo #20 SMP Thu May 21 18:05:56 2020 x86_64 Intel(R) Xeon(R) CPU E3-1240L v5 @ 2.10GHz GenuineIntel GNU/Linux

Output of cat /etc/containers/storage.conf:

[storage]
driver = "overlay"
runroot = "/space/podman/run"
graphroot = "/space/podman/storage"
rootless_storage_path = "/space/podman/user/$USER"

[storage.options]

[storage.options.aufs]
mountopt = ""

[storage.options.btrfs]
min_space = ""
size = ""

[storage.options.thinpool]
autoextend_percent = ""
autoextend_threshold = ""
basesize = ""
blocksize = ""
directlvm_device = ""
directlvm_device_force = ""
fs = ""
log_level = ""
min_free_space = ""
mkfsarg = ""
mountopt = ""
size = ""
use_deferred_deletion = ""
use_deferred_removal = ""
xfs_nospace_max_retries = ""

[storage.options.overlay]
ignore_chown_errors  =  "false"
mount_program = ""
mountopt = ""
size = ""

[storage.options.vfs]
ignore_chown_errors  =  "false"

[storage.options.zfs]
fsname = ""
mountopt = ""
size = ""

The text was updated successfully, but these errors were encountered:

srcshelton · 2020-06-25T22:40:21Z

libpod issue containers/podman#6785

srcshelton · 2020-06-25T22:54:31Z

(I've just noticed that even though the working podman binary reports the working version of buildah as being 1.15.0-dev, the source of this only references 1.15.0. buildah doesn't appear to adhere to immutable tags, and so I'm not sure if there's now any way after the fact to determine the last working buildah version without resorting to comparing dates...)

Belay the above - the podman source includes a snapshot of buildah, rather than fetching buildah as an external module!

TomSweeneyRedHat · 2020-06-26T00:20:40Z

@srcshelton thanks for the issue report. Any chance you could try Podman v2.0.0 or v2.0.1? I don't recall what versions of Buildah went into the 2.0.0.RC# variants, but I do know that we bumped it at one point from 1.14.9 to 1.15-dev. The latest release of Buildah 1.15.0 which went into Podman v2.0.0 should have the latest bits and hopefully works. Way past quitting time here and I'm losing the fight with my eyelids, will try myself in the morning if you don't beat me to it.

srcshelton · 2020-06-26T01:40:11Z

Hadn’t realised there was a v2.0.1, TBH - I’ll build that in the morning!

libpod-2.0.0_rc6/buildah-1.15.0-dev worked, libpod-2.0.0_rc7/buildah-1.15.0 & libpod-2.0.0_rc7/buildah-1.15.0 don’t appear to.

But I’m afraid that this does look to be an issue with buildah-1.15.0 😔

srcshelton · 2020-06-26T12:31:11Z

@srcshelton thanks for the issue report. Any chance you could try Podman v2.0.0 or v2.0.1? I don't recall what versions of Buildah went into the 2.0.0.RC# variants, but I do know that we bumped it at one point from 1.14.9 to 1.15-dev. The latest release of Buildah 1.15.0 which went into Podman v2.0.0 should have the latest bits and hopefully works. Way past quitting time here and I'm losing the fight with my eyelids, will try myself in the morning if you don't beat me to it.

I've just upgraded to libpod-2.0.1 and can confirm that it is also affected by the same issue:

host:
  arch: amd64
  buildahVersion: 1.15.0
  cgroupVersion: v1
  conmon:
    package: Unknown
    path: /usr/libexec/podman/conmon
    version: 'conmon version 2.0.17, commit: 41877362fc4685d55e0473d2e4a1cbe5e1debee0'
  cpus: 8
  distribution:
    distribution: gentoo
    version: unknown
  eventLogger: file
  hostname: dellr330
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.4.38-gentoo
  linkmode: dynamic
  memFree: 3446767616
  memTotal: 8132182016
  ociRuntime:
    name: runc
    package: Unknown
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc10
      commit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
      spec: 1.0.1-dev
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  rootless: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 25186459648
  swapTotal: 25769787392
  uptime: 506h 25m 10.63s (Approximately 21.08 days)
registries:
  search:
  - docker.io
  - quay.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 2
    paused: 0
    running: 1
    stopped: 1
  graphDriverName: overlay
  graphOptions:
    overlay.ignore_chown_errors: "false"
  graphRoot: /space/podman/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 604
  runRoot: /space/podman/run
  volumePath: /space/podman/storage/volumes
version:
  APIVersion: 1
  Built: 1593173493
  BuiltTime: Fri Jun 26 12:11:33 2020
  GitCommit: a11c4ead10177a66ef2810a0a92ea8ce2299da07
  GoVersion: go1.14.2
  OsArch: linux/amd64
  Version: 2.0.1

TomSweeneyRedHat · 2020-06-26T15:06:39Z

@srcshelton thanks for the tests and retests. Drats about it not being cured in the latest. I've been pulled into other fires this morning, will take a look a bit after lunch today.

TomSweeneyRedHat · 2020-06-26T18:57:33Z

I did a recent change here: https://github.com/openshift/imagebuilder/blob/07ab54df11165c27c2c78a85a093d1198d3c030b/internals.go#L106
and I think that might have been too ambitious in chopping out args. Will see if can fix that up.

TomSweeneyRedHat · 2020-06-26T18:57:47Z

@srcshelton ^^

srcshelton · 2020-06-26T21:16:49Z

I did a recent change here: https://github.com/openshift/imagebuilder/blob/07ab54df11165c27c2c78a85a093d1198d3c030b/internals.go#L106
and I think that might have been too ambitious in chopping out args. Will see if can fix that up.

I think you're right: https://github.com/openshift/imagebuilder/blob/07ab54df11165c27c2c78a85a093d1198d3c030b/internals.go#L108 looks as if it keeps the first two args, and discards the remainder - so var="a=1 b=2" is mapped to var="a" as I've observed.

srcshelton · 2020-06-26T21:29:55Z

Something along the lines of:

for _, envVal := range bEnv {
    sep := strings.Index(envVal, "=")
    if(sep >= 0) {
        envMap[envVal[sep:]] = envVal[:sep+1]
    }
}

(Disclaimer: I've not written any Go since 2016! ;)

If an ARG in a Dockerfile looks like this: ARG multivalarg="a=1 b=2 c=3 d=4" the processing would drop everything after the second equal sign leaving multivalarg="a". Addresses: containers/buildah#2424 Signed-off-by: TomSweeneyRedHat <[email protected]>

TomSweeneyRedHat · 2020-06-26T23:19:32Z

@srcshelton even simpler I just changed:

val := strings.Split(envVal, "=")

to

val := strings.SplitN(envVal, "=", 2)

Which tells golang to create two elements in val, everything to the left of the first equal sign in val[0], everything to the right of it in val[1]. Quick fix, took me about 3 times longer to write the regression test.

srcshelton · 2020-06-27T00:37:05Z

Nice 👍

rhatdan · 2020-08-06T14:31:09Z

It looks like this is fixed in upstream. Reopen if I am mistaken.

TomSweeneyRedHat · 2020-08-06T18:20:58Z

Reopening as this bug is going on my checklist darn it @rhatdan!

srcshelton mentioned this issue Jun 25, 2020

libpod-2.0.0_rc7 & libpod-2.0.0 truncate ENV values when set from ARGS on '=' containers/podman#6785

Closed

TomSweeneyRedHat mentioned this issue Jun 26, 2020

Handle args with multiple equal signs openshift/imagebuilder#167

Merged

rhatdan assigned TomSweeneyRedHat Jun 29, 2020

mythi mentioned this issue Jul 3, 2020

Prepare for release-0.18 intel/intel-device-plugins-for-kubernetes#417

Merged

rhatdan closed this as completed Aug 6, 2020

TomSweeneyRedHat reopened this Aug 6, 2020

TomSweeneyRedHat closed this as completed Aug 6, 2020

mythi mentioned this issue Feb 15, 2021

build-args are not propagated as env vars for RUN statements for buildah bud #2989

Closed

github-actions bot added the locked - please file new issue/PR label Sep 10, 2023

github-actions bot locked as resolved and limited conversation to collaborators Sep 10, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ARG values truncated at '=' when used to set ENV values #2424

ARG values truncated at '=' when used to set ENV values #2424

srcshelton commented Jun 25, 2020 •

edited

Loading

srcshelton commented Jun 25, 2020

srcshelton commented Jun 25, 2020 •

edited

Loading

TomSweeneyRedHat commented Jun 26, 2020

srcshelton commented Jun 26, 2020

srcshelton commented Jun 26, 2020

TomSweeneyRedHat commented Jun 26, 2020

TomSweeneyRedHat commented Jun 26, 2020

TomSweeneyRedHat commented Jun 26, 2020

srcshelton commented Jun 26, 2020

srcshelton commented Jun 26, 2020 •

edited

Loading

TomSweeneyRedHat commented Jun 26, 2020

srcshelton commented Jun 27, 2020

rhatdan commented Aug 6, 2020

TomSweeneyRedHat commented Aug 6, 2020

ARG values truncated at '=' when used to set ENV values #2424

ARG values truncated at '=' when used to set ENV values #2424

Comments

srcshelton commented Jun 25, 2020 • edited Loading

srcshelton commented Jun 25, 2020

srcshelton commented Jun 25, 2020 • edited Loading

TomSweeneyRedHat commented Jun 26, 2020

srcshelton commented Jun 26, 2020

srcshelton commented Jun 26, 2020

TomSweeneyRedHat commented Jun 26, 2020

TomSweeneyRedHat commented Jun 26, 2020

TomSweeneyRedHat commented Jun 26, 2020

srcshelton commented Jun 26, 2020

srcshelton commented Jun 26, 2020 • edited Loading

TomSweeneyRedHat commented Jun 26, 2020

srcshelton commented Jun 27, 2020

rhatdan commented Aug 6, 2020

TomSweeneyRedHat commented Aug 6, 2020

srcshelton commented Jun 25, 2020 •

edited

Loading

srcshelton commented Jun 25, 2020 •

edited

Loading

srcshelton commented Jun 26, 2020 •

edited

Loading