Podman build fails: runc not installed by default

[lukengda]

Description
I am trying to build an image based on a debian image. The minimal Dockerfile i used to reproduce the error is the following:

FROM debian:buster

WORKDIR /app

RUN ls

Steps to reproduce the issue:

1. podman build .

Describe the results you received:

The output is the following:

STEP 1: FROM debian:buster
STEP 2: WORKDIR /app
--> Using cache e075df80ee5363465cbdb87ad37d3f540178c2f3b7f0315fe1ac5fca392b527e
STEP 3: RUN echo $PATH
error running container: error creating container for [/bin/sh -c echo $PATH]: : exec: "runc": executable file not found in $PATH
Error: error building at STEP "RUN echo $PATH": error while running runtime: exit status 1

This is the same picture as in #1309.

podman run -it --rm debian:buster sh works without like a charm, commands in the interactive shell are processed fine.

I installed buildah manually and tried again. runc was not installed as a dependency of buildah. After i installed runc directly, the build worked.

What is the podman default installation setup? Is the buildah package not installed by default with a podman installation? Should runc be installed by default if you install buildah?

Describe the results you expected:

This minimal image should build without any problems without having to manually install additional dependencies.

Step 3/3 : RUN ls
 ---> Running in e9db2782f2f1
Removing intermediate container e9db2782f2f1
 ---> fc2ce797ab60

I hope this is the correct place to report the issue I have...

Output of rpm -q buildah or apt list buildah:

buildah/bionic 1.9.2-2~ubuntu18.04~ppa2 amd64 [installed]

Output of buildah version:

Version:         1.9.2
Go Version:      go1.10.4
Image Spec:      1.0.1
Runtime Spec:    1.0.1-dev
CNI Spec:        0.4.0
libcni Version:  
Git Commit:      
Built:           Sun Jul 28 21:44:08 2019
OS/Arch:         linux/amd64

Output of podman version if reporting a podman build issue:

Version:            1.4.3
RemoteAPI Version:  1
Go Version:         go1.10.4
OS/Arch:            linux/amd64

Output of cat /etc/*release:

DISTRIB_ID=neon
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="KDE neon User Edition 5.16"
NAME="KDE neon"
VERSION="5.16"
ID=neon
ID_LIKE="ubuntu debian"
PRETTY_NAME="KDE neon User Edition 5.16"
VARIANT="User Edition"
VERSION_ID="18.04"
HOME_URL="http://neon.kde.org/"
SUPPORT_URL="http://neon.kde.org/"
BUG_REPORT_URL="http://bugs.kde.org/"
LOGO=start-here-kde-neon
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

Output of uname -a:

Linux <machine> 4.15.0-55-generic #60-Ubuntu SMP Tue Jul 2 18:22:20 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Output of cat /etc/containers/storage.conf:

# storage.conf is the configuration file for all tools
# that share the containers/storage libraries
# See man 5 containers-storage.conf for more information

# The "container storage" table contains all of the server options.
[storage]

# Default Storage Driver
driver = "overlay"

# Temporary storage location
runroot = "/var/run/containers/storage"

# Primary read-write location of container storage
graphroot = "/var/lib/containers/storage"

[storage.options]
# AdditionalImageStores is used to pass paths to additional read-only image stores
# Must be comma separated list.
additionalimagestores = [
]

# Size is used to set a maximum size of the container image.  Only supported by
# certain container storage drivers (currently overlay, zfs, vfs, btrfs)
size = ""

# OverrideKernelCheck tells the driver to ignore kernel checks based on kernel version
override_kernel_check = "true"

[vrothberg]

@lsm5 PTAL

[lsm5]

@nalind @TomSweeneyRedHat is buildah configured to use runc at /usr/lib/cri-o-runc/sbin/runc ? If not, how can we set that up? I can't have our runc at /usr/bin or /usr/sbin since that'd conflict with docker-ce and ubuntu's default runc respectively.

[lsm5]

buildah is currently not a dependency for podman, @nalind and @TomSweeneyRedHat can correct me if I'm wrong, but buildah code gets vendored into podman.

@lukengda buildah and podman worked fine for you after you installed ubuntu's default runc package, yes? Could you please confirm if you already had the cri-o-runc package installed from our PPA and buildah/podman didn't work with that?

[lsm5]

@vrothberg could you plesae create labels 'ubuntu' and 'packaging' and assign this to me?

[lukengda]

@lukengda buildah and podman worked fine for you after you installed ubuntu's default runc package, yes? Could you please confirm if you already had the cri-o-runc package installed from our PPA and buildah/podman didn't work with that?

yes, the build worked fine after i installed buildah and runc via apt. I don't know what i could confirm for the cri-o-runc package. I had podman installed for some weeks now, i imagined that builds worked before. Today after i encountered the problems i installed buildah and it did not work either.

As podman and buildah have the cri-o-runc package as a dependency, it must have been installed before‽

# apt-cache depends buildah
buildah
  Depends: cri-o-runc
  Depends: libgpgme11
  Depends: libseccomp2
  Depends: containers-common
  Depends: uidmap

# apt-cache depends podman
podman
  Depends: libseccomp2
  Depends: libdevmapper1.02.1
  Depends: libgpgme11
  Depends: conmon
  Depends: containers-common
  Depends: cri-o-runc
  Recommends: slirp4netns
  Recommends: containernetworking-plugins
  Recommends: uidmap

[lukengda]

I can reproduce it (for now): with the removal of runc the build fails again.

[TomSweeneyRedHat]

@lsm5 I'll let @nalind correct me, but I'm not seeing us pointing to runc in a particular location within our code. So I think we're just looking for it based on the defined $PATH, generally /usr/bin/runc. Can we change the path to /sbin/runc or is the problem that we need one variant of runc and some other package needs another?

[rhatdan]

Any reason we need it at all for building the package? It is only needed for tests? I would like to get to the point where we could easily substitute crun.

Perhaps we should make buildah and podman require an ociruntime, and then make runc and crun packages provide an ociruntime.

[lsm5]

@lsm5 I'll let @nalind correct me, but I'm not seeing us pointing to runc in a particular location within our code. So I think we're just looking for it based on the defined $PATH, generally /usr/bin/runc. Can we change the path to /sbin/runc or is the problem that we need one variant of runc and some other package needs another?

So, prior reason for packaging cri-o-runc was that distro's runc was too old. I haven't kept track of how that has changed, if we find out that the default runc is now good enough, then maybe we don't need to maintain our own anymore. @TomSweeneyRedHat /usr/bin/runc is used by docker-ce and /usr/sbin/runc is used by default runc, and we've had other issues where people complained our cri-o-runc conflicted with docker-ce and runc.

@rhatdan @giuseppe should I give crun a try as a runc replacement on ubuntu? Any recommended tests before I can ship it?

[nalind]

@lsm5 I'll let @nalind correct me, but I'm not seeing us pointing to runc in a particular location within our code. So I think we're just looking for it based on the defined $PATH, generally /usr/bin/runc.

That's correct, give or take the --runtime flag.

[nalind]

So, prior reason for packaging cri-o-runc was that distro's runc was too old. I haven't kept track of how that has changed, if we find out that the default runc is now good enough, then maybe we don't need to maintain our own anymore.

I think the last time we needed a minimum runc version, it was v1.0.0-rc4, and bionic currently installs v1.0.0-rc7, so we should be fine using the distro-provided one.

[junaruga]

I wanted to care about the runc version on Ubuntu xenial. Because xenial is the latest Ubuntu used for Travis CI. As the runc version is also v1.0.0-rc7 on xenial, it looks good. https://packages.ubuntu.com/xenial/runc

[lukeyeager]

Workarounds:

1. sudo ln -s /usr/lib/cri-o-runc/sbin/runc /usr/sbin/runc
2. PATH=$PATH:/usr/lib/cri-o-runc/sbin
3. apt-get install runc (on bionic the packaged version is apparently new enough)

I'm leaning towards option 3 for now since it's the cleanest. Questions:

1. Any reason to suspect that bionic's runc will be problematic?

   ii  cri-o-runc                              1.0.0-rc8-1~ubuntu18.04~ppa1                    
   ii  runc                                    1.0.0~rc7+git20190403.029124da-0ubuntu1~18.04.2 
   

2. When is this going to get fixed?

[rhatdan]

I don't believe we have this issue any longer, reopen if I am mistaken.