Unable to build index images with opm/podman - you must use just one container

[bfabec]

When trying to build a catalog index with the latest opm 1.14.3 with podman 2.1.1 container tool, we are getting the following error:

opm index add -c podman --bundles docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20 --tag noi-operator-catalog:test
INFO[0000] building the index                            bundles="[docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20]"
INFO[0000] running /usr/bin/podman pull docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20  bundles="[docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20]"
INFO[0002] running podman create                         bundles="[docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20]"
INFO[0002] running podman cp                             bundles="[docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20]"
ERRO[0002] Error: invalid arguments time="2020-10-23T20:37:18Z" level=error msg="unable to write pod event: \"write unixgram @00036->/run/systemd/journal/socket: sendmsg: no such file or directory\""
5dd2ba05f488a980e96b48b26533e62a6c61d07c5d408cc6b3c4f5c2d425c831:/., bundle_tmp862213345 you must use just one container  bundles="[docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20]"
ERRO[0002] permissive mode disabled                      bundles="[docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20]" error="error copying container directory Error: invalid arguments time=\"2020-10-23T20:37:18Z\" level=error msg=\"unable to write pod event: \\\"write unixgram @00036->/run/systemd/journal/socket: sendmsg: no such file or directory\\\"\"\n5dd2ba05f488a980e96b48b26533e62a6c61d07c5d408cc6b3c4f5c2d425c831:/., bundle_tmp862213345 you must use just one container\n: exit status 125"
Error: error copying container directory Error: invalid arguments time="2020-10-23T20:37:18Z" level=error msg="unable to write pod event: \"write unixgram @00036->/run/systemd/journal/socket: sendmsg: no such file or directory\""
5dd2ba05f488a980e96b48b26533e62a6c61d07c5d408cc6b3c4f5c2d425c831:/., bundle_tmp862213345 you must use just one container
: exit status 125
Usage:
  opm index add [flags]

Examples:
  # Create an index image from scratch with a single bundle image
  opm index add --bundles quay.io/operator-framework/operator-bundle-prometheus@sha256:a3ee653ffa8a0d2bbb2fabb150a94da6e878b6e9eb07defd40dc884effde11a0 --tag quay.io/operator-framework/monitoring:1.0.0
  
  # Add a single bundle image to an index image
  opm index add --bundles quay.io/operator-framework/operator-bundle-prometheus:0.15.0 --from-index quay.io/operator-framework/monitoring:1.0.0 --tag quay.io/operator-framework/monitoring:1.0.1
  
  # Add multiple bundles to an index and generate a Dockerfile instead of an image
  opm index add --bundles quay.io/operator-framework/operator-bundle-prometheus:0.15.0,quay.io/operator-framework/operator-bundle-prometheus:0.22.2 --generate

Flags:
  -i, --binary-image opm        container image for on-image opm command
  -u, --build-tool string       tool to build container images. One of: [docker, podman]. Defaults to podman. Overrides part of container-tool.
  -b, --bundles strings         comma separated list of bundles to add
  -c, --container-tool string   tool to interact with container images (save, build, etc.). One of: [docker, podman]
  -f, --from-index string       previous index to add to
      --generate                if enabled, just creates the dockerfile and saves it to local disk
  -h, --help                    help for add
      --mode string             graph update mode that defines how channel graphs are updated. One of: [replaces, semver, semver-skippatch] (default "replaces")
  -d, --out-dockerfile string   if generating the dockerfile, this flag is used to (optionally) specify a dockerfile name
      --permissive              allow registry load errors
  -p, --pull-tool string        tool to pull container images. One of: [none, docker, podman]. Defaults to none. Overrides part of container-tool.
  -t, --tag string              custom tag for container image being built

Global Flags:
      --skip-tls   skip TLS certificate verification for container image registries while pulling bundles or index

opm version
Version: version.Version{OpmVersion:"v1.14.3", GitCommit:"5f55ec2", BuildDate:"2020-10-23T20:04:11Z", GoOs:"linux", GoArch:"amd64"}

podman version
Version:      2.1.1
API Version:  2.0.0
Go Version:   go1.15.2
Built:        Thu Jan  1 00:00:00 1970
OS/Arch:      linux/amd64

The same command works but with the docker container tool:

opm.1.14.3 index add -c docker --bundles docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20 --tag noi-operator-catalog:test
INFO[0000] building the index                            bundles="[docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20]"
INFO[0000] running /usr/local/bin/docker pull docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20  bundles="[docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20]"
INFO[0003] running docker create                         bundles="[docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20]"
INFO[0003] running docker cp                             bundles="[docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20]"
INFO[0003] running docker rm                             bundles="[docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20]"
INFO[0003] Could not find optional dependencies file     dir=bundle_tmp239990534 file=bundle_tmp239990534/metadata load=annotations
INFO[0003] found csv, loading bundle                     dir=bundle_tmp239990534 file=bundle_tmp239990534/manifests load=bundle
INFO[0003] loading bundle file                           dir=bundle_tmp239990534/manifests file=noi-operator.v1.0.0.clusterserviceversion.yaml load=bundle
INFO[0003] loading bundle file                           dir=bundle_tmp239990534/manifests file=noi.ibm.com_noiconnectionlayers_crd.yaml load=bundle
INFO[0003] loading bundle file                           dir=bundle_tmp239990534/manifests file=noi.ibm.com_noiformations_crd.yaml load=bundle
INFO[0003] loading bundle file                           dir=bundle_tmp239990534/manifests file=noi.ibm.com_noihybrids_crd.yaml load=bundle
INFO[0003] loading bundle file                           dir=bundle_tmp239990534/manifests file=noi.ibm.com_nois_crd.yaml load=bundle
INFO[0003] Generating dockerfile                         bundles="[docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20]"
INFO[0003] writing dockerfile: index.Dockerfile298626996  bundles="[docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20]"
INFO[0003] running docker build                          bundles="[docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20]"
INFO[0003] [docker build -f index.Dockerfile298626996 -t noi-operator-catalog:test .]  bundles="[docker.io/ibmcom/noi-operator-bundle@sha256:c5bf168b02470d48dd86281d0e12791c406fb00e3775a3f69fa22670217d8f20]"

Checking the previous versions of OPM, this used to work up until 1.12.6, 1.12.5 or below doesn't have this issue.

[bfabec]

FYI, I was able to resolve this by following containers/podman#4325. In summary, update the /usr/share/containers/containers.conf or libpod.conf and add events_logger = "file"