podman load from buildctl using type oci results in incorrect name

[matt0x6F]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

Images produced with buildctl are not resulting in the specified names when loaded into podman. This issue has been discussed before and was thought to be an issue with BuildKit (#11619). After speaking with the BuildKit team, however, it looks like other tooling like containerd implement workarounds for this.

Steps to reproduce the issue:

1. buildctl --addr=podman-container://buildkitd build --frontend dockerfile.v0 --local context=. --local dockerfile=. --export-cache type=inline --frontend-opt filename=Dockerfile --output type=oci,name=github.com/mattouille/whatever,dest=/tmp/whatever_image.tar

2. podman load -i /tmp/whatever_image.tar

3. podman images -a

Note: You can also use the stdout/in feature of podman and buildkit. The result is the same.

Describe the results you received:

The image name, regardless of the name specified, is localhost/latest:latest

Describe the results you expected:

The image name is github.com/mattouille/whatever:latest

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Client:
Version:      3.4.1
API Version:  3.4.1
Go Version:   go1.17.2
Built:        Tue Oct 19 14:14:42 2021
OS/Arch:      darwin/amd64

Server:
Version:      3.4.2
API Version:  3.4.2
Go Version:   go1.16.8
Built:        Fri Nov 12 12:25:37 2021
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.23.1
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.30-2.fc35.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.30, commit: '
  cpus: 1
  distribution:
    distribution: fedora
    variant: coreos
    version: "35"
  eventLogger: journald
  hostname: localhost.localdomain
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.15.6-200.fc35.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 1259782144
  memTotal: 2061389824
  ociRuntime:
    name: crun
    package: crun-1.3-1.fc35.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.3
      commit: 8e5757a4e68590326dafe8a8b1b4a584b10a1370
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc35.x86_64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 7h 12m 27.34s (Approximately 0.29 days)
plugins:
  log:
  - k8s-file
  - none
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 7
    paused: 0
    running: 1
    stopped: 6
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/core/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 5
  runRoot: /run/user/1000/containers
  volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
  APIVersion: 3.4.2
  Built: 1636748737
  BuiltTime: Fri Nov 12 20:25:37 2021
  GitCommit: ""
  GoVersion: go1.16.8
  OsArch: linux/amd64
  Version: 3.4.2

Package info (e.g. output of rpm -q podman or apt list podman):

N/A. Installed from source.

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

Physical

[rhatdan]

@vrothberg @mtrmac PTAL

[vrothberg]

tl;dr
I think we can support these custom files but note that buildkit is not 100 percent in line with the OCI spec, more below.

buildkit is adding a manifest.json file for OCI images which shouldn't be there but only in archives in the Docker format. That makes it harder to "guess" in which format the tar ball actually is (OCI or Docker). The OCI spec does not explicitly forbid adding more files to the layout but there is also no statement if it's OK (see https://github.com/opencontainers/image-spec/blob/main/image-layout.md#content).

Looking at the index.json:

{
  "schemaVersion": 2,
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:e3b23b57855e132673409165a5a6559180c32d273399f13cf078163fea7398a0",
      "size": 505,
      "annotations": {
        "io.containerd.image.name": "github.com/mattouille/whatever:latest",
        "org.opencontainers.image.created": "2021-12-10T10:19:54Z",
        "org.opencontainers.image.ref.name": "latest"
      }
    }
  ]
}

The ref.name annotation states "latest" and that's what Podman is using. Having a custom annotations as above (i.e., io.containerd.image.name) is custom behavior of containerd. Podman confirms with the image-spec (see https://github.com/opencontainers/image-spec/blob/main/annotations.md#pre-defined-annotation-keys).

It would be trivial to support that but I am convinced that buildkit should fix that and properly set the org.opencontainers.image.ref.name. This way, buildkit and containerd can use the custom annotation and others can use what the OCI spec wants.

[vrothberg]

containers/common#853 adds support for buildkit archives.