manifest add/annotate option doesn't look to work.

[sstosh]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

1. .manifests[].annotations field looks failed to display by podman manifest inspect.
2. When we use --feature with --annotate, --feature is ignored.

I think no.2 issue related to MIME type.
annotations field exist only oci type.
On the other hands, features field exists only docker type.
Therefore, --features should not be used with --annotation.

Steps to reproduce the issue:

manifest annotate

# podman manifest create test
5d7fcc56dea2c8e7d3890d559da0a87113187021d13274a618bfb4af735b6cb5

# podman manifest add test quay.io/libpod/testimage:20220615
5d7fcc56dea2c8e7d3890d559da0a87113187021d13274a618bfb4af735b6cb5

# podman images
REPOSITORY               TAG         IMAGE ID      CREATED         SIZE
localhost/test           latest      5d7fcc56dea2  37 seconds ago  479 B

# export SHA1=$(podman manifest inspect test | jq -r '.manifests[].digest')

# echo $SHA1
sha256:a07e678985cd67330fd30a4c8a008cefeb1ca917c25ab0f8468ea17c7b34beb2

# podman manifest annotate --annotation annotationTest1=annotationTest2 \
                                 --arch archTest --variant variantTest \
                                 --os osTest --os-version osVersionTest \
                                 --os-features osFeaturesTest \
                                 --features featuresTest \
                                 test $SHA1
5d7fcc56dea2c8e7d3890d559da0a87113187021d13274a618bfb4af735b6cb5

manifest add

# podman manifest create test
8f21a96ba85377fc7542c8ab9de3bac9207482f440ac4874a94b6ba80dd68066

# podman manifest add --annotation annotationTest1=annotationTest2 \
                            --arch archTest --variant variantTest \
                            --os osTest --os-version osVersionTest \
                            --features featuresTest \
                            test quay.io/libpod/testimage:20220615
8f21a96ba85377fc7542c8ab9de3bac9207482f440ac4874a94b6ba80dd68066

Describe the results you received:

# podman manifest inspect test
{
    "schemaVersion": 2,
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "manifests": [
        {
            "mediaType": "application/vnd.oci.image.manifest.v1+json",
            "size": 758,
            "digest": "sha256:a07e678985cd67330fd30a4c8a008cefeb1ca917c25ab0f8468ea17c7b34beb2",
            "platform": {
                "architecture": "archTest",
                "os": "osTest",
                "os.version": "osVersionTest",
                "variant": "variantTest"
            }
        }
    ]
}

Describe the results you expected:

1. podman manifest inspect display .manifests[].annotations field .

# podman manifest inspect test
{
    "schemaVersion": 2,
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "manifests": [
        {
            "mediaType": "application/vnd.oci.image.manifest.v1+json",
            "size": 758,
            "digest": "sha256:a07e678985cd67330fd30a4c8a008cefeb1ca917c25ab0f8468ea17c7b34beb2",
            "annotations": {
              "annotationTest1": "annotationTest2"
            },
            "platform": {
                "architecture": "archTest",
                "os": "osTest",
                "os.version": "osVersionTest",
                "variant": "variantTest"
            }
        }
    ]
}

2. Prohibit to use --feature with --annotate, or output warning message.

# podman manifest annotate --annotation annotationTest1=annotationTest2 \
                                 --features featuresTest \
                                 test $SHA1
Error: --annotation and --features cannot be used together

Additional information you deem important (e.g. issue happens only occasionally):

The local manifest file exists annotations field.

# cat /var/lib/containers/storage/overlay-images/8f21a96ba85377fc7542c8ab9de3bac9207482f440ac4874a94b6ba80dd68066/manifest | jq -r
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:a07e678985cd67330fd30a4c8a008cefeb1ca917c25ab0f8468ea17c7b34beb2",
      "size": 758,
      "annotations": {
        "annotationTest1": "annotationTest2"
      },
      "platform": {
        "architecture": "archTest",
        "os": "osTest",
        "os.version": "osVersionTest",
        "variant": "variantTest"
      }
    }
  ]
}

Output of podman version:

version

# podman version
Client:       Podman Engine
Version:      4.2.0-dev
API Version:  4.2.0-dev
Go Version:   go1.18.3
Git Commit:   da1f47921685f40f1b26405278cbf9cb2d06fe09
Built:        Mon Jul 25 09:40:45 2022
OS/Arch:      linux/amd64

Output of podman info --debug:

info

# podman info --debug
host:
  arch: amd64
  buildahVersion: 1.27.0-dev
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.0-2.fc36.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.0, commit: '
  cpuUtilization:
    idlePercent: 99.73
    systemPercent: 0.15
    userPercent: 0.12
  cpus: 12
  distribution:
    distribution: fedora
    variant: server
    version: "36"
  eventLogger: journald
  hostname: fedora36
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.18.11-200.fc36.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 845832192
  memTotal: 8326590464
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.4.5-1.fc36.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.4.5
      commit: c381048530aa750495cf502ddb7181f2ded5b400
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.0-0.2.beta.0.fc36.x86_64
    version: |-
      slirp4netns version 1.2.0-beta.0
      commit: 477db14a24ff1a3de3a705e51ca2c4c1fe3dda64
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 8313630720
  swapTotal: 8325689344
  uptime: 93h 51m 48.00s (Approximately 3.88 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 106285760512
  graphRootUsed: 23394533376
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 1
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.2.0-dev
  Built: 1658709645
  BuiltTime: Mon Jul 25 09:40:45 2022
  GitCommit: da1f47921685f40f1b26405278cbf9cb2d06fe09
  GoVersion: go1.18.3
  Os: linux
  OsArch: linux/amd64
  Version: 4.2.0-dev

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):
Fedora36, KVM

[vrothberg]

@flouthoc can you take a look?

[flouthoc]

@vrothberg Sure.

[flouthoc]

Found the issue: by default we generate inspect output with format: docker which does not contains annotation and not according to OCI image spec switching this default format to OCI works for me but we would need to decide a default or probably accept a flag --format which can accept either docker or oci.

Patch which fixed for me:

diff --git a/vendor/github.com/containers/common/libimage/manifest_list.go b/vendor/github.com/containers/common/libimage/manifest_list.go
index 4480df548..d68bb6aa8 100644
--- a/vendor/github.com/containers/common/libimage/manifest_list.go
+++ b/vendor/github.com/containers/common/libimage/manifest_list.go
@@ -14,6 +14,7 @@ import (
        "github.com/containers/image/v5/types"
        "github.com/containers/storage"
        "github.com/opencontainers/go-digest"
+       v1 "github.com/opencontainers/image-spec/specs-go/v1"
 )
 
 // NOTE: the abstractions and APIs here are a first step to further merge
@@ -210,8 +211,8 @@ func (i *Image) IsManifestList(ctx context.Context) (bool, error) {
 }
 
 // Inspect returns a dockerized version of the manifest list.
-func (m *ManifestList) Inspect() (*manifest.Schema2List, error) {
-       return m.list.Docker(), nil
+func (m *ManifestList) Inspect() (*v1.Index, error) {
+       return m.list.OCIv1(), nil
 }
 
 // Options for adding a manifest list.

Output of inspect

[fl@fedora bin]$ podman manifest inspect test
{
    "schemaVersion": 2,
    "mediaType": "application/vnd.oci.image.index.v1+json",
    "manifests": [
        {
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "digest": "sha256:9b2a28eb47540823042a2ba401386845089bb7b62a9637d55816132c4c3c36eb",
            "size": 528,
            "annotations": {
                "one": "two"
            },
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            }
        }
    ]
}

[flouthoc]

From above comment #15069 (comment)

switching this default format to OCI works for me but we would need to decide a default or probably accept a flag --format which can accept either docker or oci

@vrothberg WDYT about this point. I am inclined to change default to OCI in libimage or should we give a concatinated output in inspect manifest for both the formats. i.e { "docker": <>, "oci": <> }

[flouthoc]

I also like the idea of a new flag manifest inspect --format <> <name>

[vrothberg]

I think we should follow what image inspect does and create our own struct. The struct would include all fields of both formats. I imagine it to be difficult to use a --format here when inspect should help figuring out which format the image/manifest is in.

[flouthoc]

I think we should follow what image inspect does and create our own struct. The struct would include all fields of both formats. I imagine it to be difficult to use a --format here when inspect should help figuring out which format the image/manifest is in.

@vrothberg Could this be a breaking change since it changes the output struct of manifest inspect ?

[vrothberg]

@flouthoc I think we can make it as the API returns raw bytes (i.e., []byte). As long as the raw JSON contains the same fields, we should be good. The idea would be to add some new fields that are mentioned in the OCI spec.

Would need to happen in libimage.