Docker image_id missing from docker.docker-info.json under buildx & containerd_snapshotter

[chris-smith-zocdoc]

Describe the bug

When enabling buildx support as described in https://www.pantsbuild.org/2.19/docs/docker#buildx-support with containerd-snapshotter: true, the docker.docker-info.json is missing the image_id field in the output. Its set to <missing>

Pants version
2.19.0

OS
MacOS

Additional info

Example docker.docker-info.json

{
    "version": 1,
    "image_id": "<unknown>",
    "registries": [
        {
            "alias": null,
            "address": "[redacted]",
            "repository": "my_app",
            "tags": [
                {
                    "template": "latest",
                    "tag": "latest",
                    "uses_local_alias": false,
                    "name": "[redacted]/my_app:latest"
                }
            ]
        }
    ]
}

Relevant logs from the build

#9 exporting to image
#9 exporting layers done
#9 exporting manifest sha256:7802087e8e0801f6451d862a00a6ce8af3e4829b09bc890dea0dd2659c11b25a done
#9 exporting config sha256:c83bed954709ba0c546d66d8f29afaac87c597f01b03fec158f3b21977c3e143 done
#9 exporting attestation manifest sha256:399891f9628cfafaba9e034599bdd55675ac0a3bad38151ed1ebf03993669545 done
#9 exporting manifest list sha256:b2b51838586286a9e544ddb31b3dbf7f6a99654d275b6e56b5f69f90138b4c0e done
#9 naming to <registry>/my_app:latest done
#9 unpacking to <registry>/my_app:latest done
#9 DONE 0.0s

The output under this configuration doesn't match either of the regexes listed in package_image.py

So this results in a non-match. Would the fix be to include an additional regex for exporting manifest list sha256:?

[huonw]

Thanks for filing an issue, and sorry for the trouble. I agree that an additional regex may be required... however, we'd need to confirm which of those are "image ID". Of particular note, there's significantly different behaviour between the image ID and the digest: #17299 (comment)

Can you confirm which (if any) of those allow you to do docker run sha256:abc123...? That's the image ID.

(It would be separately desirable to include the digest in its own field, when available... maybe one of those is the "digest", that allows docker run .../my_app@sha256:abc123...?)

[chris-smith-zocdoc]

In this case I believe these are all digests (which is why they're sha256: prefixed) but represent different pieces of metadata. The last line exporting manifest list contains the runnable id. Which you can use via all 3 variants in a run command

docker run b2b51838586286a9e544ddb31b3dbf7f6a99654d275b6e56b5f69f90138b4c0e
docker run sha256:b2b51838586286a9e544ddb31b3dbf7f6a99654d275b6e56b5f69f90138b4c0e
docker run registry/my_app@sha256:b2b51838586286a9e544ddb31b3dbf7f6a99654d275b6e56b5f69f90138b4c0e

Having the digest be available prior to push is new behavior with the containerd-snapshotter I think

[chris-smith-zocdoc]

You can see via docker inspect that the Id field is the same as the RepoDigests value, which previously wasn't true when building without pushing the image to a registry

docker inspect sha256:b2b51838586286a9e544ddb31b3dbf7f6a99654d275b6e56b5f69f90138b4c0e
[
    {
        "Id": "sha256:b2b51838586286a9e544ddb31b3dbf7f6a99654d275b6e56b5f69f90138b4c0e",
        "RepoTags": [
            "host/registry:latest"
        ],
        "RepoDigests": [
            "host/registry@sha256:b2b51838586286a9e544ddb31b3dbf7f6a99654d275b6e56b5f69f90138b4c0e"
        ],
        ...
    }
]

[huonw]

Ah, cool, my knowledge is out of date. Thanks for that. 👍

It looks like you've identified the right place for the fix and the right fix; do you want to add another regex and add another test case or two to test_parse_image_id_from_docker_build_output?

Do you happen to know much about multiplatform builds (something like --platform linux/amd64,linux/arm64)? If so, could you ensure the output of one of those is handled too?

[chris-smith-zocdoc]

If so, could you ensure the output of one of those is handled too?

I added a test case that covers the cross platform builds too. For those there is an additional manifest/config line per platform I think. The regex captures the final manifest list though, which is what you'd want for docker run and deployments