Add a “never modify the image” option to (skopeo copy)

[mtrmac]

Per containers/buildah#3377 , to support “publish the built artifact” workflows, skopeo copy (and probably skopeo sync) should have an option that makes a bit-exact copy, without changing the image representation (no {de,}compression, no format changes).

I.e. c/image/copy.Options should get an option to explicitly trigger the existing canModifyManifest == false behavior, and Skopeo should expose it.

[mtrmac]

This should also apply to manifest list copies (and they should get the same “destination is a digested reference” treatment, BTW).

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[zfrhv]

there is an option skopeo copy --all docker://registry01:5000/foo/bar@sha256:ff7c936109de38ae5776ccc9658453c2777e9a0c9d15b365f260e02d8a81a366 docker://registry02:5000/foo/bar@sha256:ff7c936109de38ae5776ccc9658453c2777e9a0c9d15b365f260e02d8a81a366, and it doesnt changes the digest.
but when using this option to save the image straight as a file, the digest does changes.
(i tried to use all 4 file formats: "docker-archive", "tarball", "dir", "oci-archive". and in all of them the digest is being changed)

which makes everything very difficult, because then i cant use most of the redhat images in restricted environment (for example openshift images), because redhat products pull the images by digest (and it changes when saving the images as a file), so the digest doesnt matches and everything fails.
and the workaround is very difficult. (i can tell if anyone is interested)

so can you please make that the --all option wouldnt change the digest also when saving the image into a file, and also when doing skopeo sync ¿

[mtrmac]

but when using this option to save the image straight as a file, the digest does changes.

Please file a separate report, including version numbers and steps to reproduce.

[konrad-ohms]

@zfrhv I guess your local workaround could be similar to containers/buildah#3377 (comment).

1. Use skopeo inspect to detect the digest
2. Copy the image to local directory by digest instead of the tag

If your deployment uses digests anyhow, I guess it should not care for the missing image tag and copy by digest should not change anything during the copy statement. It seem to work for multi-arch images as well.

Example:

$ skopeo inspect --format "{{.Digest}}" docker://registry.access.redhat.com/ubi8-micro:latest
sha256:16ce476bc3a97180501deb8dba1ac2b327a2e10f40a456e653f89ea15d8132ac

$ skopeo copy --all docker://registry.access.redhat.com/ubi8-micro@sha256:16ce476bc3a97180501deb8dba1ac2b327a2e10f40a456e653f89ea15d8132ac dir:ubi8-micro
Getting image list signatures
Copying 4 of 4 images in list
Copying image sha256:0dd5a0d59c981b471b618d145d06a988354babadcc7c5246c0cd7f1112ac089c (1/4)
Getting image source signatures
Checking if image destination supports signatures
Copying blob 4f4fb700ef54 done  
Copying blob 098a109c8679 done  
Copying config c5ba898d36 done  
Writing manifest to image destination
Storing signatures
Copying image sha256:a45b03f0813ace894aabe6e5b8e9511e2c04ba0370a99af1eefa0505df1ce940 (2/4)
Getting image source signatures
Checking if image destination supports signatures
Copying blob 4f4fb700ef54 skipped: already exists  
Copying blob afe8038a6ce5 done  
Copying config f80f27d51b done  
Writing manifest to image destination
Storing signatures
Copying image sha256:e113a7f3a3792f149aba4f4f1014e9aca40edcf47b7287246007f1a6b4b560c1 (3/4)
Getting image source signatures
Checking if image destination supports signatures
Copying blob 4f4fb700ef54 skipped: already exists  
Copying blob 73a281fddf15 done  
Copying config fcad606d24 done  
Writing manifest to image destination
Storing signatures
Copying image sha256:ee8f6d321f1612f43b429e99c6a90a22fee70a3b45648cad9c125338e9020471 (4/4)
Getting image source signatures
Checking if image destination supports signatures
Copying blob 4f4fb700ef54 skipped: already exists  
Copying blob 5a85248f6fa4 done  
Copying config 2d74d8374b done  
Writing manifest to image destination
Storing signatures
Writing manifest list to image destination
Storing list signatures

Comparison of results:

# original image
$ skopeo inspect docker://registry.access.redhat.com/ubi8-micro:latest
{
    "Name": "registry.access.redhat.com/ubi8-micro",
    "Digest": "sha256:16ce476bc3a97180501deb8dba1ac2b327a2e10f40a456e653f89ea15d8132ac",
    "RepoTags": [
        "8.4",
        "8.4-81-source",
        "8.4-72-source",
        "8.4-72",
        "8.4-84-source",
        "8.4-81",
        "8.4-84",
        "latest"
    ],
    "Created": "2021-08-03T16:22:55.109973263Z",
    "DockerVersion": "1.13.1",
    "Labels": {
        "architecture": "x86_64",
        "build-date": "2021-08-03T16:22:06.389689",
        "com.redhat.build-host": "cpt-1003.osbs.prod.upshift.rdu2.redhat.com",
        "com.redhat.component": "ubi8-micro-container",
        "com.redhat.license_terms": "https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI",
        "description": "Very small image which doesn't install the package manager.",
        "distribution-scope": "public",
        "io.k8s.description": "Very small image which doesn't install the package manager.",
        "io.k8s.display-name": "Ubi8-micro",
        "io.openshift.expose-services": "",
        "maintainer": "Red Hat, Inc.",
        "name": "ubi8/ubi-micro",
        "release": "84",
        "summary": "ubi8 micro image",
        "url": "https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/ubi-micro/images/8.4-84",
        "vcs-ref": "c20f4a2add7d519164f7cf64842bc9f024d225ab",
        "vcs-type": "git",
        "vendor": "Red Hat, Inc.",
        "version": "8.4"
    },
    "Architecture": "amd64",
    "Os": "linux",
    "Layers": [
        "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1",
        "sha256:098a109c86792f7772205abf2d76ada8a8db27761809c8924674cd38da355524"
    ],
    "Env": [
        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ]
}

# local image
$ skopeo inspect dir:ubi8-micro
{
    "Digest": "sha256:16ce476bc3a97180501deb8dba1ac2b327a2e10f40a456e653f89ea15d8132ac",
    "RepoTags": [],
    "Created": "2021-08-03T16:22:55.109973263Z",
    "DockerVersion": "1.13.1",
    "Labels": {
        "architecture": "x86_64",
        "build-date": "2021-08-03T16:22:06.389689",
        "com.redhat.build-host": "cpt-1003.osbs.prod.upshift.rdu2.redhat.com",
        "com.redhat.component": "ubi8-micro-container",
        "com.redhat.license_terms": "https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI",
        "description": "Very small image which doesn't install the package manager.",
        "distribution-scope": "public",
        "io.k8s.description": "Very small image which doesn't install the package manager.",
        "io.k8s.display-name": "Ubi8-micro",
        "io.openshift.expose-services": "",
        "maintainer": "Red Hat, Inc.",
        "name": "ubi8/ubi-micro",
        "release": "84",
        "summary": "ubi8 micro image",
        "url": "https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/ubi-micro/images/8.4-84",
        "vcs-ref": "c20f4a2add7d519164f7cf64842bc9f024d225ab",
        "vcs-type": "git",
        "vendor": "Red Hat, Inc.",
        "version": "8.4"
    },
    "Architecture": "amd64",
    "Os": "linux",
    "Layers": [
        "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1",
        "sha256:098a109c86792f7772205abf2d76ada8a8db27761809c8924674cd38da355524"
    ],
    "Env": [
        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ]
}

# file structure on disk
$ ls ubi8-micro/
098a109c86792f7772205abf2d76ada8a8db27761809c8924674cd38da355524                c5ba898d36450f8f4b0e04f5b06d794f53d96283c14e5f7df0bcd805c6fe1c08
0dd5a0d59c981b471b618d145d06a988354babadcc7c5246c0cd7f1112ac089c.manifest.json  e113a7f3a3792f149aba4f4f1014e9aca40edcf47b7287246007f1a6b4b560c1.manifest.json
0dd5a0d59c981b471b618d145d06a988354babadcc7c5246c0cd7f1112ac089c.signature-1    e113a7f3a3792f149aba4f4f1014e9aca40edcf47b7287246007f1a6b4b560c1.signature-1
0dd5a0d59c981b471b618d145d06a988354babadcc7c5246c0cd7f1112ac089c.signature-2    e113a7f3a3792f149aba4f4f1014e9aca40edcf47b7287246007f1a6b4b560c1.signature-2
0dd5a0d59c981b471b618d145d06a988354babadcc7c5246c0cd7f1112ac089c.signature-3    e113a7f3a3792f149aba4f4f1014e9aca40edcf47b7287246007f1a6b4b560c1.signature-3
0dd5a0d59c981b471b618d145d06a988354babadcc7c5246c0cd7f1112ac089c.signature-4    e113a7f3a3792f149aba4f4f1014e9aca40edcf47b7287246007f1a6b4b560c1.signature-4
0dd5a0d59c981b471b618d145d06a988354babadcc7c5246c0cd7f1112ac089c.signature-5    e113a7f3a3792f149aba4f4f1014e9aca40edcf47b7287246007f1a6b4b560c1.signature-5
0dd5a0d59c981b471b618d145d06a988354babadcc7c5246c0cd7f1112ac089c.signature-6    e113a7f3a3792f149aba4f4f1014e9aca40edcf47b7287246007f1a6b4b560c1.signature-6
2d74d8374b38e907550cdd664d95914f6ea056a6d48c912f21aea3cfa1a2d4dd                ee8f6d321f1612f43b429e99c6a90a22fee70a3b45648cad9c125338e9020471.manifest.json
4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1                ee8f6d321f1612f43b429e99c6a90a22fee70a3b45648cad9c125338e9020471.signature-1
5a85248f6fa4a219ccd8e0b9423bf6f6eaab32b16a4b14fdbb34c2ef80cd69bf                ee8f6d321f1612f43b429e99c6a90a22fee70a3b45648cad9c125338e9020471.signature-2
73a281fddf1551e72da18a7c9a82f4ac5ab48b5c852a5021f3a5db847b153a03                ee8f6d321f1612f43b429e99c6a90a22fee70a3b45648cad9c125338e9020471.signature-3
a45b03f0813ace894aabe6e5b8e9511e2c04ba0370a99af1eefa0505df1ce940.manifest.json  ee8f6d321f1612f43b429e99c6a90a22fee70a3b45648cad9c125338e9020471.signature-4
a45b03f0813ace894aabe6e5b8e9511e2c04ba0370a99af1eefa0505df1ce940.signature-1    ee8f6d321f1612f43b429e99c6a90a22fee70a3b45648cad9c125338e9020471.signature-5
a45b03f0813ace894aabe6e5b8e9511e2c04ba0370a99af1eefa0505df1ce940.signature-2    ee8f6d321f1612f43b429e99c6a90a22fee70a3b45648cad9c125338e9020471.signature-6
a45b03f0813ace894aabe6e5b8e9511e2c04ba0370a99af1eefa0505df1ce940.signature-3    f80f27d51b89c254997c8b712f5d9865b3dd5a7464b8810636bef3e8e775a513
a45b03f0813ace894aabe6e5b8e9511e2c04ba0370a99af1eefa0505df1ce940.signature-4    fcad606d245baaf948b8fb9937b406e0506610c5404e966d68b756f4983f1547
a45b03f0813ace894aabe6e5b8e9511e2c04ba0370a99af1eefa0505df1ce940.signature-5    manifest.json
a45b03f0813ace894aabe6e5b8e9511e2c04ba0370a99af1eefa0505df1ce940.signature-6    version
afe8038a6ce51c0ca9f3c6787c1928ad9b19f3036bc0a00adefb0704dd8a7619

[zfrhv]

@konrad-ohms yes 👍, or if copying a lot of images at the same time then its better to use registry, as I explained in the new case #1440 (comment)

[github-actions]

A friendly reminder that this issue had no activity for 30 days.