IPFS: layer not found - follow-up to #3425

[apostasie]

Description

The patch in #3435 did not address the case of IPFS push, which may be a specialized code path, or at least require special treatment compared to normal push.

ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	84.403s
ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	87.423s
ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	84.780s
ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	76.292s
ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	78.071s
ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	74.118s
ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	93.913s
ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	74.331s
ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	77.883s
ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	77.921s
ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	75.057s
ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	81.912s
ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	75.338s
ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	72.620s
ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	84.847s
ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	72.791s
ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	87.423s
ok  	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	73.117s
test target: "nerdctl"
time="2024-10-02T18:12:25-07:00" level=error msg="`buildctl` needs to be installed and `buildkitd` needs to be running, see https://github.com/moby/buildkit , and `containerd-rootless-setuptool.sh install-buildkit` for OCI worker or `containerd-rootless-setuptool.sh install-buildkit-containerd` for containerd worker" error="failed to ping to host unix:///run/user/501/buildkit-nerdctl-test/buildkitd.sock: exit status 1\nfailed to ping to host unix:///run/user/501/buildkit-default/buildkitd.sock: exit status 1\nfailed to ping to host unix:///run/user/501/buildkit/buildkitd.sock: exit status 1"
--- FAIL: TestIPFSComposeUpNoBuildDefault (5.49s)
    helpers.go:40: /usr/local/bin/nerdctl --namespace=nerdctl-test pull --quiet ghcr.io/stargz-containers/registry:2-org
    helpers.go:40: /usr/local/bin/nerdctl --namespace=nerdctl-test pull --quiet ghcr.io/stargz-containers/cesanta/docker_auth:1.7-org
    helpers.go:40: /usr/local/bin/nerdctl --namespace=nerdctl-test pull --quiet ghcr.io/stargz-containers/ipfs/kubo:v0.16.0-org
    helpers.go:40: /usr/local/bin/nerdctl --namespace=nerdctl-test rm -f testipfscomposeupnobuilddefault-kubo-registry-server-5000-false-25b9f3c7
    helpers.go:40: /usr/local/bin/nerdctl --namespace=nerdctl-test run --pull=never -d -p 0.0.0.0:5000:5000 --name testipfscomposeupnobuilddefault-kubo-registry-server-5000-false-25b9f3c7 --entrypoint=/bin/sh ghcr.io/stargz-containers/ipfs/kubo:v0.16.0-org -c -- ipfs init && ipfs config Addresses.API /ip4/0.0.0.0/tcp/5000 && ipfs daemon --offline
    cesanta.go:93: /usr/local/bin/nerdctl --namespace=nerdctl-test container inspect testipfscomposeupnobuilddefault-kubo-registry-server-5000-false-25b9f3c7
    helpers.go:60: /usr/local/bin/nerdctl --namespace=nerdctl-test logs testipfscomposeupnobuilddefault-kubo-registry-server-5000-false-25b9f3c7
    helpers.go:40: /usr/local/bin/nerdctl --namespace=nerdctl-test pull --quiet ghcr.io/stargz-containers/wordpress:5.7-org
    helpers.go:40: /usr/local/bin/nerdctl --namespace=nerdctl-test pull --quiet ghcr.io/stargz-containers/mariadb:10.5-org
    ipfs_registry_linux_test.go:39: /usr/local/bin/nerdctl --namespace=nerdctl-test push ipfs://ghcr.io/stargz-containers/wordpress:5.7-org
    ipfs_registry_linux_test.go:39: assertion failed: expect.ExitCode is not result.ExitCode: Expected exit code: 0

        Command:  /usr/local/bin/nerdctl --namespace=nerdctl-test push ipfs://ghcr.io/stargz-containers/wordpress:5.7-org
        ExitCode: 1
        Error:    exit status 1
        Stdout:
        Stderr:   time="2024-10-02T18:12:29-07:00" level=info msg="pushing image \"ghcr.io/stargz-containers/wordpress:5.7-org\" to IPFS"
        time="2024-10-02T18:12:29-07:00" level=info msg="ensuring image contents"
        time="2024-10-02T18:12:30-07:00" level=warning msg="ipfs push failed" error="content digest sha256:5625a2e212348c7f566e6686e3706a49f9ba12eb8f129c74fcc5b730939de178: not found"
        time="2024-10-02T18:12:30-07:00" level=fatal msg="content digest sha256:5625a2e212348c7f566e6686e3706a49f9ba12eb8f129c74fcc5b730939de178: not found"

        Env:
        SHELL=/bin/bash
        LOGNAME=dmp
        XDG_SESSION_TYPE=tty
        HOME=/home/dmp.linux
        LANG=C.UTF-8
        SSH_CONNECTION=192.168.5.2 51297 192.168.5.15 22
        LESSCLOSE=/usr/bin/lesspipe %s %s
        XDG_SESSION_CLASS=user
        TERM=xterm-256color
        LESSOPEN=| /usr/bin/lesspipe %s
        USER=dmp
        SHLVL=2
        XDG_SESSION_ID=2
        XDG_RUNTIME_DIR=/run/user/501
        SSH_CLIENT=192.168.5.2 51297 22
        XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop
        DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/501/bus
        SSH_TTY=/dev/pts/1
        OLDPWD=/Users/dmp
        _=/usr/local/go/bin/go
        PATH=/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin:/sbin:/usr/sbin:/sbin:/usr/sbin:/sbin:/usr/local/go/bin
        PWD=/Users/dmp/Projects/go/nerd/nerdctl/cmd/nerdctl/ipfs
        IPFS_PATH=/home/dmp.linux/.local/share/ipfs
        DOCKER_CONFIG=/tmp/TestIPFSComposeUpNoBuildDefault1952484318/001
        NERDCTL_TOML=/tmp/TestIPFSComposeUpNoBuildDefault1952484318/001/nerdctl.toml
    helpers.go:47: /usr/local/bin/nerdctl --namespace=nerdctl-test rm -f testipfscomposeupnobuilddefault-kubo-registry-server-5000-false-25b9f3c7
    helpers.go:47: /usr/local/bin/nerdctl --namespace=nerdctl-test rmi
    helpers.go:47: /usr/local/bin/nerdctl --namespace=nerdctl-test rmi
time="2024-10-02T18:12:31-07:00" level=error msg="`buildctl` needs to be installed and `buildkitd` needs to be running, see https://github.com/moby/buildkit , and `containerd-rootless-setuptool.sh install-buildkit` for OCI worker or `containerd-rootless-setuptool.sh install-buildkit-containerd` for containerd worker" error="failed to ping to host unix:///run/user/501/buildkit-nerdctl-test/buildkitd.sock: exit status 1\nfailed to ping to host unix:///run/user/501/buildkit-default/buildkitd.sock: exit status 1\nfailed to ping to host unix:///run/user/501/buildkit/buildkitd.sock: exit status 1"
FAIL
FAIL	github.com/containerd/nerdctl/v2/cmd/nerdctl/ipfs	80.191s
FAIL
FAIL

Steps to reproduce the issue

Hammer the tests repeatedly until they bleed...

Describe the results you received and expected

na

What version of nerdctl are you using?

main

Are you using a variant of nerdctl? (e.g., Rancher Desktop)

None

Host information

No response

[apostasie]

Note: enforcing content retrieval for the image does not seem to be enough.

Next come:

        time="2024-10-02T22:27:59-07:00" level=warning msg="ipfs push failed" error="Post \"http://192.168.5.15:5001/api/v0/add?cid-version=1&pin=true\": rpc error: code = NotFound desc = content digest sha256:378f1d8abdcf9e9c8bab77adbb0689ab5e36d627121730332d227c794f08d6cf: not found"
        time="2024-10-02T22:27:59-07:00" level=fatal msg="Post \"http://192.168.5.15:5001/api/v0/add?cid-version=1&pin=true\": rpc error: code = NotFound desc = content digest sha256:378f1d8abdcf9e9c8bab77adbb0689ab5e36d627121730332d227c794f08d6cf: not found"

Which happens at push time.

The digest presented here is not in the original image - it is the digest of the converted image though.

?