podman-remote: run --rm: Error removing container

[edsantiago]

Another podman-remote flake, probably a race condition. I don't think this is the same as #7195 because when it happens I actually get output.

$ podman rm -a;podman volume rm -a
$ expect=$(./bin/podman-remote run --rm -v testvol1:/etc/apk alpine_labels ls /etc/apk)
$ while :;do got=$(./bin/podman-remote run --rm -v testvol1:/etc/apk alpine_labels ls /etc/apk); test "$got" = "$expect" || echo "got: $got";done
ERRO[0000] Error removing container 96e24969a069225db3cfdc27a724172fac29465ac911fa2974f68a94f3706b28: no container with name or ID 96e24969a069225db3cfdc27a724172fac29465ac911fa2974f68a94f3706b28 found: no such container
ERRO[0000] Error removing container e8834ac68b23e1720beed42e68b3e6de250629af7bd0295b9c9f62bf179c4912: container e8834ac68b23e1720beed42e68b3e6de250629af7bd0295b9c9f62bf179c4912 does not exist in database: no such container
ERRO[0000] Error removing container 3e4fcf50ba72d454ba81548dbfdff17d10a439001bb1a28a76f7165ede45eb4d: no container with name or ID 3e4fcf50ba72d454ba81548dbfdff17d10a439001bb1a28a76f7165ede45eb4d found: no such container
ERRO[0000] Error removing container 933bb8451d833c6c04399315edb91a8fa4900991f4f4f8f73c5cf0ac9e5f6ff9: container 933bb8451d833c6c04399315edb91a8fa4900991f4f4f8f73c5cf0ac9e5f6ff9 does not exist in database: no such container
^C

Note that there are two distinct error messages:

• no container with name or ID ...
• container ... does not exist in database

Exit status, FWIW, is 0 even with the error.

master @ 47108e2, root and rootless

I've seen it even without the -v testvol, but it reproduces much more quickly with -v. As in, less than five seconds to get an error message on my f32 laptop.

[edsantiago]

CI flakes in the field:

Podman run with volumes [It] podman named volume copyup

• fedora-32 : special_testing_rootless
  • PR fix pod creation with "new:" syntax #7152
    • 07-31 13:34
    • 07-31 13:34
    • 07-31 13:34
• ubuntu-19 : test ubuntu-19
  • PR Change /sys/fs/cgroup/systemd mount to rprivate #7306
    • 08-12 10:33
    • 08-12 10:33
    • 08-12 10:33

(The above might also show evidence of #7195. I haven't looked closely enough yet)

[vrothberg]

Looking into this now. Thanks a lot for providing all the details, @edsantiago !

[vrothberg]

Looking at the remote code I found that --rm --detach is broken. Will prepare a fix for that as well.

[vrothberg]

#7561 to fix both issues