CI: flake: read unixpacket [conmon path]

[edsantiago]

As predicted, we're starting to see flakes in CI due to podman-remote tests being enabled. This one manifests as:

$ [some podman command]
[+0242s] # read unixpacket @->/var/run/libpod/socket/99a264c5e5fcaed668d0413741c63bfb90bdcdb02e1ad4ad109d7f6fe064fd84/attach: read: connection reset by peer

Seen in special testing rootless in #7224

Seen in fedora 31 test in #7222

I'm sorry not to be able to provide more information. I tried to reproduce this one last week and was unable to.

[edsantiago]

It is possible that this and #7195 are the same problem, but until there's evidence for that let's leave them as separate issues.

[edsantiago]

• fedora-32 : test fedora-32
  • PR fix bug podman sign storage path #7081
    • 08-04 16:39

[edsantiago]

Wow - a triple failure

[edsantiago]

See #7195 (comment) for a reproducer.

I'm still not convinced that this is the same as #7195: #7195 is "no output received, and sometimes there are other errors". This one is "read unixpacket error". The above reproducer triggers this error sometimes, but not reliably. Right now #7195 is the higher priority; until that one is understood, I still lean slightly toward keeping these two separate. (But as always I'm willing to listen to counterarguments).

[edsantiago]

Whoa - just now seeing this in podman gating tests, which don't even pretend to test podman-remote:

$ /usr/bin/podman run --rm build_test cat /subtest/myfile1
Error: error attaching to container 100864fd24cecb8c1cd7c736911e96bcfbf4e4125a4f5aecfaeb9695d28c8343: read unixpacket @->/run/user/1001/libpod/tmp/socket/100864fd24cecb8c1cd7c736911e96bcfbf4e4125a4f5aecfaeb9695d28c8343/attach: read: connection reset by peer
# [ rc=126 (** EXPECTED 0 **) ]

[edsantiago]

Another flake in Fedora gating tests:

 $ /usr/bin/podman run quay.io/libpod/alpine_labels:latest sh -c exit 32
Error: error attaching to container 9033150f125682b802f12e2463e825b767393d9fce698d08b1b84e3533e76d7b: read unixpacket @->/run/user/1001/libpod/tmp/socket/9033150f125682b802f12e2463e825b767393d9fce698d08b1b84e3533e76d7b/attach: read: connection reset by peer
[ rc=126 (** EXPECTED 32 **) ]

[edsantiago]

Yet another flake in Fedora gating tests:

$ /usr/bin/podman run --rm -i --preserve-fds=2 quay.io/libpod/alpine_labels:latest sh -c cat <&4
Error: error attaching to container 15a23c97a2032cc22a1144c91aee76290964ebf5d5203674b0e256cea53100b6: read unixpacket @->/run/user/1001/libpod/tmp/socket/15a23c97a2032cc22a1144c91aee76290964ebf5d5203674b0e256cea53100b6/attach: read: connection reset by peer
[ rc=126 (** EXPECTED 0 **) ]

This failed just minutes ago. The fix for #7195 was committed last week. This really proves to me that the root cause is unrelated to 7195.

[edsantiago]

Another gating-test flake. podman-2.1.0-0.261.dev.gitfd7cdb2.fc34, rootless. (log).

not ok 83 podman pod create - hashtag AllTheOptions
$  /usr/bin/podman run --pod mypod --name dsfsdfsdf quay.io/libpod/testimage:20200902 nc -l -p 5257
Error: error attaching to container 60ca5032d40b33e29138da01426b487ad5c72fa039f2a8b55752b2e6824145b5: read unixpacket @->/run/user/1001/libpod/tmp/socket/60ca5032d40b33e29138da01426b487ad5c72fa039f2a8b55752b2e6824145b5/attach: read: connection reset by peer
[ rc=126 (** EXPECTED 1 **) ]

Removing the remote label since gating tests are local only.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[edsantiago]

I honestly don't know if this is still present or not. My flake-summarize script isn't good enough to delve into the logs and analyze error messages, it only reports the failing test - and this failure happens on lots of different tests.

[edsantiago]

I don't see any recent instances of this. (I haven't checked all error logs, though, only the most likely-looking ones). (And, unfortunately, Fedora gating tests haven't been running for some weeks now). Closing; will reopen if I see it again.