system tests: enable more remote tests; cleanup

[edsantiago]

info, images, run, networking tests: remove some skip_if_remote()s
that were added in the varlink days. All of these tests now seem
to work with APIv2.

help test: check that first output line from 'podman --help'
is the program description (regression check for #7273).

load test: clean up stray images, rewrite test to make it conform
to existing convention. In the process, discover and file #7337

exec test (and networking): file #7360, and add FIXME comment
to skip()s suggesting evaluating those tests once that is fixed.

pod test: now that #6328 is fixed, use 'podman pod inspect --format'
instead of relying on jq

Various other tests: add an explanation of why test is disabled
so we can more easily distinguish "this will never be meaningful
under remote" vs "hey, doesn't work for now, but maybe someday".

Signed-off-by: Ed Santiago [email protected]

[rhatdan]

LGTM
I like more tests.

[rhatdan]

/approve

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: edsantiago, rhatdan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [rhatdan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[QiWang19]

LGTM

[edsantiago]

This is really alarming. The failure is:

# $ podman load -i /tmp/podman_bats.VjElw1/myimage-5bJ5lXXL.tar
...
# $ podman images -a --format {{.ID}} {{.Repository}}:{{.Tag}}
# dd8b0a90b986 quay.io/libpod/alpine_labels:latest
# #/vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
# #|     FAIL: Image ID of loaded image == original
# #| expected: '4fab981df737'
# #|   actual: 'dd8b0a90b986'
# #\^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

That is: the image restored by podman load has the same tag, but different image ID. It is failing consistently across fedora and ubuntu. The returned image ID is always the same: dd8b0.

I was able to reproduce this once, only once, on my laptop just now. I then tried to git bisect, thinking it might've been something pulled into master today ... and nope, now I can't reproduce it on my laptop at all.

Possibly very stupid question: is there any possibility that this may have been a time- or network-dependent failure? That there was something in the ether for an hour or two, causing these tests to fail in CI and on my setup, but then it cleared up? What exactly goes into the image ID outside of values in the tarball itself?

[TomSweeneyRedHat]

@edsantiago it feels like the image somehow got refreshed sometime after the archive was made, but before the check was done. I'm not sure how that could happen though looking at the code path.

[edsantiago]

'podman save --format oci-archive' is the culprit. Filed: #7371

[edsantiago]

This is ready for review. Big change since initial submission is in 120-load.bats, it still does a load but does not compare IID (due to #7371). I'm still not 100% comfortable with this test as it is, because it's going to leave the system in a funky state: the IID of alpine_labels, after running this test, will differ from the actual IID of alpine_labels as pulled from quay.io. I'm totally OK with someone saying "don't do that", I could cobble up something that save/loads a generated image instead. If there is belief that #7371 will be fixed in a reasonable time, then I'm OK with just waiting instead.

[mheon]

LGTM on my side

[rhatdan]

/lgtm