Exec support using libpod api

[towe75]

Alternative implementation to #85
It uses libpod api directly, following the minimal client strategy of our 0.2.0 release.

It's pretty complete and support both interactive use and scripted health checks. The only known issue is related to terminal resize events due to containers/podman#7102 which should be fixed in the upcoming podman 3.0 release.

[towe75]

@drewbailey PTAL. I think it's low risk to merge because it adds a totally new feature without changing any of the existing functions.

[drewbailey]

I tried this out locally and it works great! Excellent work 🎉 Just a few comments, Some tests would be good too

[towe75]

@drewbailey : i rebased this branch to include the latest changes from main. Please check my earlier reply on your review.

Are we good to go? I think it would be worth a release.

[drewbailey]

I'm not sure if an operator would find this something that they need to act upon or be notified about, Trace seems better

Suggested change

	c.logger.Warn("Resize handler is done")
	c.logger.Trace("Resize handler is done")

[drewbailey]

@towe75 this looks really great! A few tests would be great to ensure everything works as expected as new podman versions roll out. There should be a few good examples in the Nomad repo

[towe75]

@drewbailey : thank you for feedback. I'll try to add some tests.

[drewbailey]

@towe75 We can also merge this and do a follow up PR with some tests before releasing if you prefer, I'll approve the PR and leave it up to you. If we can fix those two log levels that would be great

[towe75]

@drewbailey i started to derive a test from the other task driver and only two sub-tests from StreamingConformingTests fail:

--- FAIL: TestPodmanDriver_execTaskStreaming (76.20s)
--- PASS: TestPodmanDriver_execTaskStreaming/basic:_notty:_basic (0.13s)
--- PASS: TestPodmanDriver_execTaskStreaming/basic:_notty:_streaming (3.12s)
--- PASS: TestPodmanDriver_execTaskStreaming/basic:_notty:_stty_check (0.11s)
--- PASS: TestPodmanDriver_execTaskStreaming/basic:_notty:_stdin_passing (0.12s)
--- FAIL: TestPodmanDriver_execTaskStreaming/basic:_notty:_children_processes (1.09s)
--- PASS: TestPodmanDriver_execTaskStreaming/basic:_tty:_basic (0.13s)
--- PASS: TestPodmanDriver_execTaskStreaming/basic:_tty:_streaming (3.07s)
--- PASS: TestPodmanDriver_execTaskStreaming/basic:_tty:_stty_check (1.10s)
--- PASS: TestPodmanDriver_execTaskStreaming/basic:_tty:_stdin_passing (0.11s)
--- PASS: TestPodmanDriver_execTaskStreaming/basic:_tty:_children_processes (1.10s)
--- FAIL: TestPodmanDriver_execTaskStreaming/isolation (0.28s)

notty:children_processes has to be tackled, could be potential bug. But "isolation" test has a hardcoded docker assumption in exec_testing.go . Should this become a bug in the nomad repo?

I did not commit the test yet to not break the build.

[drewbailey]

@towe75 that is great news! sounds like TestExecFSIsolation should pass in the expected freezer cgroup instead of assuming nomad or docker

[towe75]

I looked into the other failing test and it seems to be a podman problem OR wrong assumption.
Here is what i tried:

podman run -d -t busybox --name tester

podman exec tester /bin/sh -c "(( sleep 3; echo from background ) & ); echo from main; exec sleep 1"
from main

A plain run without a container acts differently:

/bin/sh -c "(( sleep 3; echo from background ) & ); echo from main; exec sleep 1"
from main
from background

Should i raise a podman issue? Seems they handle it differently compared to docker.

[drewbailey]

@towe75 fyi we are removing this test hashicorp/nomad#10291 from nomad

[towe75]

@drewbailey thank you for keeping me posted. I will file a follow-up issue to add the tests later because there is still the TestExecFSIsolation blocker.
After that we can merge this PR, IMHO.