-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix runc events error in cgroup v2 #2352
Conversation
Thanks, please update https://github.com/opencontainers/runc/blob/master/tests/integration/events.bats as well |
b0eba0c
to
760cf31
Compare
enable cgroup v2 tests in events.bats now. |
I have add a test now, but not easy, we should disable memory swap, or else it’s very hard to cause oom. |
retry 10 1 eval "grep -q 'oom' events.log" | ||
__runc delete -f test_busybox | ||
) & | ||
wait # wait for the above sub shells to finish |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe add a small timeout in case runc won't ever exit?
timeout 10 wait
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Once #2370 merged, I'll update it together with some other changes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kolyshkin timeout
and wait
can't be used together.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, you're right.
I am still concerned that this case might hang forever. Maybe we can run both __runc
invocations with a timeout then. Something big, say 1 minute. Obviously, if OOM has not happened within a minute, the test has failed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Travis CI will return an error if there is no output after 150 seconds.
I met this error before I disable swap.
If you worry about this, I can write a simple C program to ensure causing oom kill.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kolyshkin use sh -c 'test=$(dd if=/dev/urandom ibs=5120k)'
to ensure oom kill.
Because the memory limit is 4M.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's getting much better and is almost ready. Just a few nits.
0789d76
to
284d9b2
Compare
Signed-off-by: lifubang <[email protected]>
How to test (from opencontainers/runc#2352 (comment)): (host)$ sudo swapoff -a (host)$ sudo ctr run -t --rm --memory-limit $((1024*1024*32)) docker.io/library/alpine:latest foo (container)$ sh -c 'VAR=$(seq 1 100000000)' An event `/tasks/oom {"container_id":"foo"}` will be displayed in `ctr events`. Signed-off-by: Akihiro Suda <[email protected]>
How to test (from opencontainers/runc#2352 (comment)): (host)$ sudo swapoff -a (host)$ sudo ctr run -t --rm --memory-limit $((1024*1024*32)) docker.io/library/alpine:latest foo (container)$ sh -c 'VAR=$(seq 1 100000000)' An event `/tasks/oom {"container_id":"foo"}` will be displayed in `ctr events`. Signed-off-by: Akihiro Suda <[email protected]>
How to test (from opencontainers/runc#2352 (comment)): (host)$ sudo swapoff -a (host)$ sudo ctr run -t --rm --memory-limit $((1024*1024*32)) docker.io/library/alpine:latest foo (container)$ sh -c 'VAR=$(seq 1 100000000)' An event `/tasks/oom {"container_id":"foo"}` will be displayed in `ctr events`. Signed-off-by: Akihiro Suda <[email protected]>
fix #2351
Because there is no
oom notifier
incgroup v2
now.I think we should use
Inotify
to watchOOM
inmemory.events
.Signed-off-by: lifubang [email protected]