-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docker: kill signal API should include timeout context #9502
Conversation
This pull request is being automatically deployed with Vercel (learn more). nomad-storybook-and-ui – ./ui🔍 Inspect: https://vercel.com/hashicorp/nomad-storybook-and-ui/npcav2w35 [Deployment for a1bfc03 canceled] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Don't forget a changelog line under bug fixes like:
driver/docker: Fixed a bug where the Docker daemon could block longer than the `kill_timeout` [[GH-9502](https://github.com/hashicorp/nomad/issues/9502)
When the Docker driver kills as task, we send a request via the Docker API for dockerd to fire the signal. We send that signal and then block for the `kill_timeout` waiting for the container to exit. But if the Docker API blocks, we will block indefinitely because we haven't configured the API call with the same timeout. This changeset is a minimal intervention to add the timeout to the Docker API call _only_ when we have the `kill_timeout` set. Future work should examine whether we should be threading contexts through other `go-dockerclient` API calls.
3f9cf73
to
a1bfc03
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Good catch!
I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions. |
Possible fix for #9375 and #9376, based on the discovery of open Docker bugs
that might trigger this behavior (as described in #9375 (comment)). I've smoke-tested
this for safety but haven't had a chance yet to see if it really improves the situation from
those issues.
When the Docker driver kills as task, we send a request via the Docker API for
dockerd to fire the signal. We send that signal and then block for the
kill_timeout
waiting for the container to exit. But if the Docker APIblocks, we will block indefinitely because we haven't configured the API call
with the same timeout.
This changeset is a minimal intervention to add the timeout to the Docker API
call only when we have the
kill_timeout
set. Future work should examinewhether we should be threading contexts through other
go-dockerclient
APIcalls.