Nomad doesn't respect Dockerfile's STOPSIGNAL directive #9989
Assignees
Labels
hcc/cst
Admin - internal
stage/accepted
Confirmed, and intend to work on. No timeline committment though.
theme/driver/docker
type/bug
Comments
tgross
added
theme/driver/docker
type/bug
stage/accepted
|
Thanks @rc407! We'll get this onto the roadmap. |
|
I did a bit of testing of this and the behavior doesn't quite seem to line up with what you're saying here @rc407. If I look at that Docker image, I can see the But the behavior is still odd. If I use that Docker image in this simple jobspec: job "example" {
datacenters = ["dc1"]
task "api" {
driver = "docker"
config {
image = "monacosl/connect-flask:latest"
}
}
}Stopping that job results in a Having And then run this jobspec with it: job "example" {
datacenters = ["dc1"]
task "api" {
driver = "docker"
config {
image = "0x74696d/stopsignal"
command = "httpd"
args = ["-v", "-f", "-p", "8001", "-h", "/local"]
}
}
}I get Whereas this jobspec, with an explicitly configured job "example" {
datacenters = ["dc1"]
task "api" {
driver = "docker"
kill_signal = "SIGSTOP"
config {
image = "monacosl/connect-flask:latest"
}
}
}Stopping that job results in And then this jobspec, with no job "example" {
datacenters = ["dc1"]
task "api" {
driver = "docker"
config {
image = "busybox:1"
command = "httpd"
args = ["-v", "-f", "-p", "8001", "-h", "/local"]
}
}
}Stopping that job results in So there are two problems here:
|
tgross
changed the title
isabeldepapel
added a commit
that referenced
this issue
Apr 23, 2021
This fixes a bug where Nomad overrides a Dockerfile's STOPSIGNAL with the default kill_signal (SIGTERM). This adds a check for kill_signal. If it's not set, it calls StopContainer instead of Signal, which uses STOPSIGNAL if it's specified. If both kill_signal and STOPSIGNAL are set, Nomad tries to stop the container with kill_signal first, before then calling StopContainer. Fixes #9989
isabeldepapel
added a commit
that referenced
this issue
May 5, 2021
This fixes a bug where Nomad overrides a Dockerfile's STOPSIGNAL with the default kill_signal (SIGTERM). This adds a check for kill_signal. If it's not set, it calls StopContainer instead of Signal, which uses STOPSIGNAL if it's specified. If both kill_signal and STOPSIGNAL are set, Nomad tries to stop the container with kill_signal first, before then calling StopContainer. Fixes #9989
|
I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Issue
Nomad stopsends a SIGTERM signal first instead of sending STOPSIGNAL when terminating a taskWhen you issue a
docker stop, docker sends a SIGTERM by default to the main process inside the container, and after a grace period a SIGKILL. The first signal can be overridden by setting the STOPSIGNAL instruction in the dockerfile. If this is set to let's say 9 then docker will send 9 to the container.When you issue a
nomad stopand STOPSIGNAL is not the default , Nomad calls thekilldocker API endpoint first, sends a SIGTERM, after a grace period a SIGKILL and then calls thestopendpoint and sends the signal specified in the STOPSIGNAL instruction.Expected behavior
Nomad calls two different docker API endpoints, one after another, and both send a signal to the container when it should use only one call to one of the endpoints.
For docker based tasks that use the default
kill_signal(SIGTERM), we could just use one call to StopContainer with the kill_timeout set in docker/handle.goTo Reproduce
1 - Either add the
STOPSIGNAL 9instruction to a dockerfile and build a new image or run the following existing image/job and save it as servicea.nomad:2 - On another session run
docker system events3 - Run
nomad stop servicea. The output ofdocker system eventswill show 3 signals being sent to the container:4 - Run the same job again
5 - Run
docker psand thendocker stop <container_id>. This time the output will show only one signal (the STOPSIGNAL one)Environment
Nomad Version
Nomad v1.0.3Docker Version
The text was updated successfully, but these errors were encountered: