-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
kill: resync the container if runtime fails #16320
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -352,6 +352,8 @@ func generateResourceFile(res *spec.LinuxResources) (string, []string, error) { | |
return f.Name(), flags, nil | ||
} | ||
|
||
var errSendingSignal = errors.New("sending signal to container") | ||
|
||
// KillContainer sends the given signal to the given container. | ||
// If all is set, send to all PIDs in the container. | ||
// All is only supported if the container created cgroups. | ||
|
@@ -370,15 +372,7 @@ func (r *ConmonOCIRuntime) KillContainer(ctr *Container, signal uint, all bool) | |
args = append(args, "kill", ctr.ID(), fmt.Sprintf("%d", signal)) | ||
} | ||
if err := utils.ExecCmdWithStdStreams(os.Stdin, os.Stdout, os.Stderr, env, r.path, args...); err != nil { | ||
// Update container state - there's a chance we failed because | ||
// the container exited in the meantime. | ||
if err2 := r.UpdateContainerStatus(ctr); err2 != nil { | ||
logrus.Infof("Error updating status for container %s: %v", ctr.ID(), err2) | ||
} | ||
if ctr.ensureState(define.ContainerStateStopped, define.ContainerStateExited) { | ||
return fmt.Errorf("%w: %s", define.ErrCtrStateInvalid, ctr.state.State) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is also going to break SigProxy. We need this to remain ErrCtrStateInvalid. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you elaborate a bit more? At least for kill and stop, the container is not locked, so we should not fiddle with the state. Maybe we need to make this conditional? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The container is locked during kill. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I can see only one place where KillContainer is run unlocked. I'll fix that. We should not be touching KillContainer as such. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You're right. It's unlocked during stop but not kill. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. #16323 catches the last case where KillContainer is run unlocked There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
So you will fix #16142? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sure |
||
} | ||
return fmt.Errorf("sending signal to container %s: %w", ctr.ID(), err) | ||
return fmt.Errorf("%w %s: %v", errSendingSignal, ctr.ID(), err) | ||
} | ||
|
||
return nil | ||
|
@@ -407,15 +401,14 @@ func (r *ConmonOCIRuntime) StopContainer(ctr *Container, timeout uint, all bool) | |
|
||
if timeout > 0 { | ||
if err := r.KillContainer(ctr, stopSignal, all); err != nil { | ||
// Is the container gone? | ||
// If so, it probably died between the first check and | ||
// our sending the signal | ||
// The container is stopped, so exit cleanly | ||
err := unix.Kill(ctr.state.PID, 0) | ||
if err == unix.ESRCH { | ||
if !errors.Is(err, errSendingSignal) { | ||
return err | ||
} | ||
// Maybe sending the signal has failed because the | ||
// container is already gone. | ||
if goneErr := unix.Kill(ctr.state.PID, 0); goneErr == unix.ESRCH { | ||
return nil | ||
} | ||
|
||
return err | ||
} | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't do this, this is going to break things further up the stack re: Sigproxy. The update needs to remain here.