Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a shutdown signal handler #7999

Merged
merged 3 commits into from
Oct 20, 2020

Conversation

mheon
Copy link
Member

@mheon mheon commented Oct 12, 2020

We need a unified package for handling signals that shut down Libpod and Podman. We need to be able to do different things on receiving such a signal (system service wants to shut down the service gracefully, while most other commands just want to exit) and we need to be able to inhibit this shutdown signal while we are waiting for some critical operations (e.g. creating a container) to finish. We also want to turn off this handling when --sig-proxy is active.

Fixes #7941

We need a unified package for handling signals that shut down
Libpod and Podman. We need to be able to do different things on
receiving such a signal (`system service` wants to shut down the
service gracefully, while most other commands just want to exit)
and we need to be able to inhibit this shutdown signal while we
are waiting for some critical operations (e.g. creating a
container) to finish. This takes the first step by defining the
package that will handle this.

Signed-off-by: Matthew Heon <[email protected]>
@openshift-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mheon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 12, 2020
@mheon
Copy link
Member Author

mheon commented Oct 12, 2020

Potential improvements:

  • Pass the signal into the shutdown handler so we can switch behaviour based on SIGINT vs SIGTERM
  • Order shutdown handlers LIFO so we could use both the server and libpod handlers as the same time, server handler coming first

@mheon mheon force-pushed the signal_handler branch 2 times, most recently from d8a0ccf to 3e8a118 Compare October 12, 2020 20:42
Expand the use of the Shutdown package such that we now use it
to handle signals any time we run Libpod. From there, add code to
container creation to use the Inhibit function to prevent a
shutdown from occuring during the critical parts of container
creation.

We also need to turn off signal handling when --sig-proxy is
invoked - we don't want to catch the signals ourselves then, but
instead to forward them into the container via the existing
sig-proxy handler.

Fixes containers#7941

Signed-off-by: Matthew Heon <[email protected]>
@mheon
Copy link
Member Author

mheon commented Oct 13, 2020

@jwhonce PTAL, I'd like to make sure I got signal handling for the server right.

// Start begins handling SIGTERM and SIGINT and will run the given on-signal
// handlers when one is called. This can be cancelled by calling Stop().
func Start() error {
if sigChan != nil && !stopped {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would appear to be a race on reading/setting stopped

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me just remove that, it's not important that we be able to restart the handler right now.

Comment on lines +91 to +93
if _, ok := handlers[name]; ok {
return errors.Errorf("handler with name %s already exists", name)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if _, ok := handlers[name]; ok {
return errors.Errorf("handler with name %s already exists", name)
}

Why not just reset handler to new func?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to allow multiple handlers at the same time - theoretically the Libpod handler could run after the server handler and do any extra shutdown needed there.

}
// Unregister the libpod handler, which just calls exit(1).
// Ignore errors if it doesn't exist.
_ = shutdown.Unregister("libpod")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems smelly to remove someone elses handler by name...

libpod/shutdown/handler.go Outdated Show resolved Hide resolved
@mheon
Copy link
Member Author

mheon commented Oct 13, 2020

@jwhonce PTAL again, comments should be fixed

This allows us to run both the Libpod and Server handlers at the
same time without unregistering one.

Also, pass the signal that killed us into the handlers, in case
they want to use it to determine what to do (e.g. what exit code
to set).

Signed-off-by: Matthew Heon <[email protected]>
@jwhonce
Copy link
Member

jwhonce commented Oct 20, 2020

Restarted test flakes

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Oct 20, 2020
@openshift-merge-robot openshift-merge-robot merged commit 6c0b600 into containers:master Oct 20, 2020
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 24, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 24, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Container storage creation fragile
4 participants