Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix a deadlock if a panic happens during request handling #6920

Merged
merged 2 commits into from
Jun 19, 2019

Conversation

jefferai
Copy link
Member

During request handling, if a panic is created, deferred functions are
run but otherwise execution stops. #5889 changed some locks to
non-defers but had the side effect of causing the read lock to not be
released if the request panicked. This fixes that and addresses a few
other potential places where things could go wrong:

  1. In sealInitCommon we always now defer a function that unlocks the
    read lock if it hasn't been unlocked already
  2. In StepDown we defer the RUnlock but we also had two error cases that
    were calling it manually. These are unlikely to be hit but if they were
    I believe would cause a panic.

During request handling, if a panic is created, deferred functions are
run but otherwise execution stops. #5889 changed some locks to
non-defers but had the side effect of causing the read lock to not be
released if the request panicked. This fixes that and addresses a few
other potential places where things could go wrong:

1) In sealInitCommon we always now defer a function that unlocks the
read lock if it hasn't been unlocked already
2) In StepDown we defer the RUnlock but we also had two error cases that
were calling it manually. These are unlikely to be hit but if they were
I believe would cause a panic.
@jefferai jefferai added this to the 1.2 milestone Jun 19, 2019
@jefferai jefferai requested review from briankassouf and kalafut June 19, 2019 01:41
briankassouf
briankassouf previously approved these changes Jun 19, 2019
@@ -261,14 +262,12 @@ func (c *Core) StepDown(httpCtx context.Context, req *logical.Request) (retErr e
if entity != nil && entity.Disabled {
c.logger.Warn("permission denied as the entity on the token is disabled")
retErr = multierror.Append(retErr, logical.ErrPermissionDenied)
c.stateLock.RUnlock()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch!

kalafut
kalafut previously approved these changes Jun 19, 2019
vishalnayak
vishalnayak previously approved these changes Jun 19, 2019
@jefferai jefferai dismissed stale reviews from vishalnayak, kalafut, and briankassouf via 9d95878 June 19, 2019 13:24
@jefferai jefferai merged commit 4184e72 into master Jun 19, 2019
@jefferai jefferai deleted the fix-request-panic-deadlock branch June 19, 2019 13:41
@jefferai jefferai modified the milestones: 1.2, 1.1.4 Jun 20, 2019
jefferai added a commit that referenced this pull request Jun 20, 2019
* Fix a deadlock if a panic happens during request handling

During request handling, if a panic is created, deferred functions are
run but otherwise execution stops. #5889 changed some locks to
non-defers but had the side effect of causing the read lock to not be
released if the request panicked. This fixes that and addresses a few
other potential places where things could go wrong:

1) In sealInitCommon we always now defer a function that unlocks the
read lock if it hasn't been unlocked already
2) In StepDown we defer the RUnlock but we also had two error cases that
were calling it manually. These are unlikely to be hit but if they were
I believe would cause a panic.

* Add panic recovery test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants