Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

transport/server: fix race between writing status and header #2074

Merged
merged 2 commits into from
May 21, 2018

Conversation

MakMukhi
Copy link
Contributor

@MakMukhi MakMukhi commented May 15, 2018

Since RecvMsg can lead to a WriteStatus call, it is possible that WriteHeader and WriteStatus are run in parallel. Thus we must synchronize on header and trailer metadata accessed by both.

This doesn't have significant performance impact; adds ~500ns to a 1 byte req/resp unary RPC.

Fixes #1972

@@ -185,13 +185,17 @@ type Stream struct {

headerChan chan struct{} // closed to indicate the end of header metadata.
headerDone uint32 // set when headerChan is closed. Used to avoid closing headerChan multiple times.
header metadata.MD // the received header metadata.
trailer metadata.MD // the key-value map of trailer metadata.
// hdrMu protects header and trailer metadata on the server-side.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a blank line above this to set it off from the above fields.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.


headerOk bool // becomes true from the first header is about to send
state streamState
// On the server-side, headerOK is atomically set to 1 when the headers are sent out.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"headerSent"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.


status *status.Status // the status error received from the server
// On client-side it is the status error received from the server.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Unused on server-side"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it is unused. Not sure if you're suggesting that I add it or merely asking?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was asking, and now requesting you to add a comment like that since you confirmed that it's true.

@@ -683,20 +683,38 @@ func (t *http2Server) handleWindowUpdate(f *http2.WindowUpdateFrame) {
})
}

func createHeaderFieldsFromMD(headerFields []hpack.HeaderField, md metadata.MD) []hpack.HeaderField {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: maybe "appendHeaderFieldsFromMD"? This modifies the slice and returns it just like append does.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

@MakMukhi MakMukhi assigned dfawley and unassigned menghanl May 18, 2018
Copy link
Contributor Author

@MakMukhi MakMukhi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL.

Copy link
Member

@dfawley dfawley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM modulo the minor comment nit.

@MakMukhi MakMukhi merged commit 8f06f82 into grpc:master May 21, 2018
a-robinson added a commit to a-robinson/cockroach that referenced this pull request May 23, 2018
Un-reverts cockroachdb#24883 to include grpc/grpc-go#2074

This also removed hdrhistogram-writer because it apparently isn't used
anymore.

Release note: None
craig bot pushed a commit to cockroachdb/cockroach that referenced this pull request May 23, 2018
25676: storage: Fix flaky TestReplicaNotLeaseHolderError r=a-robinson a=a-robinson

Caused by the proactive renewal of expiration-based leases (see #25322
and #25625 for more detail).

Fixes #25731

Release note: None

I'm not sure how many more of these flakes to expect over the next couple weeks. I'm currently running `stressrace` on all of `pkg/storage` on a gceworker to try and ferret them out if there are any more, but there's no guarantee that'll find all of them.

25816: ui: don't show login indicator unless login is enabled r=couchand a=couchand

In #25005 I accidentally let the login indicator leak even if the environment
variable wasn't set.  This corrects that issue.

Release note: None
Fixes: #25843

25853: dep: Bump grpc-go to include perf improvements and data race fix r=a-robinson a=a-robinson

Un-reverts #24883 to include grpc/grpc-go#2074

This also removed hdrhistogram-writer because it apparently isn't used
anymore.

Release note: None

Maybe we want to wait until they put this into a more formal release SHA, though.

Co-authored-by: Alex Robinson <[email protected]>
Co-authored-by: Andrew Couch <[email protected]>
@dfawley dfawley changed the title Synchronize WriteStatus with WriteHeader on server. transport/server: fix race between writing status and header May 24, 2018
@dfawley dfawley added this to the 1.13 Release milestone May 24, 2018
@MakMukhi MakMukhi deleted the no_race branch May 29, 2018 21:01
@gyuho
Copy link
Contributor

gyuho commented Jun 6, 2018

Can we release v1.12.1 with this cherry-picked? Thanks!

@menghanl
Copy link
Contributor

menghanl commented Jun 6, 2018

@gyuho Sorry for the delay. Just did: https://github.com/grpc/grpc-go/releases/tag/v1.12.1

@gyuho
Copy link
Contributor

gyuho commented Jun 6, 2018

@menghanl No problem! And thanks a lot!

@lock lock bot locked as resolved and limited conversation to collaborators Dec 3, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants