Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http: fully close streams on remote close in AsyncClient #8467

Merged
merged 4 commits into from
Oct 4, 2019

Conversation

goaway
Copy link
Contributor

@goaway goaway commented Oct 2, 2019

Description: The rest of Envoy currently drops stream state on remote closure, regardless of the state of local closure. Where necessary, pending local send calls are also silently swallowed. This updates the AsyncStreamImpl in Http::AsyncClient to also conform to this behavior, which makes things consistent and prevents half-closed streams from crashing the server.
Risk Level: moderate - but this behavior is already followed everywhere else
Testing: compiled and tested locally

Signed-off-by: Mike Schore [email protected]

@mattklein123 mattklein123 self-assigned this Oct 2, 2019
@goaway goaway changed the title http: fully close streams on remote close in AsynClient http: fully close streams on remote close in AsyncClient Oct 2, 2019
Copy link
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks this make sense, but please add crashing tests that this PR fixes.

/wait

Signed-off-by: Mike Schore <[email protected]>
@goaway
Copy link
Contributor Author

goaway commented Oct 3, 2019

Added tests that cover:

  1. streams are bi-directionally closed after receiving a complete response
  2. sendData or sendTrailers do not get propagated after closure

(2) is what ultimately triggers a crash in the router.

Copy link
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with a small question. Thank you!

/wait-any

@@ -93,6 +93,9 @@ void AsyncStreamImpl::encodeHeaders(HeaderMapPtr&& headers, bool end_stream) {
ASSERT(!remote_closed_);
stream_callbacks_.onHeaders(std::move(headers), end_stream);
closeRemote(end_stream);
// At present, the router cleans up stream state as soon as the remote is closed, making a
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

qq: technically if local is not closed in all of these cases, shouldn't we effectively reset it? I think that would be the most correct thing to do? What's the reasoning behind that? Other code would need to change? Should we TODO that more correct behavior?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In some ways yes, resetting would be more correct - technically speaking it's the only way for one side to signal bi-directional closure of the stream. (In fact in Envoy Mobile, when we receive a local response that we interpret as an error, we already do issue a stream reset, triggering cleanup, for that very reason.)

But yes, it would require more code changes to do that here (pretty sure both here in Envoy, and externally where we're relying on this.)

And, everything else (AsyncRequestImpl, Grpc::AsyncClient, the HttpConnectionManager, the Router) currently functions around the idea that a complete response represents the conditions for graceful cleanup. So this seems more in line with current behavior.

Ideally, at some point I think going in and fixing things up so that half-open streams are safe and supported would be the best path forward, since I think it would actually reduce edge cases. But that's a bigger re-evaluation of the L7 logic and codec interaction.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As another aside, one possibility I mentioned to @junr03 is that once we're on a real filter chain, we could add an actual error callback pathway, and it could be a filter that maps errors to local responses. This could allow for better signaling depending on whether we're in library or server mode, and could eliminate a lot of these asymmetric cases as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK can you add more comments in the code around this conversation? Instead of duplicating the same comment multiple times you can just have all of the comments minus one refer to the first comment? Thank you.

/wait

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure thing, updated. Let me know if you think this provides enough context.

Signed-off-by: Mike Schore <[email protected]>
Copy link
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice cleanup, thanks.

@goaway
Copy link
Contributor Author

goaway commented Oct 4, 2019

thanks @mattklein123!

@mattklein123 mattklein123 merged commit f046e37 into envoyproxy:master Oct 4, 2019
rebello95 added a commit to envoyproxy/envoy-mobile that referenced this pull request Oct 4, 2019
Pulls in the fix from envoyproxy/envoy#8467.

Signed-off-by: Michael Rebello <[email protected]>
rebello95 added a commit to envoyproxy/envoy-mobile that referenced this pull request Oct 4, 2019
Pulls in the fix from envoyproxy/envoy#8467.

Signed-off-by: Michael Rebello <[email protected]>
danzh2010 pushed a commit to danzh2010/envoy that referenced this pull request Oct 4, 2019
nandu-vinodan pushed a commit to nandu-vinodan/envoy that referenced this pull request Oct 17, 2019
jpsim pushed a commit that referenced this pull request Nov 28, 2022
Pulls in the fix from #8467.

Signed-off-by: Michael Rebello <[email protected]>
Signed-off-by: JP Simard <[email protected]>
jpsim pushed a commit that referenced this pull request Nov 29, 2022
Pulls in the fix from #8467.

Signed-off-by: Michael Rebello <[email protected]>
Signed-off-by: JP Simard <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants