Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky test QuicHttpIntegrationTest::DeferredLoggingWithRetransmission #27850

Merged
merged 4 commits into from
Jun 8, 2023

Conversation

pksohn
Copy link
Contributor

@pksohn pksohn commented Jun 7, 2023

Fixes #27841:

  • Increase the sleep from 500->1000 ms.
  • Properly clean up the SocketInterfaceSwap.

Commit Message: Fix flaky test DeferredLoggingWithRetransmission
Risk Level: N/A, just testing

@repokitteh-read-only
Copy link

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #27850 was opened by pksohn.

see: more, trace.

@pksohn pksohn changed the title Fix one source of flakiness in QuicHttpIntegrationTest::DeferredLoggingWithRetransmission Fix flaky test QuicHttpIntegrationTest::DeferredLoggingWithRetransmission Jun 7, 2023
// Allow the response to be sent downstream again.
socket_swap.write_matcher_->setWriteOverride(nullptr);
{
SocketInterfaceSwap socket_swap(downstreamProtocol() == Http::CodecType::HTTP3
Copy link
Contributor Author

@pksohn pksohn Jun 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This s/upstream/downstream doesn't functionally change anything, but is semantically more correct.

@pksohn pksohn marked this pull request as ready for review June 7, 2023 18:25
@pksohn
Copy link
Contributor Author

pksohn commented Jun 7, 2023

/assign @alyssawilk

socket_swap.write_matcher_->setDestinationPort(lookupPort("http"));
socket_swap.write_matcher_->setWriteOverride(ebadf);
upstream_request_->encodeHeaders(Http::TestResponseHeaderMapImpl{{":status", "200"}}, true);
absl::SleepFor(absl::Seconds(TSAN_TIMEOUT_FACTOR));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry I overlooked this in the prior version. Why are we doing sleepfor? Is there a variable we can wait on instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sleep is to wait for a retransmission to occur (such that QuicStatsGatherer::OnPacketRetransmitted is executed). I couldn't think of a good integration-test-level event to wait on. Is there a good way to either 1) detect a retransmitted packet from the server to downstream, or 2) create a custom event to wait on from the QuicStatsGatherer?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we use timeSystem().advanceTimeWait ? I don't remember if this test uses simtime. If so let's switch over. if not, maybe we can try to dig up some stats

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, timeSystem().advanceTimeWait seems to work.

@alyssawilk alyssawilk merged commit 9b3d089 into envoyproxy:main Jun 8, 2023
@pksohn pksohn deleted the flaky-retransmission-msan branch June 8, 2023 13:27
asheryerm pushed a commit to asheryerm/envoy that referenced this pull request Jul 5, 2023
…sion (envoyproxy#27850)

Fixes envoyproxy#27841:

Increase the sleep from 500->1000 ms.
Properly clean up the SocketInterfaceSwap.
Commit Message: Fix flaky test DeferredLoggingWithRetransmission
Risk Level: N/A, just testing

Signed-off-by: Paul Sohn <[email protected]>
Signed-off-by: asheryer <[email protected]>
reskin89 pushed a commit to reskin89/envoy that referenced this pull request Jul 11, 2023
…sion (envoyproxy#27850)

Fixes envoyproxy#27841:

Increase the sleep from 500->1000 ms.
Properly clean up the SocketInterfaceSwap.
Commit Message: Fix flaky test DeferredLoggingWithRetransmission
Risk Level: N/A, just testing

Signed-off-by: Paul Sohn <[email protected]>
Signed-off-by: Ryan Eskin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

flake in DeferredLoggingWithRetransmission
2 participants