-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[QUIC] Intermittent hangs in CI #55642
Comments
Tagging subscribers to this area: @dotnet/ncl Issue DetailsSporadic hangs of
|
Author: | ManickaP |
---|---|
Assignees: | - |
Labels: |
|
Milestone: | 6.0.0 |
If we decide to punt this, we should be sure this is not a product bug. |
I can reproduce locally. Definitely seems like regression. |
We should not punt this. |
We seems to always hang in synchronous Read or Write. We are generally in ConnectionClose state. |
It seems like we have problem with completing the tasks. WriteAsync(buffer.ToArray()).AsTask().GetAwaiter().GetResult(); The continuation hangs on thread-pool without being completed. also note that other tasks get stuck too at this point. I added using CancellationTokenSource cts = new CancellationTokenSource();
cts.CancelAfter(_readTimeout); and the cancellation would not expire either at this point. |
- In dotnet#53471 the thread count goal was moved out of `ThreadCounts`, it turns out that are a few subtle races that it was avoiding. There are other ways to fix it, but I've added the goal back into `ThreadCounts` for now. - Reverted PR dotnet#55985, which worked around the issue in the CI Fixes dotnet#55642
Tagging subscribers to this area: @mangod9 Issue DetailsSporadic hangs of
|
Build | Kind | Start Time |
---|---|---|
1236570 | PR 55594 | 2021-14-07 |
Author: | ManickaP |
---|---|
Assignees: | wfurt |
Labels: |
|
Milestone: | 6.0.0 |
* Fix thread pool hang - In #53471 the thread count goal was moved out of `ThreadCounts`, it turns out that are a few subtle races that it was avoiding. There are other ways to fix it, but I've added the goal back into `ThreadCounts` for now. - Reverted PR #55985, which worked around the issue in the CI Fixes #55642 * Revert "mitigation for quic tests hangs (#55985)" This reverts commit 0a5e93b.
Sporadic hangs of
System.Net.Quic.Functional.Tests
.Since yesterday we're seeing that sometime the work item fails with "timeout" meaning a test hang and subsequent kill by the helix agent.
The potential culprit that might have caused this is #55505. Firstly, because the time when it started occurring correlates with the time of its merge. Secondly, the changes come from #52929 and there's been multiple test hangs historically.
Unique tests picked from the console logs:
Kusto queries to get the data
Failing work items (9 records):
Success for comparison (465 total, 135 since the first failure):
cc: @CarnaViire @karelz
The text was updated successfully, but these errors were encountered: