RunContinueWithStressTestsNoState timing out in CI

[jkotas]

C:\h\w\B76F0A24\w\A2D508E5\e>"C:\h\w\B76F0A24\p\dotnet.exe" exec --runtimeconfig System.Threading.Tasks.Tests.runtimeconfig.json --depsfile System.Threading.Tasks.Tests.deps.json xunit.console.dll System.Threading.Tasks.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing -notrait category=nonnetcoreapptests -notrait category=nonwindowstests  
  Discovering: System.Threading.Tasks.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Threading.Tasks.Tests (found 540 of 719 test cases)
  Starting:    System.Threading.Tasks.Tests (parallel test collections = on, max threads = 2)
    System.Threading.Tasks.Tests.TaskSchedulerTests.GetTaskSchedulersForDebugger_DebuggerAttached_ReturnsAllSchedulers [SKIP]
      Condition(s) not met: "DebuggerIsAttached"
    System.Threading.Tasks.Tests.TaskSchedulerTests.GetScheduledTasksForDebugger_DebuggerAttached_ReturnsTasksFromCustomSchedulers [SKIP]
      Condition(s) not met: "DebuggerIsAttached"
   System.Threading.Tasks.Tests: [Long Running Test] 'System.Threading.Tasks.Tests.TaskContinueWithTests.RunContinueWithStressTestsNoState', Elapsed: 00:02:01
   System.Threading.Tasks.Tests: [Long Running Test] 'System.Threading.Tasks.Tests.TaskContinueWithTests.RunContinueWithStressTestsNoState', Elapsed: 00:04:02
   System.Threading.Tasks.Tests: [Long Running Test] 'System.Threading.Tasks.Tests.TaskContinueWithTests.RunContinueWithStressTestsNoState', Elapsed: 00:06:02
   System.Threading.Tasks.Tests: [Long Running Test] 'System.Threading.Tasks.Tests.TaskContinueWithTests.RunContinueWithStressTestsNoState', Elapsed: 00:08:02
   System.Threading.Tasks.Tests: [Long Running Test] 'System.Threading.Tasks.Tests.TaskContinueWithTests.RunContinueWithStressTestsNoState', Elapsed: 00:10:02
   System.Threading.Tasks.Tests: [Long Running Test] 'System.Threading.Tasks.Tests.TaskContinueWithTests.RunContinueWithStressTestsNoState', Elapsed: 00:12:02
   System.Threading.Tasks.Tests: [Long Running Test] 'System.Threading.Tasks.Tests.TaskContinueWithTests.RunContinueWithStressTestsNoState', Elapsed: 00:14:02

I have seen this on multiple PRs.

Runfo Tracking Issue: system.threading.tasks.tests timeout

Build	Definition	Kind	Run Name	Console	Core Dump	Test Results	Run Client
143308	runtime	PR 80797	net8.0-browser-Release-wasm-Mono_Release-WasmTestOnBrowser-(Ubuntu.1804.Amd64)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-webassembly-staging	console.log			runclient.py
143308	runtime	PR 80797	net8.0-browser-Release-wasm-Mono_Release-normal-(Ubuntu.1804.Amd64)[email protected]/dotnet-buildtools/prereqs:ubuntu-18.04-helix-webassembly-staging	console.log			runclient.py
142576	runtime	PR 80806	net8.0-linux-Release-arm-CoreCLR_checked-(Alpine.314.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.14-helix-arm32v7	console.log			runclient.py
142319	runtime	PR 80808	net8.0-browser-Release-wasm-Mono_Release-WasmTestOnBrowser-Ubuntu.2004.Amd64.Open	console.log			runclient.py
132384	runtime	PR 80431	net8.0-Linux-Release-arm-CoreCLR_checked-(Alpine.314.Arm32.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.14-helix-arm32v7	console.log	core dump		runclient.py
132384	runtime	PR 80431	net8.0-Linux-Release-arm64-CoreCLR_checked-(Alpine.314.Arm64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.14-helix-arm64v8	console.log	core dump		runclient.py
131722	runtime	PR 80421	net8.0-Browser-Release-wasm-Mono_Release-normal-Ubuntu.1804.Amd64.Open	console.log			runclient.py
131722	runtime	PR 80421	net8.0-Browser-Release-wasm-Mono_Release-normal-Ubuntu.1804.Amd64.Open	console.log			runclient.py
120942	runtime	PR 79416	net8.0-Browser-Release-wasm-Mono_Release-normal-Ubuntu.1804.Amd64.Open	console.log			runclient.py

Build Result Summary

Day Hit Count	Week Hit Count	Month Hit Count
1	3	6

[jkotas]

Failed in #2237 on "Libraries Test Run checked coreclr Windows_NT x86 Release"

[jkotas]

Failed in #30121

[safern]

Also failing here: #5523

@jkotas would it make sense to disable this test from a Checked runtime at the moment?

[safern]

Also failing in latest rolling build: https://helix.dot.net/api/2019-06-17/jobs/fb420b0a-1e5d-4e84-8bd4-0caed6c2076b/workitems/System.Threading.Tasks.Tests/console

[jkotas]

Yes, it makes sense. Are you going to do it?

[safern]

Are you going to do it?

Yep, I'll put up the PR for it.

[safern]

#31789

[stephentoub]

This test does a lot of work (using words like "stress" and "torture" in it). Seems like simply disabling it on stress runtime runs was the right fix.

[vcsjones]

@stephentoub this failed for me on #34086 in "Libraries Test Run release mono Linux arm64 Debug failed". Is that expected? Should it be disabled in other scenarios?

[stephentoub]

Should it be disabled in other scenarios?

Probably. And probably outerloop. Almost anything with "stress" in its name should be relegated as such :)

[CoffeeFlux]

Can we avoid reopening old issues like this, and instead just open a new one and link back to the previous issue?

It makes it very difficult for me (as someone who would like to look into the problem) to tell what's new and what's from over a year ago, and it's quite possible in this case that the cause is different. It should be a separate issue by default.

cc: @dotnet/runtime-infrastructure (or whoever maintains runfo)

[akoeplinger]

I can't find any recent instances of this and I agree with @CoffeeFlux that we should open a new issue rather than reviving a 1+ years old one if we see it again.

[ericstj]

If you're going to close this then you need to fix this test: 9332582#diff-3123496c18984a1a24e59feeace7f7ef7cb655b8867d6657dea18de934f50bfeR45

Or file a new issue and update the code to point to that new issue.

[akoeplinger]

Ah, didn't notice that since we didn't have the disabled-test label on the issue like we usually do when we disable a test in source. I also renamed the issue to make clear it's about this specific test.