dotnet test hangs when running integration tests with UseWebpackDevMiddleware in Azure DevOps

[ChristopherHaws]

Description

Running the dotnet test task in Azure DevOps Pipelines on a test assembly that is running integration tests (via Microsoft.AspNetCore.Mvc.Testing) while also using UseWebpackDevMiddleware causes the test run to hang forever.

To Reproduce

1. Create a new ASP.NET Core SPA website (which uses UseWebpackDevMiddleware)
2. Create a unit test project that uses Microsoft.AspNetCore.Mvc.Testing to execute an integration test
3. Create an Azure DevOps Pipeline that builds and runs tests via dotnet build and dotnet test (I am using the Visual Studio 2019 on Windows 2019 build agent)
4. Run the pipeline and notice that when it gets to dotnet test, the test run hangs after logging the following:

Starting test execution, please wait...
info: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[0]
      User profile is available. Using 'C:\Users\VssAdministrator\AppData\Local\ASP.NET\DataProtection-Keys' as key repository and Windows DPAPI to encrypt keys at rest.
info: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[0]
      User profile is available. Using 'C:\Users\VssAdministrator\AppData\Local\ASP.NET\DataProtection-Keys' as key repository and Windows DPAPI to encrypt keys at rest.
info: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[58]
      Creating key {5653a870-1f93-4b30-b7d2-29fd94a34191} with creation date 2019-05-23 17:34:18Z, activation date 2019-05-23 17:34:18Z, and expiration date 2019-08-21 17:34:18Z.
info: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[58]
      Creating key {02253cc1-903b-43b6-909a-61faccc67c88} with creation date 2019-05-23 17:34:18Z, activation date 2019-05-23 17:34:18Z, and expiration date 2019-08-21 17:34:18Z.
info: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[39]
      Writing data to file 'C:\Users\VssAdministrator\AppData\Local\ASP.NET\DataProtection-Keys\key-5653a870-1f93-4b30-b7d2-29fd94a34191.xml'.
info: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[39]
      Writing data to file 'C:\Users\VssAdministrator\AppData\Local\ASP.NET\DataProtection-Keys\key-02253cc1-903b-43b6-909a-61faccc67c88.xml'.

Expected behavior

The test run should not hang due to the use of UseWebpackDevMiddleware.

Additional context

• I am using Azure DevOps Pipeline
• I am using version 2.2.300 of the dotnet SDK
• All of my packages are up to date (with no prerelease packages being used)
• The test run executes fine via Visual Studio and CLI locally
• I have tried using an agent hosted on a VM and the issue still occurs

[mkArtakMSFT]

Thanks for contacting us, @ChristopherHaws.
The repro here is quite complicated. Please provide something very minimal, reduced to something self-contained, so there are no dependencies. If, however, you feel that this is no really a code issue on our end, but something related to Azure DevOps or some other external system, consider filing an issue through the respective feedback channel.

[ChristopherHaws]

@mkArtakMSFT Thanks for the reply. I took our solution and removed everything from it except the necessary code to repro this issue. Once piece of information that I missed in the inital report is that you need to create two test classes in order to repro the issue. I created a AzDo repository and yaml pipeline that reproduces the issue.

Repo: https://dev.azure.com/chaws/_git/webpack-issue
Build: https://dev.azure.com/chaws/webpack-issue/_build/results?buildId=19&view=results

Let me know if you need any more info.

[mkArtakMSFT]

@javiercn can you please look into this? Thanks!

[javiercn]

There must be something here between test server and the webpack dev middleware that doesn't play well. @Tratcher might know more.

I don't think there is anything specific to Microsoft.AspNetCore.Mvc.Testing here.

[Tratcher]

Any way to capture a dump of the hung process and see the active call stacks?

[vitalybibikov]

I have the same issue. My test solution with xUnit integration tests works on local enviornment: both on linux and windows.

• .NET Core sdk 2.1.700

After series of experiments I can say that on Azure DevOPS

1. running dotnet build goes fine.
2. Running dotnet test --no-build hangs on the first test executed (sometimes it executes several tests, but still hangs)

[axthosarouris]

Although I did not have exactly the same problem the solution suggested here worked for me.

In short if you are using xUnit you can try the following.

Adding to the .csproj file the following element:

 <ItemGroup>
        <None Update="xunit.runner.json">
            <CopyToOutputDirectory>Always</CopyToOutputDirectory>
        </None>
    </ItemGroup>

And the file xunit.runner.json in the test project:

{
  "parallelizeTestCollections": true,
  "maxParallelThreads": -1
}

[mkArtakMSFT]

Hey, if this is still an issue, please provide us with more details including logs and memory dumps, so we can help out.

[ChristopherHaws]

We ended up switching to UseReactDevelopmentServer since UserWebpackDevMiddleware is being deprecated. The issue does not happen with UseReactDevelopmentServer. Thanks!

[SidShetye]

Another issue is using code like

result = SomeAsyncMethodAsync(arg).Result;

in your test code. It basically comes from blocking in a multi-threaded scenario. Switching to the proper multi-threaded async/await pattern like

result = await SomeAsyncMethodAsync(arg);

gets things working again. Probably also why @axthosarouris 's work-around works, since it effectively disables multi-threading, masking this test code design problem.

Discovered this is older tests written back when async/await was new ...

[cyptus]

also watch out for Wait(); on Task