Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Link to Failure log is not always present in Send to Helix logs of the failed CI runs #45357

Closed
adamsitnik opened this issue Nov 30, 2020 · 5 comments
Closed

Link to Failure log is not always present in Send to Helix logs of the failed CI runs #45357

adamsitnik opened this issue Nov 30, 2020 · 5 comments
Labels
area-Infrastructure untriaged New issue has not been triaged by the area owner

Comments

@adamsitnik
Copy link
Member

I've sent a very simple PR and the CI went red: https://github.com/dotnet/runtime/pull/45281/checks?check_run_id=1465351303

So I wanted to find out which tests has failed, but I was unable to find the failed test in the logs

Example of log with no link to the failed tests:

Starting: Send to Helix
==============================================================================
Task         : Command line
Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
==============================================================================
Generating script.
Script contents:
./eng/common/msbuild.sh --ci /datadisks/disk1/workspace/_work/1/s/src/libraries/sendtohelix.proj /p:RuntimeFlavor=coreclr /p:TargetArchitecture=x64 /p:TargetRuntimeIdentifier= /p:Configuration=Debug /p:TargetOS=Linux /p:MonoForceInterpreter= /p:TestScope=innerloop /p:TestRunNamePrefixSuffix=CoreCLR_release /p:HelixBuild=20201127.21 /p:Creator=dotnet-bot /p:BuildTargetFramework=net6.0 /bl:/datadisks/disk1/workspace/_work/1/s/artifacts/log/Debug/SendToHelix.binlog
========================== Starting Command Output ===========================
/bin/bash --noprofile --norc /datadisks/disk1/workspace/_work/_temp/b0b3d0f2-1935-442b-be1d-164073b927e8.sh
/datadisks/disk1/workspace/_work/1/s/.dotnet/sdk/5.0.100-rc.2.20479.15/MSBuild.dll /nologo -logger:/datadisks/disk1/workspace/_work/1/s/.packages/microsoft.dotnet.arcade.sdk/6.0.0-beta.20573.2/tools/netcoreapp2.1/Microsoft.DotNet.Arcade.Sdk.dll -maxcpucount /m -verbosity:m /v:minimal /bl:/datadisks/disk1/workspace/_work/1/s/artifacts/log/Debug/SendToHelix.binlog /clp:Summary /nr:false /p:TreatWarningsAsErrors=true /p:ContinuousIntegrationBuild=true /p:RuntimeFlavor=coreclr /p:TargetArchitecture=x64 /p:TargetRuntimeIdentifier= /p:Configuration=Debug /p:TargetOS=Linux /p:MonoForceInterpreter= /p:TestScope=innerloop /p:TestRunNamePrefixSuffix=CoreCLR_release /p:HelixBuild=20201127.21 /p:Creator=dotnet-bot /p:BuildTargetFramework=net6.0 /warnaserror /datadisks/disk1/workspace/_work/1/s/src/libraries/sendtohelix.proj
  Using Queues: (Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59
  BuildSettings: net6.0-Linux-Debug-x64
  BuildTargetFramework: net6.0
  TestArchiveTestsRoot: /datadisks/disk1/workspace/_work/1/s/artifacts/helix/tests/
  TestHostRootPath: /datadisks/disk1/workspace/_work/1/s/artifacts/bin/testhost/net6.0-Linux-Debug-x64/
  TestArchiveRoot: /datadisks/disk1/workspace/_work/1/s/artifacts/helix/
  TestArchiveRuntimeRoot: /datadisks/disk1/workspace/_work/1/s/artifacts/helix/runtime/
  TestArchiveRuntimeFile: /datadisks/disk1/workspace/_work/1/s/artifacts/helix/runtime/test-runtime-net6.0-Linux-Debug-x64.zip
  Compressing runtime directory
  Creating directory /datadisks/disk1/workspace/_work/1/s/artifacts/helix/runtime/
  Zipping directory "/datadisks/disk1/workspace/_work/1/s/artifacts/bin/testhost/net6.0-Linux-Debug-x64/" to "/datadisks/disk1/workspace/_work/1/s/artifacts/helix/runtime/test-runtime-net6.0-Linux-Debug-x64.zip".
  Correlation directory prepared
  Using Queues: (Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59
  Build TargetFramework: net6.0
  Building Helix work items
  Using TestRunNamePrefix: net6.0-Linux-Debug-x64-CoreCLR_release-
  Using HelixCorrelationPayload: /datadisks/disk1/workspace/_work/1/s/artifacts/helix/runtime/test-runtime-net6.0-Linux-Debug-x64.zip
  Using HelixCommand: ./RunTests.sh --runtime-path "$HELIX_CORRELATION_PAYLOAD"
  Using HelixType: test/functional/cli/innerloop/
  Using WorkItemArchiveWildCard: /datadisks/disk1/workspace/_work/1/s/artifacts/helix/tests/**/*.zip
  Using Timeout: 00:15:00
  Done building Helix work items. Work item count: 240
  Starting Azure Pipelines Test Run net6.0-Linux-Debug-x64-CoreCLR_release-(Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59
  Using Queues: (Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59
  Build TargetFramework: net6.0
  Building Helix work items
  Using TestRunNamePrefix: net6.0-Linux-Debug-x64-CoreCLR_release-
  Using HelixCorrelationPayload: /datadisks/disk1/workspace/_work/1/s/artifacts/helix/runtime/test-runtime-net6.0-Linux-Debug-x64.zip
  Using HelixCommand: ./RunTests.sh --runtime-path "$HELIX_CORRELATION_PAYLOAD"
  Using HelixType: test/functional/cli/innerloop/
  Using WorkItemArchiveWildCard: /datadisks/disk1/workspace/_work/1/s/artifacts/helix/tests/**/*.zip
  Using Timeout: 00:15:00
  Done building Helix work items. Work item count: 240
  Uploading payloads for Job on (Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59...
  Finished uploading payloads for Job on (Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59...
  Sending Job to (Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59...
  Sent Helix Job; see work items at https://helix.dot.net/api/jobs/4b3c383d-b7cb-48ce-8347-a56c4d098fe2/workitems?api-version=2019-06-17
  Waiting for completion of job 4b3c383d-b7cb-48ce-8347-a56c4d098fe2 on (Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59
  Job 4b3c383d-b7cb-48ce-8347-a56c4d098fe2 on (Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59 is completed with 241 finished work items.
  Stopping Azure Pipelines Test Run net6.0-Linux-Debug-x64-CoreCLR_release-(Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59
/datadisks/disk1/workspace/_work/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/azure-pipelines/AzurePipelines.MultiQueue.targets(43,5): error : Test run 28802594 has one or more failing tests. [/datadisks/disk1/workspace/_work/1/s/src/libraries/sendtohelixhelp.proj]
##[error].packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/azure-pipelines/AzurePipelines.MultiQueue.targets(43,5): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Test run 28802594 has one or more failing tests.

Build FAILED.

/datadisks/disk1/workspace/_work/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/azure-pipelines/AzurePipelines.MultiQueue.targets(43,5): error : Test run 28802594 has one or more failing tests. [/datadisks/disk1/workspace/_work/1/s/src/libraries/sendtohelixhelp.proj]
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:11:31.37
##[error]Build failed (exit code '1').
##[error]Bash exited with code '1'.
Finishing: Send to Helix

I've asked @krwq for a sanity check and he found out that WASM builds produce the link to failed tests:

  Done building Helix work items for scenario normal. Work item count: 213
  Using Queues: Ubuntu.1804.Amd64.Open
  Build TargetFramework: net6.0
  Building Helix work items for scenario wasmtestonbrowser
  Using TestRunNamePrefix: net6.0-Browser-Release-wasm-Mono_Release-wasmtestonbrowser-
  Using HelixCorrelationPayload: /__w/1/s/artifacts/helix/runtime/test-runtime-net6.0-Browser-Release-wasm.zip
  Using HelixCommand: ./RunTests.sh
  Using HelixType: test/functional/cli/innerloop/
  Using WorkItemArchiveWildCard: /__w/1/s/artifacts/helix/tests/**/*.zip
  Using Timeout: 00:30:00
  Done building Helix work items for scenario wasmtestonbrowser. Work item count: 213
  Tool 'microsoft.dotnet.xharness.cli' (version '1.0.0-prerelease.20576.1') is already installed.
  Tool 'microsoft.dotnet.xharness.cli' failed to install. This failure may have been caused by:
  
  * You are attempting to install a preview release and did not use the --version option to specify the version.
  * A package by this name was found, but it was not a .NET tool.
  * The required NuGet feed cannot be accessed, perhaps because of an Internet connection problem.
  * You mistyped the name of the tool.
  
  For more reasons, including package naming enforcement, visit https://aka.ms/failure-installing-tool
  Uploading payloads for Job on Ubuntu.1804.Amd64.Open...
  Finished uploading payloads for Job on Ubuntu.1804.Amd64.Open...
  Sending Job to Ubuntu.1804.Amd64.Open...
  You can invoke the tool using the following command: xharness
  Tool 'microsoft.dotnet.xharness.cli' (version '1.0.0-prerelease.20576.1') was successfully installed.
  Uploading payloads for Job on Ubuntu.1804.Amd64.Open...
  Finished uploading payloads for Job on Ubuntu.1804.Amd64.Open...
  Sending Job to Ubuntu.1804.Amd64.Open...
  Sent Helix Job; see work items at https://helix.dot.net/api/jobs/8c6ddb22-9b5b-40b9-b699-2f481727aa56/workitems?api-version=2019-06-17
  Sent Helix Job; see work items at https://helix.dot.net/api/jobs/8733ee54-dcfe-404e-8b0d-adef4ad65c7c/workitems?api-version=2019-06-17
  Waiting for completion of job 8c6ddb22-9b5b-40b9-b699-2f481727aa56 on Ubuntu.1804.Amd64.Open
  Waiting for completion of job 8733ee54-dcfe-404e-8b0d-adef4ad65c7c on Ubuntu.1804.Amd64.Open
  Job 8c6ddb22-9b5b-40b9-b699-2f481727aa56 on Ubuntu.1804.Amd64.Open is completed with 214 finished work items.
  Stopping Azure Pipelines Test Run net6.0-Browser-Release-wasm-Mono_Release-wasmtestonbrowser-Ubuntu.1804.Amd64.Open
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Work item System.Runtime.Tests in job 8c6ddb22-9b5b-40b9-b699-2f481727aa56 has failed. [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Failure log: https://helix.dot.net/api/2019-06-17/jobs/8c6ddb22-9b5b-40b9-b699-2f481727aa56/workitems/System.Runtime.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
##[error].packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Work item System.Runtime.Tests in job 8c6ddb22-9b5b-40b9-b699-2f481727aa56 has failed.
Failure log: https://helix.dot.net/api/2019-06-17/jobs/8c6ddb22-9b5b-40b9-b699-2f481727aa56/workitems/System.Runtime.Tests/console
  Job 8733ee54-dcfe-404e-8b0d-adef4ad65c7c on Ubuntu.1804.Amd64.Open is completed with 214 finished work items.
  Stopping Azure Pipelines Test Run net6.0-Browser-Release-wasm-Mono_Release-normal-Ubuntu.1804.Amd64.Open
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Work item System.Runtime.Tests in job 8733ee54-dcfe-404e-8b0d-adef4ad65c7c has failed. [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Failure log: https://helix.dot.net/api/2019-06-17/jobs/8733ee54-dcfe-404e-8b0d-adef4ad65c7c/workitems/System.Runtime.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
##[error].packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Work item System.Runtime.Tests in job 8733ee54-dcfe-404e-8b0d-adef4ad65c7c has failed.
Failure log: https://helix.dot.net/api/2019-06-17/jobs/8733ee54-dcfe-404e-8b0d-adef4ad65c7c/workitems/System.Runtime.Tests/console

Build FAILED.

/__w/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Work item System.Runtime.Tests in job 8c6ddb22-9b5b-40b9-b699-2f481727aa56 has failed. [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Failure log: https://helix.dot.net/api/2019-06-17/jobs/8c6ddb22-9b5b-40b9-b699-2f481727aa56/workitems/System.Runtime.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Work item System.Runtime.Tests in job 8733ee54-dcfe-404e-8b0d-adef4ad65c7c has failed. [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Failure log: https://helix.dot.net/api/2019-06-17/jobs/8733ee54-dcfe-404e-8b0d-adef4ad65c7c/workitems/System.Runtime.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
    0 Warning(s)
    2 Error(s)

Time Elapsed 00:33:42.17
##[error]Build failed (exit code '1').
##[error]Bash exited with code '1'.
Finishing: Send to Helix

cc @ViktorHofer
@Dotnet-GitSync-Bot Dotnet-GitSync-Bot added the untriaged New issue has not been triaged by the area owner label Nov 30, 2020
@Dotnet-GitSync-Bot
Copy link
Collaborator

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@adamsitnik adamsitnik mentioned this issue Nov 30, 2020
Merged
@adamsitnik
Copy link
Member Author

I have no idea if this is somehow related, but when I try to re-run the failing tests locally I can see that the number of failed tests is not zero, but the build is reported as successful (I remember some issues with Helix related to program exit code):

    Discovering: System.Runtime.Tests (method display = ClassAndMethod, method display options = None)
    Discovered:  System.Runtime.Tests (found 1 of 6075 test case)
    Starting:    System.Runtime.Tests (parallel test collections = on, max threads = 12)
      System.Tests.DateTimeOffsetTests.Ctor_Long_TimeSpan_Invalid [FAIL]
        Assert.Throws() Failure
        Expected: typeof(System.ArgumentException)
        Actual:   typeof(System.ArgumentOutOfRangeException): The UTC time represented when the offset is applied must be between year 0 and 10,000. (Parameter 'offset')
        ---- System.ArgumentOutOfRangeException : The UTC time represented when the offset is applied must be between year 0 and 10,000. (Parameter 'offset')
        Stack Trace:
          C:\Projects\runtime\src\libraries\System.Private.CoreLib\src\System\DateTimeOffset.cs(793,0): at System.DateTimeOffset.ValidateDate(DateTime dateTime, TimeSpan offset)
          C:\Projects\runtime\src\libraries\System.Private.CoreLib\src\System\DateTimeOffset.cs(64,0): at System.DateTimeOffset..ctor(Int64 ticks, TimeSpan offset)
          C:\Projects\runtime\src\libraries\System.Runtime\tests\System\DateTimeOffsetTests.cs(137,0): at System.Tests.DateTimeOffsetTests.<>c.<Ctor_Long_TimeSpan_Invalid>b__8_0()
          ----- Inner Stack Trace -----
          C:\Projects\runtime\src\libraries\System.Private.CoreLib\src\System\DateTimeOffset.cs(793,0): at System.DateTimeOffset.ValidateDate(DateTime dateTime, TimeSpan offset)
          C:\Projects\runtime\src\libraries\System.Private.CoreLib\src\System\DateTimeOffset.cs(64,0): at System.DateTimeOffset..ctor(Int64 ticks, TimeSpan offset)
          C:\Projects\runtime\src\libraries\System.Runtime\tests\System\DateTimeOffsetTests.cs(137,0): at System.Tests.DateTimeOffsetTests.<>c.<Ctor_Long_TimeSpan_Invalid>b__8_0()
    Finished:    System.Runtime.Tests
  === TEST EXECUTION SUMMARY ===
     System.Runtime.Tests  Total: 1, Errors: 0, Failed: 1, Skipped: 0, Time: 15.636s
  ----- end Mon 11/30/2020 13:10:19.59 ----- exit code 1 ----------------------------------------------------------

Build succeeded.
    0 Warning(s)
    0 Error(s)

@ghost
Copy link

ghost commented Nov 30, 2020

Tagging subscribers to this area: @ViktorHofer
See info in area-owners.md if you want to be subscribed.

Issue Details

I've sent a very simple PR and the CI went red: https://github.com/dotnet/runtime/pull/45281/checks?check_run_id=1465351303

So I wanted to find out which tests has failed, but I was unable to find the failed test in the logs

Example of log with no link to the failed tests:

Starting: Send to Helix
==============================================================================
Task         : Command line
Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
==============================================================================
Generating script.
Script contents:
./eng/common/msbuild.sh --ci /datadisks/disk1/workspace/_work/1/s/src/libraries/sendtohelix.proj /p:RuntimeFlavor=coreclr /p:TargetArchitecture=x64 /p:TargetRuntimeIdentifier= /p:Configuration=Debug /p:TargetOS=Linux /p:MonoForceInterpreter= /p:TestScope=innerloop /p:TestRunNamePrefixSuffix=CoreCLR_release /p:HelixBuild=20201127.21 /p:Creator=dotnet-bot /p:BuildTargetFramework=net6.0 /bl:/datadisks/disk1/workspace/_work/1/s/artifacts/log/Debug/SendToHelix.binlog
========================== Starting Command Output ===========================
/bin/bash --noprofile --norc /datadisks/disk1/workspace/_work/_temp/b0b3d0f2-1935-442b-be1d-164073b927e8.sh
/datadisks/disk1/workspace/_work/1/s/.dotnet/sdk/5.0.100-rc.2.20479.15/MSBuild.dll /nologo -logger:/datadisks/disk1/workspace/_work/1/s/.packages/microsoft.dotnet.arcade.sdk/6.0.0-beta.20573.2/tools/netcoreapp2.1/Microsoft.DotNet.Arcade.Sdk.dll -maxcpucount /m -verbosity:m /v:minimal /bl:/datadisks/disk1/workspace/_work/1/s/artifacts/log/Debug/SendToHelix.binlog /clp:Summary /nr:false /p:TreatWarningsAsErrors=true /p:ContinuousIntegrationBuild=true /p:RuntimeFlavor=coreclr /p:TargetArchitecture=x64 /p:TargetRuntimeIdentifier= /p:Configuration=Debug /p:TargetOS=Linux /p:MonoForceInterpreter= /p:TestScope=innerloop /p:TestRunNamePrefixSuffix=CoreCLR_release /p:HelixBuild=20201127.21 /p:Creator=dotnet-bot /p:BuildTargetFramework=net6.0 /warnaserror /datadisks/disk1/workspace/_work/1/s/src/libraries/sendtohelix.proj
  Using Queues: (Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59
  BuildSettings: net6.0-Linux-Debug-x64
  BuildTargetFramework: net6.0
  TestArchiveTestsRoot: /datadisks/disk1/workspace/_work/1/s/artifacts/helix/tests/
  TestHostRootPath: /datadisks/disk1/workspace/_work/1/s/artifacts/bin/testhost/net6.0-Linux-Debug-x64/
  TestArchiveRoot: /datadisks/disk1/workspace/_work/1/s/artifacts/helix/
  TestArchiveRuntimeRoot: /datadisks/disk1/workspace/_work/1/s/artifacts/helix/runtime/
  TestArchiveRuntimeFile: /datadisks/disk1/workspace/_work/1/s/artifacts/helix/runtime/test-runtime-net6.0-Linux-Debug-x64.zip
  Compressing runtime directory
  Creating directory /datadisks/disk1/workspace/_work/1/s/artifacts/helix/runtime/
  Zipping directory "/datadisks/disk1/workspace/_work/1/s/artifacts/bin/testhost/net6.0-Linux-Debug-x64/" to "/datadisks/disk1/workspace/_work/1/s/artifacts/helix/runtime/test-runtime-net6.0-Linux-Debug-x64.zip".
  Correlation directory prepared
  Using Queues: (Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59
  Build TargetFramework: net6.0
  Building Helix work items
  Using TestRunNamePrefix: net6.0-Linux-Debug-x64-CoreCLR_release-
  Using HelixCorrelationPayload: /datadisks/disk1/workspace/_work/1/s/artifacts/helix/runtime/test-runtime-net6.0-Linux-Debug-x64.zip
  Using HelixCommand: ./RunTests.sh --runtime-path "$HELIX_CORRELATION_PAYLOAD"
  Using HelixType: test/functional/cli/innerloop/
  Using WorkItemArchiveWildCard: /datadisks/disk1/workspace/_work/1/s/artifacts/helix/tests/**/*.zip
  Using Timeout: 00:15:00
  Done building Helix work items. Work item count: 240
  Starting Azure Pipelines Test Run net6.0-Linux-Debug-x64-CoreCLR_release-(Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59
  Using Queues: (Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59
  Build TargetFramework: net6.0
  Building Helix work items
  Using TestRunNamePrefix: net6.0-Linux-Debug-x64-CoreCLR_release-
  Using HelixCorrelationPayload: /datadisks/disk1/workspace/_work/1/s/artifacts/helix/runtime/test-runtime-net6.0-Linux-Debug-x64.zip
  Using HelixCommand: ./RunTests.sh --runtime-path "$HELIX_CORRELATION_PAYLOAD"
  Using HelixType: test/functional/cli/innerloop/
  Using WorkItemArchiveWildCard: /datadisks/disk1/workspace/_work/1/s/artifacts/helix/tests/**/*.zip
  Using Timeout: 00:15:00
  Done building Helix work items. Work item count: 240
  Uploading payloads for Job on (Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59...
  Finished uploading payloads for Job on (Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59...
  Sending Job to (Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59...
  Sent Helix Job; see work items at https://helix.dot.net/api/jobs/4b3c383d-b7cb-48ce-8347-a56c4d098fe2/workitems?api-version=2019-06-17
  Waiting for completion of job 4b3c383d-b7cb-48ce-8347-a56c4d098fe2 on (Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59
  Job 4b3c383d-b7cb-48ce-8347-a56c4d098fe2 on (Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59 is completed with 241 finished work items.
  Stopping Azure Pipelines Test Run net6.0-Linux-Debug-x64-CoreCLR_release-(Alpine.312.Amd64.Open)[email protected]/dotnet-buildtools/prereqs:alpine-3.12-helix-20200602002622-e06dc59
/datadisks/disk1/workspace/_work/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/azure-pipelines/AzurePipelines.MultiQueue.targets(43,5): error : Test run 28802594 has one or more failing tests. [/datadisks/disk1/workspace/_work/1/s/src/libraries/sendtohelixhelp.proj]
##[error].packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/azure-pipelines/AzurePipelines.MultiQueue.targets(43,5): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Test run 28802594 has one or more failing tests.

Build FAILED.

/datadisks/disk1/workspace/_work/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/azure-pipelines/AzurePipelines.MultiQueue.targets(43,5): error : Test run 28802594 has one or more failing tests. [/datadisks/disk1/workspace/_work/1/s/src/libraries/sendtohelixhelp.proj]
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:11:31.37
##[error]Build failed (exit code '1').
##[error]Bash exited with code '1'.
Finishing: Send to Helix

I've asked @krwq for a sanity check and he found out that WASM builds produce the link to failed tests:

  Done building Helix work items for scenario normal. Work item count: 213
  Using Queues: Ubuntu.1804.Amd64.Open
  Build TargetFramework: net6.0
  Building Helix work items for scenario wasmtestonbrowser
  Using TestRunNamePrefix: net6.0-Browser-Release-wasm-Mono_Release-wasmtestonbrowser-
  Using HelixCorrelationPayload: /__w/1/s/artifacts/helix/runtime/test-runtime-net6.0-Browser-Release-wasm.zip
  Using HelixCommand: ./RunTests.sh
  Using HelixType: test/functional/cli/innerloop/
  Using WorkItemArchiveWildCard: /__w/1/s/artifacts/helix/tests/**/*.zip
  Using Timeout: 00:30:00
  Done building Helix work items for scenario wasmtestonbrowser. Work item count: 213
  Tool 'microsoft.dotnet.xharness.cli' (version '1.0.0-prerelease.20576.1') is already installed.
  Tool 'microsoft.dotnet.xharness.cli' failed to install. This failure may have been caused by:
  
  * You are attempting to install a preview release and did not use the --version option to specify the version.
  * A package by this name was found, but it was not a .NET tool.
  * The required NuGet feed cannot be accessed, perhaps because of an Internet connection problem.
  * You mistyped the name of the tool.
  
  For more reasons, including package naming enforcement, visit https://aka.ms/failure-installing-tool
  Uploading payloads for Job on Ubuntu.1804.Amd64.Open...
  Finished uploading payloads for Job on Ubuntu.1804.Amd64.Open...
  Sending Job to Ubuntu.1804.Amd64.Open...
  You can invoke the tool using the following command: xharness
  Tool 'microsoft.dotnet.xharness.cli' (version '1.0.0-prerelease.20576.1') was successfully installed.
  Uploading payloads for Job on Ubuntu.1804.Amd64.Open...
  Finished uploading payloads for Job on Ubuntu.1804.Amd64.Open...
  Sending Job to Ubuntu.1804.Amd64.Open...
  Sent Helix Job; see work items at https://helix.dot.net/api/jobs/8c6ddb22-9b5b-40b9-b699-2f481727aa56/workitems?api-version=2019-06-17
  Sent Helix Job; see work items at https://helix.dot.net/api/jobs/8733ee54-dcfe-404e-8b0d-adef4ad65c7c/workitems?api-version=2019-06-17
  Waiting for completion of job 8c6ddb22-9b5b-40b9-b699-2f481727aa56 on Ubuntu.1804.Amd64.Open
  Waiting for completion of job 8733ee54-dcfe-404e-8b0d-adef4ad65c7c on Ubuntu.1804.Amd64.Open
  Job 8c6ddb22-9b5b-40b9-b699-2f481727aa56 on Ubuntu.1804.Amd64.Open is completed with 214 finished work items.
  Stopping Azure Pipelines Test Run net6.0-Browser-Release-wasm-Mono_Release-wasmtestonbrowser-Ubuntu.1804.Amd64.Open
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Work item System.Runtime.Tests in job 8c6ddb22-9b5b-40b9-b699-2f481727aa56 has failed. [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Failure log: https://helix.dot.net/api/2019-06-17/jobs/8c6ddb22-9b5b-40b9-b699-2f481727aa56/workitems/System.Runtime.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
##[error].packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Work item System.Runtime.Tests in job 8c6ddb22-9b5b-40b9-b699-2f481727aa56 has failed.
Failure log: https://helix.dot.net/api/2019-06-17/jobs/8c6ddb22-9b5b-40b9-b699-2f481727aa56/workitems/System.Runtime.Tests/console
  Job 8733ee54-dcfe-404e-8b0d-adef4ad65c7c on Ubuntu.1804.Amd64.Open is completed with 214 finished work items.
  Stopping Azure Pipelines Test Run net6.0-Browser-Release-wasm-Mono_Release-normal-Ubuntu.1804.Amd64.Open
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Work item System.Runtime.Tests in job 8733ee54-dcfe-404e-8b0d-adef4ad65c7c has failed. [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Failure log: https://helix.dot.net/api/2019-06-17/jobs/8733ee54-dcfe-404e-8b0d-adef4ad65c7c/workitems/System.Runtime.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
##[error].packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Work item System.Runtime.Tests in job 8733ee54-dcfe-404e-8b0d-adef4ad65c7c has failed.
Failure log: https://helix.dot.net/api/2019-06-17/jobs/8733ee54-dcfe-404e-8b0d-adef4ad65c7c/workitems/System.Runtime.Tests/console

Build FAILED.

/__w/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Work item System.Runtime.Tests in job 8c6ddb22-9b5b-40b9-b699-2f481727aa56 has failed. [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Failure log: https://helix.dot.net/api/2019-06-17/jobs/8c6ddb22-9b5b-40b9-b699-2f481727aa56/workitems/System.Runtime.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Work item System.Runtime.Tests in job 8733ee54-dcfe-404e-8b0d-adef4ad65c7c has failed. [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/6.0.0-beta.20573.2/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Failure log: https://helix.dot.net/api/2019-06-17/jobs/8733ee54-dcfe-404e-8b0d-adef4ad65c7c/workitems/System.Runtime.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
    0 Warning(s)
    2 Error(s)

Time Elapsed 00:33:42.17
##[error]Build failed (exit code '1').
##[error]Bash exited with code '1'.
Finishing: Send to Helix

cc @ViktorHofer

Author: adamsitnik
Assignees: -
Labels:

area-Infrastructure, untriaged
Milestone: -

@akoeplinger
Copy link
Member

This was a side-effect of #45164. We agree it's confusing and we'll fix it, in the meantime you can view the test failures by going to the "Tests" tab in the Azure DevOps build summary page.

(note: the reason why you still see the console log in WASM builds is because they use a different way of running the tests).

@akoeplinger
Copy link
Member

The link to the console log is printed again now.

@ghost ghost locked as resolved and limited conversation to collaborators Jan 17, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
area-Infrastructure untriaged New issue has not been triaged by the area owner
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@akoeplinger @adamsitnik @jeffschwMSFT @Dotnet-GitSync-Bot