Add `runtime-wasm` pipeline #64830

radical · 2022-02-04T21:22:58Z

This adds support for a runtime-wasm pipeline, which uses
runtime-extra-platforms.yml.

The wasm jobs have been moved to runtime-extra-platforms-wasm.yml, and
the rest to runtime-extra-platforms-other.yml. And the two files are
included conditionally.

Effectively, this still runs all these jobs when
runtime-extra-platforms is run (same as before). But it allows running
just the wasm jobs, triggered manually with:

`/azp run runtime-wasm`

When runtime-wasm is manually run:

LibraryTests will run all the tests on linux+windows
LibraryTests_EAT will run all the tests on linux
LibraryTests_AOT will run all the tests on linux+windows
Wasm.Build.Tests, and Wasm debugger tests will run on linux+windows
Runtime tests will run on linux

When runtime-extra-platforms is manually run:

LibraryTests will run all the tests, if paths changed
LibraryTests_EAT will run all the tests on linux, if paths changed
LibraryTests_AOT will run all the tests on linux+windows, if paths changed
Wasm.Build.Tests, and Wasm debugger tests will never run
Runtime tests will never run

On rolling builds, runtime-extra-platforms runs by default:

LibraryTests will run all the tests
LibraryTests_EAT will run all the tests on linux
LibraryTests_AOT will run all the tests on linux+windows
Wasm.Build.Tests, and Wasm debugger tests will never run
Runtime tests will never run

With feedback from @steveisok, and @safern.

ghost · 2022-02-04T21:23:10Z

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Issue Details

This adds support for a wasm-only pipeline, which use
runtime-extra-platforms.yml.

When wasm-only is manually run:

LibraryTests will run all the tests on linux+windows
LibraryTests_EAT will run all the tests on linux
LibraryTests_AOT will run all the tests on linux+windows
Wasm.Build.Tests, and Wasm debugger tests will run on linux+windows
Runtime tests will run on linux

When runtime-extra-platforms is manually run:

LibraryTests will run all the tests, if paths changed
LibraryTests_EAT will run all the tests on linux, if paths changed
LibraryTests_AOT will run all the tests on linux+windows, if paths changed
Wasm.Build.Tests, and Wasm debugger tests will never run
Runtime tests will never run

On rolling builds, runtime-extra-platforms runs by default:

LibraryTests will run all the tests
LibraryTests_EAT will run all the tests on linux
LibraryTests_AOT will run all the tests on linux+windows
Wasm.Build.Tests, and Wasm debugger tests will never run
Runtime tests will never run

Author:	radical
Assignees:	radical
Labels:	`area-Infrastructure-libraries`
Milestone:	-

radical · 2022-02-04T21:25:11Z

This is the new state:

jobs with only-pc are run only if the path checks are true

PR:

.	runtime	runtime-staging	runtime-extra-platforms(manual only)	wasm-only (manual only)
libtests	linux: all, only-pc	windows: all, only-pc	linux+windows: all, only-pc	linux+windows: all
libtests eat	linux: smoke, only-pc	-	linux: all, only-pc	linux: all
libtests aot	linux: smoke, only-pc	windows: smoke, only-pc	linux+windows: all, only-pc	linux+windows: all

Wasm.Build.Tests	linux: only-pc	windows: only-pc	-	linux+windows
Debugger tests	-	linux+windows: only-pc	-	linux+windows
Runtime tests	linux: only-pc	-	-	linux

Rolling build:

jobs w/o only-pc are always run

.	runtime	runtime-staging	runtime-extra-platforms (always run)	wasm-only (manual only)
libtests	linux: all	windows: all, only-pc	linux+windows: all	NA
libtests eat	linux: smoke(=all)	-	linux: all
libtests aot	linux: smoke(=all)	windows: smoke(=all), only-pc	linux+windows: all

Wasm.Build.Tests	linux: all	windows: only-pc	linux+windows
Debugger tests	-	linux+windows	-
Runtime tests	linux: all	-	-

There is an existing inconsistency in rolling builds:

runtime is always running the tests
runtime-staging is running the tests only on on path changes
runtime-extra-platforms

radical · 2022-02-04T21:26:46Z

/azp run wasm-only

azure-pipelines · 2022-02-04T21:26:55Z

Azure Pipelines failed to run 1 pipeline(s).

radical · 2022-02-04T21:28:37Z

/azp run wasm-only

azure-pipelines · 2022-02-04T21:28:45Z

Azure Pipelines failed to run 1 pipeline(s).

radical · 2022-02-04T21:30:38Z

/azp run wasm-only

azure-pipelines · 2022-02-04T21:30:56Z

Azure Pipelines failed to run 1 pipeline(s).

radical · 2022-02-04T21:32:27Z

/azp run wasm-only

azure-pipelines · 2022-02-04T21:32:46Z

Azure Pipelines failed to run 1 pipeline(s).

radical · 2022-02-04T21:36:18Z

/azp run wasm-only

azure-pipelines · 2022-02-04T21:36:38Z

Azure Pipelines successfully started running 1 pipeline(s).

radical · 2022-02-04T21:52:28Z

/azp run wasm-only

azure-pipelines · 2022-02-04T21:52:43Z

Azure Pipelines failed to run 1 pipeline(s).

radical · 2022-02-04T21:53:45Z

/azp run wasm-only

azure-pipelines · 2022-02-04T21:54:07Z

Azure Pipelines successfully started running 1 pipeline(s).

radical · 2022-02-04T22:00:38Z

/azp run wasm-only

azure-pipelines · 2022-02-04T22:00:46Z

Azure Pipelines failed to run 1 pipeline(s).

radical · 2022-02-04T22:03:30Z

/azp run wasm-only

azure-pipelines · 2022-02-04T22:03:42Z

Azure Pipelines successfully started running 1 pipeline(s).

radical · 2022-02-04T22:11:24Z

/azp run wasm-only

azure-pipelines · 2022-02-04T22:11:38Z

Azure Pipelines successfully started running 1 pipeline(s).

safern · 2022-02-04T22:16:40Z

May I suggest that we name it runtime-wasm to match all our other naming conventions?

radical · 2022-02-07T16:50:17Z

/azp run runtime-wasm,runtime-extra-platforms

azure-pipelines · 2022-02-07T16:50:28Z

Azure Pipelines failed to run 2 pipeline(s).

safern · 2022-02-07T17:02:16Z

eng/pipelines/runtime-extra-platforms.yml

@@ -13,7 +13,7 @@ variables:
  - template: /eng/pipelines/common/variables.yml

 - ${{ if eq(variables.isNotSpecificPlatformOnlyBuild, true) }}:


I don't think expand time conditions work for schedules. We would have to go to the AzDo UI for this pipeline and override the CI triggers/schedule.

I don't think it'll work either.

If you want such a behavior and not use AzDo UI, I think you'd have to make a template and then have two pipelines. One with a schedule and one w/o.

I disabled it from the UI for runtime-wasm specifically.

This reverts commit 0aac405.

This reverts commit c929f5c.

radical · 2022-02-07T17:26:01Z

/azp run runtime-wasm,runtime-extra-platforms

azure-pipelines · 2022-02-07T17:26:36Z

Azure Pipelines successfully started running 2 pipeline(s).

radical · 2022-02-07T21:55:02Z

Merging main to pick up some test fixes for Android.
The windows/x64 test failures are failing on rolling builds too.

radical · 2022-02-07T21:57:00Z

/azp run runtime-extra-platforms

azure-pipelines · 2022-02-07T21:57:33Z

Azure Pipelines successfully started running 1 pipeline(s).

radical · 2022-02-08T03:32:53Z

Anything else needed for this PR?

premun · 2022-02-08T14:42:53Z

@radical love the table overview!

MaximLipnin · 2022-02-08T15:22:14Z

Does it need to be reflected in somewhere in docs, e.g. https://github.com/dotnet/runtime/blob/main/docs/pr-builds.md ?

radical · 2022-02-08T15:35:30Z

Does it need to be reflected in somewhere in docs, e.g. https://github.com/dotnet/runtime/blob/main/docs/pr-builds.md ?

I can add it in a follow up PR.

radical requested a review from safern February 4, 2022 21:23

ghost assigned radical Feb 4, 2022

dotnet-issue-labeler bot added the area-Infrastructure-libraries label Feb 4, 2022

radical requested review from steveisok and lewing February 4, 2022 21:23

radical requested a review from akoeplinger February 4, 2022 21:23

radical added 2 commits February 7, 2022 11:37

Merge remote-tracking branch 'origin/main' into wasm-manual

2779d1d

runtime-wasm: don't trigger for scheduled builds

c929f5c

fix yml

0aac405

safern reviewed Feb 7, 2022

View reviewed changes

radical added 2 commits February 7, 2022 12:20

Revert "fix yml"

87475c7

This reverts commit 0aac405.

Revert "runtime-wasm: don't trigger for scheduled builds"

3505480

This reverts commit c929f5c.

This was referenced Feb 7, 2022

System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_SslProtocols_Test #64941

Closed

System.Net.Security.Tests timing out #64943

Closed

Merge remote-tracking branch 'origin/main' into wasm-manual

3a05fab

runfoapp bot mentioned this pull request Feb 7, 2022

System.Net.Quic.Functional.Tests timing out #64944

Closed

akoeplinger approved these changes Feb 8, 2022

View reviewed changes

radical requested a review from safern February 8, 2022 17:14

runfoapp bot mentioned this pull request Feb 8, 2022

System.Text.Tests.EncodingExtensionsTests.Convert_Encoder_ReadOnlySpan_IBufferWriter failing on Windows Server 2022 #65026

Closed

safern approved these changes Feb 8, 2022

View reviewed changes

radical merged commit 05dbb13 into dotnet:main Feb 8, 2022

radical deleted the wasm-manual branch February 8, 2022 21:56

runfoapp bot mentioned this pull request Feb 11, 2022

baseservices\\exceptions\\stackoverflow\\stackoverflowtester\\stackoverflowtester.cmd #65201

Closed

ghost locked as resolved and limited conversation to collaborators Mar 11, 2022

		@@ -13,7 +13,7 @@ variables:
		- template: /eng/pipelines/common/variables.yml

		- ${{ if eq(variables.isNotSpecificPlatformOnlyBuild, true) }}:

Add runtime-wasm pipeline #64830

Add runtime-wasm pipeline #64830

Conversation

radical commented Feb 4, 2022 • edited Loading

ghost commented Feb 4, 2022

radical commented Feb 4, 2022 • edited Loading

PR:

Rolling build:

radical commented Feb 4, 2022

azure-pipelines bot commented Feb 4, 2022

radical commented Feb 4, 2022

azure-pipelines bot commented Feb 4, 2022

radical commented Feb 4, 2022

azure-pipelines bot commented Feb 4, 2022

radical commented Feb 4, 2022

azure-pipelines bot commented Feb 4, 2022

radical commented Feb 4, 2022

azure-pipelines bot commented Feb 4, 2022

radical commented Feb 4, 2022

azure-pipelines bot commented Feb 4, 2022

radical commented Feb 4, 2022

azure-pipelines bot commented Feb 4, 2022

radical commented Feb 4, 2022

azure-pipelines bot commented Feb 4, 2022

radical commented Feb 4, 2022

azure-pipelines bot commented Feb 4, 2022

radical commented Feb 4, 2022

azure-pipelines bot commented Feb 4, 2022

safern commented Feb 4, 2022

radical commented Feb 7, 2022

azure-pipelines bot commented Feb 7, 2022

safern Feb 7, 2022

Choose a reason for hiding this comment

steveisok Feb 7, 2022

Choose a reason for hiding this comment

radical Feb 7, 2022 • edited Loading

Choose a reason for hiding this comment

radical commented Feb 7, 2022

azure-pipelines bot commented Feb 7, 2022

radical commented Feb 7, 2022

radical commented Feb 7, 2022

azure-pipelines bot commented Feb 7, 2022

radical commented Feb 8, 2022

premun commented Feb 8, 2022

MaximLipnin commented Feb 8, 2022

radical commented Feb 8, 2022

Add `runtime-wasm` pipeline #64830

Add `runtime-wasm` pipeline #64830

radical commented Feb 4, 2022 •

edited

Loading

radical commented Feb 4, 2022 •

edited

Loading

radical Feb 7, 2022 •

edited

Loading