Split out Smoke Tests and Unit Tests into a separate Jobs

[michael-hawker]

Fixes #3560

This PR does the following:

• Split our tests out of the single Job into two separates Jobs, these can hopefully run in parallel? We should also get separate status checks for each job? (Will see when we run this here.)
• Adds a UWP Baseline Blank App, this will be useful for [Feature] Smoke Test Pipeline Should Report Metrics on Application Sizes #3600 to compare what the minimum app size is currently with no extra dependencies.
• Split out UI Tests as a separate step from the Unit Tests
• Add Application Metrics Report to Smoke Tests [Feature] Smoke Test Pipeline Should Report Metrics on Application Sizes #3600 (could be a separate PR I suppose)

PR Type

• Build or CI related changes

What is the current behavior?

Everything is done in one big job.

What is the new behavior?

We have more granularity on our pipeline

PR Checklist

Please check if your PR fulfills the following requirements:

• Tested code with current supported SDKs
• Pull Request has been submitted to the documentation repository instructions. Link:
• Sample in sample app has been added / updated (for bug fixes / features)
  • Icon has been created (if new sample) following the Thumbnail Style Guide and templates
• Tests for the changes have been added (for bug fixes / features) (if applicable)
• Header has been added to all new source files (run build/UpdateHeaders.bat)
• Contains NO breaking changes

Other information

[ghost]

Thanks michael-hawker for opening a Pull Request! The reviewers will test the PR and highlight if there is any conflict or changes required. If the PR is approved we will proceed to merge the pull request 🙌

[michael-hawker]

Woot!

[azchohfi]

Different jobs might (will) be executed on different machines on ADO Hosted environments, so to do this, you would need to make the artifacts from the build system available to the tests.
The smoke tests would be easier, since they use PackageReferences and are very disconnected from the build itself, so they only need the nuget packages in the correct output, but the unit tests have a strong ProjectReference to the main toolkit projects, so building it in a separate job would rebuild every project that the unit test references. Btw, the Unit tests are already built during the build stage that we have right now, but they are not executed.

[michael-hawker]

When I merge the UI Tests, I should have a split condition to include package references for TAEF but keep the Project Reference for MSTest, this means we could split it out as a separate job to run in parallel.

[michael-hawker]

Alright, cleaned up the YAML edits, think this should be good to run the Smoke Tests as a separate Job now. 🤞 Will check in the morning and look at splitting out the UI Integration Tests as another Job (as they're currently bundled together under the Unit Tests) as well doing the file measuring for the Smoke Tests, have some insights on that one. I don't think it should be too hard. 🤞

[michael-hawker]

Still some improvements we could do for #3600, but this gives us at least a starting point, so we can look to merge this in first and get back to improving things more later after 7.0.

For now, this should give us a good human-readable breakdown of both dependencies and the footprint size on an application.

Definitely some interesting finds... FYI @RosarioPulella

[michael-hawker]

Aww, it doesn't colorize in Azure (seeing if I can get https://developercommunity.visualstudio.com/content/problem/440605/write-host-foreground-color-with-powershell-task-i.html re-opened), but worked like a charm still!

This is ready for review for the first version of this.

[Rosuavio]

Other than some superfluous white space, Looks good!

[ghost]

Hello @michael-hawker!

Because this pull request has the auto merge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

[michael-hawker]

@azchohfi this is all set to go for now. Look good to you?