[ci] Use dotnet to create workload .pkg

[pjcollins]

Commit 12edc2c introduced a feature that seems to only be available
in newer versions of MSBuild:

<Import Project="Sdk.props" Sdk="Microsoft.DotNet.SharedFramework.Sdk" Version="$(MicrosoftDotNetBuildTasksFeedPackageVersion)" />

This caused an issue during the dotnet_create_pkg job on our
Xamarin.Android pipeline, which uses a different machine pool than
Xamarin.Android-PR:

/Users/builder/azdo/_work/2/s/xamarin-android/build-tools/create-packs/Directory.Build.targets(11,3): error : Version 5.0.201 of the .NET Core SDK requires at least version 16.8.0 of MSBuild. The current available version of MSBuild is 16.6.0.52201. Change the .NET Core SDK specified in global.json to an older version that requires the MSBuild version currently available.

"/Users/builder/azdo/_work/2/s/xamarin-android/Xamarin.Android.sln" (CreateWorkloadInstallers target) (1) ->
"/Users/builder/azdo/_work/2/s/xamarin-android/build-tools/create-packs/Microsoft.Android.Sdk.proj" (ExtractWorkloadPacks target) (2) ->
  /Users/builder/azdo/_work/2/s/xamarin-android/build-tools/create-packs/Directory.Build.targets(11,31): error MSB4236: The SDK 'Microsoft.DotNet.SharedFramework.Sdk/$(MicrosoftDotNetBuildTasksFeedPackageVersion)' specified could not be found. [/Users/builder/azdo/_work/2/s/xamarin-android/build-tools/create-packs/Microsoft.Android.Sdk.proj]

I believe this wasn't seen on the PR bots as some of them now have the
a newer mono installed which supports MSBuild 16.9 (see PR #5806, which
is still waiting for designer test fixes).

Fix this issue by using dotnet build on macOS when we build the
CreateWorkloadInstallers target.

Finally, I hit a considerable amount of disk space related issues when
testing this PR which will hopefully be alleviated by migrating to the
MicroBuild Virtual Machine Scale Set pool. I've also moved the Build
Asset Registry manifest promotion to the end of the job so that it won't
run multiple times when retrying a failed job.

[pjcollins]

Xamarin.Android CI build queued here for testing: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=4655622&view=results

[pjcollins]

I thought this was ready but it looks like it may be causing a file lock issue on Windows, I'll change this to only use dotnet on macOS instead. I also want to migrate to the VSEng scale set pool which will hopefully reduce the number of "out of disk space" issues I've been seeing...

WARNING: Install failed. Rolling back...
Executing nuget actions took 3.6 sec
Error downloading 'MicroBuild.Plugins.Signing.1.1.368'
There is not enough space on the disk.
##[error]Process 'nuget.exe' exited with code '1'

 (_PackageMsi target) -> 
         light.exe : error LGHT0297: An error (ERROR_DISK_FULL) was returned while creating a CAB file. This means you have insufficient disk space - please clear more disk space and try this operation again. [C:\A\1\43\s\build-tools\create-dotnet-msi\create-dotnet-msi.csproj]
         C:\A\1\43\s\build-tools\create-dotnet-msi\create-dotnet-msi.csproj(106,5): error MSB3073: The command ""C:\Users\dlab14\android-toolchain\wix\light.exe" -ext WixUIExtension -cultures:en-US -sice:ICE03 -sice:ICE61 Microsoft.NET.Workload.Android.wixobj" exited with code 297.

[pjcollins]

New test run: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=4656130&view=results

[jonathanpeppers]

I thought this was ready but it looks like it may be causing a file lock issue on Windows,

What was the locked file?

[pjcollins]

I saw this a couple of times during the "package build and test results" step with the initial changes:

2021-04-15T04:21:02.6537765Z (CopyFilesToOutputDirectory target) -> 
2021-04-15T04:21:02.6540169Z   C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(4455,5): error MSB3027: Could not copy "obj\Release\netstandard2.0\Xamarin.Android.Tools.BootstrapTasks.dll" to "C:\A\1\25\s\bin\BuildRelease\netstandard2.0\Xamarin.Android.Tools.BootstrapTasks.dll". Exceeded retry count of 10. Failed. The file is locked by: ".NET Host (9784)" [C:\A\1\25\s\build-tools\Xamarin.Android.Tools.BootstrapTasks\Xamarin.Android.Tools.BootstrapTasks.csproj]
2021-04-15T04:21:02.6544269Z   C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(4455,5): error MSB3021: Unable to copy file "obj\Release\netstandard2.0\Xamarin.Android.Tools.BootstrapTasks.dll" to "C:\A\1\25\s\bin\BuildRelease\netstandard2.0\Xamarin.Android.Tools.BootstrapTasks.dll". The process cannot access the file 'C:\A\1\25\s\bin\BuildRelease\netstandard2.0\Xamarin.Android.Tools.BootstrapTasks.dll' because it is being used by another process. [C:\A\1\25\s\build-tools\Xamarin.Android.Tools.BootstrapTasks\Xamarin.Android.Tools.BootstrapTasks.csproj]

https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=4655617&view=logs&j=0e8c8f50-6fef-52c9-bd1f-1d6d378b134b&t=12b672b7-d918-51ce-b1d6-7c03e23a056a&l=240