Upgrades should be tested where applicable during Installation Validation

[jedieaston]

Description of the new feature/enhancement

I know this will eat compute time so it's fine if it can't happen, but I wonder if upgrades (once they are stable) should be tested in the pipeline. It seems like a lot of users use winget mostly for winget upgrade --all so we should probably make sure that works, especially since the client is going to start telling the user why upgrades aren't working (microsoft/winget-cli#1649). If we could bubble those messages up to the logs and give warnings when upgrades are unsuccessful, it would help a lot! (because maybe we can't avoid upgrade errors because of installer types, but maybe we can?)

Proposed technical implementation details (optional)

Only note I have here is that upgrades should only be tested for installers that don't use vanity URLs (so not Chrome or Spotify). Although those get upgraded sometimes via winget, we can't get the older installers. We can probably figure this out by checking to see if the commit is replacing files or not.

[solomoncyj]

relatable. once i did a PR for an application that pushed updates into the same URL which caused hashing problems. luckily it was caught by the moderators so I could update the commit.

[denelon]

I thought I had already created this issue, but it looks like I haven't. We've been discussing how we would test the upgrade scenario for some time now. There are a few edge cases that make this a bit tricky to capture the challenges with multiple packages that seem to match what's installed. Teams seems to be one of the most common with the machine wide installer and the application.

We do intend to build some logic to test the upgrade scenario. It's likely we will use logic like:

If a previous version exists in the repository, perform and install, and then perform an upgrade with the version submitted.

We will also likely need to look and see if an even "newer" version is available than what is submitted so we can test the upgrade from the newly added version to either the next newer version and / or the latest version. However, there is still some internal debate on this.

[Trenly]

@denelon - can you label "Area-Validation-Pipeline"