Release pipelines should have package pre-install and post-install checks

[danieljurek]

[edit @benbp] Updating title and scope of epic.

Pre-checks

There should be some sanity check of the assets for installation ability before release. In specific, we've encountered an issue where a package depended on an un-released package in the same service.

Given a service (e.g. communication) and two or more packages (e.g. @azure/communication-common and @azure/communication-network-traversal) where one package depends on another package (e.g. @azure/communication-network-traversal depends on @azure/communication-core) it is possible for the source version (not published) of @azure/communication-network-traversal to depend on the source version (not published) of @azure/communication-core and to release the source version of @azure/communication-network-traversal without releasing the source version of @azure/communication-core. In these cases installations of the released @azure/communication-network-traversal will fail because they cannot resolve the dependncy version of @azure/communication-core

In cases where there is significant delay between releasing of a package and the package appearing in the package manager (e.g. Java) it might be easier to fall back to verifying that the dependencies have been "released" by looking for tags in the SDK repo and assuming that a tag present is a sufficient indicator that a dependency has released.

Post-checks

We already do some post-install checks via smoke test on release, where we ensure the package we published exists and is installable. We should extend this check to work against all packages, not just those included in the smoke test projects.

[kurtzeborn]

@danieljurek, shouldn't smoke tests cover this? The packages have to be installed for the smoke tests to run. What are we looking for here that we don't already have with smoke tests or could add there with some tweaks?

[weshaggard]

After a little investigation we have a couple of issues with our smoke-tests as they are currently implemented that I believe we can address and it should detect these issues.

1. Currently smoke-tests don't run if all the packages for a service don't release. We should fix this and run them as part of each individual package release and scope them to the individual package.
2. If there are no samples/smoke tests for a given package we don't run anything. We should add a basic blank test for every package just to be sure that we correctly install/restore the package even if we don't reference the code. This should help ensure we always find basic package issues. This will be similar to the suggestion of installing the package but instead it will install it via a test project instead of a direct install command which will represent a closer customer experience.

[weshaggard]

After further discussion and investigation we should also try to do a local install of the package that we are about to publish before we publish just to be sure that any missing dependencies are caught before we release.

[kurtzeborn]

We're in a pretty stable state at this point. Not sure we would prioritize making changes here unless we're seeing actual cases where we made a release which did not load at all.