Dist Tags for NuGet or advanced pre-release Wildcards

[Danielku15]

Hello there. 👋

I was wondering if the community ever discussed the following usecases:

1. Assume you have a nuget package with multiple levels of maturity but all of those level are considered pre-release. e.g. alpha, beta etc. and a customer wants to consume exactly such a specific level and there the 'latest' version of it.
2. Assume you have multiple teams working on a package in parallel on different branches. From each branch there are packages published with specific pre-release tags like 1.0.0-alpha.team01.build.02. Now a customer wants to consume packages from a specific branch/team and limit the set of pre-release versions to be considered further.

With the current floating-version range possibilities in NuGet none of these use-cases seem to be possible and the whole version ranges and floating version topic is quite sensitive in NuGet.

For case 1 in the NPM world they have introduced dist-tags where you can additionaly flag versions with a tag and clients can consume those tags.

For case 2 I was thinking of more flexible wild-cards in the currently supported ranges like allowing (1.0.*-alpha.team01.*, 1.0.9999) where the * are then the typical placeholders used for alphabetical sorting. We would then even like to parameterize this range with properties like (1.0.*-alpha.$(TeamBranch).*, 1.0.9999).

I think the NPM and the wildcard approach would both fit our needs but there the implication is quite different.

The NPM apporach would mean a large scale adjustment in the NuGet ecosystems as all servers (nuget.org, lokal disk source, Artifactory, GitHub Packages,..) and clients (NuGet.exe, dotnet nuget, Rider) would need support for it.

The wildcard approach is likely something that could be maybe handled on the client side (filtering on the server like today, and then advanced filter on client side).

Disclaimer: This is more an enterprise level requirement where really many teams are working in parallel on multiple products and libraries and it would be the goal not to have clashes between the developed features.

Looking forward to some feedback on this idea and maybe how others have solved similar challenges.

Cheers
Daniel

[erdembayar]

Fyi @joelverhagen @JonDouglas @nkolev92

^ This sounds interesting, similar to github tagging.

[joelverhagen]

It's a really interesting use case. The True Identity of a package is Version + ID + feed (barring mutability complexities). Do you think using a dedicated feed or ID would express this intent better? If we use a tagging aspect, it's almost like a fourth pivot that we can decide package selection based on.

Said another way, we don't have to pivot on version necessarily. We could also consider pivoting on package ID or feed.

Something that should be mentioned is that the * in version ranges means floating version. Calling it a wild card can maybe help a little bit but folks assumption about wild card in software world don't always translate well to NuGet floating versions.

I think encoding this pattern into the package version and using the * character would cause problems. The package consumer might actually want the lowest version matching that pattern rather than opting into the floating version behavior.

If changes to the core of package identity or version range syntax is not desired, the easiest way to achieve this in the current world is to have a dedicated feed per team. This is relatively easy on a Azure DevOps.

[nkolev92]

Do you think using a dedicated feed or ID would express this intent better? If we use a tagging aspect, it's almost like a fourth pivot that we can decide package selection based on.

I think this is a very important point.
The prerelease labels in packages are ordered. Having team01 and team02 basically means that team02s versions are always "newer".
The version is probably not for this scenario.

[Danielku15]

Do you think using a dedicated feed or ID would express this intent better? If we use a tagging aspect, it's almost like a fourth pivot that we can decide package selection based on.

A dedicated feed is likely not easy achievable by everyone and would be likely also complex to manage. Managing different feeds on multiple branches to guarantee correct restores gives quite a complexity and uncertainty.

Multiple IDs could be a technical solution but I think it would be rather a workaround. Semantically it is the same package but they are just different versions built from different branches, having multiple different packages from team specific branches does not feel right to me.

If we use a tagging aspect, it's almost like a fourth pivot that we can decide package selection based on.

This is sad but true. Additional tags need quite wide-ranging support and development. The NuGet registries/feeds need to support them on the APIs to tag and filter packages additionally. Nevertheless the tagging seems to have the widest adoption compared to a wildcard approach (NPM, Git, Docker, more?). Conan has the support for additional users and channels as part of the references.

Also I think considering that tags are rather fixed pointers with a name (and not floating) versions, the resolution is fairly simple: Always use the exact version to which the tag is pointing. But the extensions in all the tooling around NuGet (registries, clients, UIs, ..) needs extension.

The prerelease labels in packages are ordered. Having team01 and team02 basically means that team02s versions are always "newer".
The version is probably not for this scenario.

This lately was flaw we have fixed in our setup. 😅 We changed from 1.0.0-alpha.. to 1.0.0-alpha.. to ensure ordering is respected correctly and later builds win. With a filtering mechanism where we would point with a tag to the latest version of a certain team this would not have happened.

Something that should be mentioned is that the * in version ranges means floating version. Calling it a wild card can maybe help a little bit but folks assumption about wild card in software world don't always translate well to NuGet floating versions.

Generally we would be fine with the approach with only floating (not wildcard) approaches but having the support of more complex patterns as described. Some examples:

• 1.0.0-*.team Resolving to the latest any prerelease version of a specific team
• 1.0.0-alpha.* Resolving to the latest alpha builds of a any team
• 1.0.0-alpha.*.team01 Resolving to the latest alpha builds of a specific team
• 1.0.0-alpha.team01.* Resolving to the latest alpha builds of a specific team
• 1.0.0-alpha.*.$(TeamName) Resolving to the latest alpha builds of a specific team given by MSBuild Property
• 1.0.0-alpha.$(TeamName).* Resolving to the latest alpha builds of a specific team given by MSBuild Property

This way no additional tagging is needed, but the prerelease chunks can be used as tags. This would not allow us to have many independent tags pointing to a specific verison, and also not tags pointing to release versions, but this would rather be an extension to the existing resolution strategies. The current one to just allow * and not additional substrings is the limiting part.

I haven't thought through yet any details in this approach like how it should be used with together with version ranges.

If changes to the core of package identity or version range syntax is not desired, the easiest way to achieve this in the current world is to have a dedicated feed per team. This is relatively easy on a Azure DevOps.

I think this is quite a limiting approach and not possible in many enterprise setups. We have strict rules in regards of available feeds and registries and also only those selected ones are reachable from our CI/CD environment to avoid supply chain attacks. We are not using Azure DevOps and also not GitHub Package Registry but a JFROG Artifactory instance with department wide guidelines.

Also managing those feeds across the branches is a very tricky thing to achieve because you have to ensure that as part of branching/merging you need to guarantee that each branch keeps its correct feed configuration. There might be also possibilities we could attempt (e.g. using environment variables) but it is already not compliant with our enterprise restrictions hence we cannot go this path.