slnx support in the dotnet CLI

[baronfel]

The dotnet CLI should support the new slnx format for building and in the existing solution management commands. It should also help interested users migrate to the new format.

The new format

slnx is an XML-based format that simplifies the current sln file format. When released, it will have an open-source parser, so tools like MSBuild and the dotnet CLI can consistently operate on the format. The new format is intended to reduce common customer pains like merge conflicts and readability, but not to drastically change the experience of working with solutions.

dotnet experiences using solutions

There are three primary ways that the dotnet CLI interacts with solution files today

• building solutions
  • dotnet build myapp.sln
• adding or removing projects and other files from solutions
  • dotnet sln myapp.sln add src/migrations
• creating new solutions
  • dotnet new solution -n lodestone

Each of these should be made to work with the new format to some degree. In addition, a fourth new capability should be added:

• migrating from sln to slnx
  • dotnet sln <sln file> migrate

Building solutions

Status: Done, will be shipping in .NET 9.0.200. Available in VS 17.13 preview 2/.NET SDK 9.0.200-preview

This should be mostly transparent to the dotnet CLI. Much like solutions today, building a solution involves passing the path to the solution file to the MSBuild engine, which has the sole responsibility of converting the build configurations into a 'metaproject' - a kind of MSBuild representation that MSBuild can actually execute - and then building the requested targets on that metaproject.

The same process would hold with slnx - the CLI would forward along the slnx file provided (if any) and MSBuild itself would translate that file into a metaproject and execute that metaproject. Very few changes should be required in the CLI codebase to support this. MSBuild's tracking issue for this is dotnet/msbuild#10266.

Manipulating solution content

Status: In progress - see the linked PRs below.

The CLI has several commands that allow for adding and removing projects in a solution, as well as listing the existing projects. All of these commands should work with slnx as well. This is the area that will require the most investment. The CLI will need to

• learn that slnx files are valid inputs to these commands
• provide some kind of alternative implementation for these commands that works on slnx files
• pivot the implementation used based on the file type provided

These commands allow for selection of the solution file. If invoked in a location where multiple potential solution files are present, the command should error and prompt the user to choose one of the possible solutions.

We'll also need to invest in test coverage to make sure we have parity between our sln support and slnx support.

Creating new solutions

Status: Done, will be shipping in .NET 9.0.200. Available in VS 17.13 preview 2/.NET SDK 9.0.200-preview

The CLI currently ships a solution template that create a new, barebones solution file.

We should provide a template that can create an empty slnx file for users to begin with. The new slnx template should use UTF-8 without a BOM. One major question: should we supplant the existing solution template to create an slnx file? Should the old solution format be accessible via a template parameter?

Migrating existing solutions

Status: Done, will be shipping in .NET 9.0.200. Available in VS 17.13 preview 3

An entirely new capability to migrate a sln file to a new slnx file should be implemented as well to ease onboarding and allow for automation. This command would load the existing sln file and analyze it, translating it into an equivalent slnx file. Ideally any data that matches the default conventions of the new slnx format (for example, default build configurations like Debug and Release) could be omitted from the generated slnx file. The migrated slnx file will be written alongside the existing solution to allow for easy reverting/recovery of the change. The solution file to act upon will follow our existing SlnArgument pattern - if an exact path is used, use that sln. If a directory is provided, and there is only one solution file, use that solution file. If a directory is provided and there are multiple solution files, error.

Note that the addition of this slnx file will likely cause 'bare', unspecified MSBuild commands (like dotnet build without an explicit solution argument) as well as dotnet sln commands with no arguments to fail when used in the working directory - this is acceptable. Users can test by explicitly building the new slnx, and then delete the previous .sln file when they are satisfied with the behaviors.

References

• Twitter Survey
• Mastodon survey

[glen-84]

Regarding the solution template, would it possible to exclude the byte order mark (BOM) now, or are there still known issues with VS or msbuild?

I know that:

1. The Unicode Standard does not recommend its use.
2. Even applications like Notepad now use UTF-8 without BOM by default. (source)

[baronfel]

@glen-84 I'm inclined to make that change, but would like to discuss with @rainersigwald and the rest of the MSBuild team about any gaps there.

[rainersigwald]

For .sln I would leave it as it has been (I believe that's the longstanding Windows 16-bit chars with BOM).

For .slnx we should definitely avoid the BOM but would need to check with the VS folks to make sure they agree.

[baronfel]

I've reached out to the VS side to check compatibility with no-BOM.

[baronfel]

The VS team confirmed that slnx defaults to UTF-8 No-BOM, so we are clear to default the slnx template contents to no-BOM as well. I've updated the description to match this.

[vaind]

Is there any concrete plan for this? I'm looking forward to build/test/restore support for slnx (also when referenced from a .slnf) so that we can update Sentry .NET SDK to use the new format.. Adding projects to the old format is so cumbersome when you need to keep a PR reviewable.

[baronfel]

We are still waiting for an official parser library for the new format to be made available by our partner teams. We're unable to make any progress without that library (and the stabilized format spec implied by that)

[kasperk81]

related microsoft/slngen#585

[kasperk81]

if the plan is still to include slnx support in sdk 9.0, is there any estimate when parsing library is going to be available on github?

[baronfel]

Not at this time, no.

[kasperk81]

it will have an open-source parser

even if it's not fully ready it can still be open sourced today and people can help fixing the remaining bugs in a few hours. someone did similar work starting five months ago https://github.com/winscripter/Slnx so they can collaborate with community

[ssg]

dotnet test also parses sln files to find test projects and fails to work with slnx, fyi.

[aradalvand]

Still waiting for this.
Stuff like this makes it feel like Microsoft still sees non-Visual-Studio users of .NET as second-class citizens; which is disappointing after all these years.

[ite-klass]

Still waiting for this. Stuff like this makes it feel like Microsoft still sees non-Visual-Studio users of .NET as second-class citizens; which is disappointing after all these years.

It's not like slnx works very well in VS. It's an experimental preview feature. And even with VS, CI requires dotnet CLI.

[chucker]

Stuff like this makes it feel like Microsoft still sees non-Visual-Studio users of .NET as second-class citizens

It's not a non-preview feature yet in VS (I believe the release notes still don't mention it at all), and the API isn't considered stable.

[baronfel]

A status update for those following this issue:

• The new parser is now open-source, and the repository can be found at https://github.com/microsoft/vs-solutionpersistence
• MSBuild has a PR that enables support for building slnx files at .SLNX format support msbuild#10794
• We will have to wait for the slnx parser to be onboarded to source-build-externals in this PR before we can merge the MSBuild PR.
• The SDK codeflow for the slnx functionality is [release/9.0.2xx] Update dependencies from dotnet/msbuild #44283

Our current plan is to try to merge this change and get support for building slnx files from the .NET SDK in a preview release of the 9.0.200 SDK, but that is subject to some timelines that we are still navigating.

[kasperk81]

Proposal:
dotnet new sln -n lodestone -f slnx short for dotnet new solution --name lodestone --format slnx

In .NET 9.0, the default format (-f) is still .sln. However, by .NET 10.0 or 11.0, consider making .slnx the default format to encourage developers to upgrade to Visual Studio versions that would fully support it by then. This gradual switch ensures backward compatibility while pushing adoption of the cleaner, modern solution format.

[baronfel]

@kasperk81 From the issue description:

One major question: should we supplant the existing solution template to create an slnx file? Should the old solution format be accessible via a template parameter?

Switching the defaults for .NET 10 is a fine plan, and is what we had been internally considering anyway.

[richshadman]

Understanding the support policy for .NET favors the even releases, once this is out in the wild with the the .NET 9 release, will the slnx support be brought back into .NET 8, or will we have to wait for 10 for LTS support with slnx in the sdk?

The slnx format being so much simpler could be very useful for me with complex CICD scenarios - thinking of dynamic slnx files on the fly to target unit tests projects based on code changes, things of that nature. As we are currently bound to the LTS release for maintainability reasons, use of this feature for our team would possibly depend on it being brought into .NET 8 (or wait until 10) as we try to target the same sdk as our target framework (especially for containerized builds and tests where we start with the sdk base image).

[chucker]

The slnx format being so much simpler could be very useful for me with complex CICD scenarios - thinking of dynamic slnx files on the fly to target unit tests projects based on code changes, things of that nature.

Semi-OT, but it sounds like https://github.com/microsoft/slngen might help you.

[kasperk81]

that also doesn't support slnx yet microsoft/slngen#585

[chucker]

@kasperk81 I know. But the CI/CD, unit test, etc. use case sounded a bit like what slngen is trying to solve.

[richshadman]

I'm actually solving it currently with dotnet new sln and dotnet sln add - works well enough for now. however generating the solution file without being dependent on dotnet cli would give us more flexibility, and slnx is a much simpler format towards that end. Also easier to parse for a human.

[rainersigwald]

@richshadman have you considered using MSBuild projects that point to the projects you care about (maybe using Microsoft.Build.Traversal, instead of synthesizing solutions? Then you could have MSBuild conditions, and it'd probably be generally easier to debug.

[alrz]

Microsoft.Build.Traversal

It did not occur to me that projects could be used this way, I mean at this point what is the purpose of slnx/sln file when you can have a "meta project" (which already supports microsoft/vs-solutionpersistence#61 as well)?

[rainersigwald]

@alrz IMO .sln/.slnx are only for Visual Studio/C# Dev Kit support.

[alrz]

I understand, but I suppose slnx needed dedicated support in tooling as well its own design and implementation. In my opinion it would have been ideal to add support for the sdk-based Traversal projects that existed for some time now instead of adding a completely new format... Just my 2c.

[richshadman]

@richshadman have you considered using MSBuild projects that point to the projects you care about (maybe using Microsoft.Build.Traversal, instead of synthesizing solutions? Then you could have MSBuild conditions, and it'd probably be generally easier to debug.

No I have not - I will look into that.

VS 17.12 was released today. Any update on .NET 8 support for slnx by chance?

[JohnGalt1717]

I can tell you that in 17.12 and 17.13 P1 don't even support SLNX for jumplists properly, and the upgrade tool also doesn't work.

It's a half-baked mess.

[bamancio-futura]

how is this still "untriaged" ?

[kasperk81]

sln-add is now available in the daily (net10) build, and I’ve got some strong opinions about the mandatory random GUID it’s adding to each <Project node: #44570 (comment). What do you all think?

The whole point of .slnx was to avoid GUIDs wherever possible. Without the Id={guid} part in <Project Path="{path}" Id="{guid}" />, Visual Studio opens the solution just fine, and MSBuild has no issues building it. So, why is the SDK randomly generating project GUIDs for .slnx?

Currently, if a project file has a <ProjectGuid> (legacy) property, it uses that value, which is fine. The problem is when <ProjectGuid> isn’t present (which is a common occurrence in modern projects) it randomly generates a GUID, and that’s not the desired behavior.

[julealgon]

sln-add is now available in the daily (net10) build, and I’ve got some strong opinions about the mandatory random GUID it’s adding to each <Project node: #44570 (comment). What do you all think?

Oh boy please don't add guids back in this, that substantially defeats the purpose of the new format.

[edvilme]

sln-add is now available in the daily (net10) build, and I’ve got some strong opinions about the mandatory random GUID it’s adding to each <Project node: #44570 (comment). What do you all think?

The whole point of .slnx was to avoid GUIDs wherever possible. Without the Id={guid} part in <Project Path="{path}" Id="{guid}" />, Visual Studio opens the solution just fine, and MSBuild has no issues building it. So, why is the SDK randomly generating project GUIDs for .slnx?

Currently, if a project file has a <ProjectGuid> (legacy) property, it uses that value, which is fine. The problem is when <ProjectGuid> isn’t present (which is a common occurrence in modern projects) it randomly generates a GUID, and that’s not the desired behavior.

THanks for pointing this out! You are absolutely right, this is addressed in #45366

[philipp-naused]

It looks like .NET SDK 9.0.200-preview.0.24575.35 is now included in Visual Studio 2022 17.13.0-pre.2.0.
Both the dotnet.exe and MSBuild.exe included in this preview support the new format.

[kasperk81]

9.0.200-preview.0.24575.35 is missing sln-add and sln-list command support for slnx. 9.0 backport is in progress #45408. last part (sln remove) is also in progress #45160. matter of few more days