Stryker fails with .NET 8 RC2 SDK

[martincostello]

Describe the bug

Stryker fails to compile code when run with the .NET 8 SDK.

Having updated Polly to use the .NET 8 SDK to build and adopt some C# 12 language syntax to resolve analyser warnings, our mutation tests that use Stryker are no longer working. This was working in earlier previews, but seems to have stopped working since RC1.

The latest attempt at upgrading with minimal changes to reduce the number of variables that could have broken things can be found here.

An older PR that's had various force pushes over the last few months can be found here, which contains more changes related to actually using .NET 8. We tried a few things in that PR to fix things, such as having Stryker use .NET 6, 7 and 8 to see if it was a configuration issue, or something we could side-step.

We have some .NET 8-specific changes we'd like to release in November once 8.0.0 is available, so if we can't get things working we're likely going to have to disable our mutation tests as otherwise it will be a blocker for that release as we have to use the .NET 8 SDK for those changes.

I happened to notice in the last hour that there was a new version of Stryker available, so I'm in the process of applying that update too in case that would resolve it.

Logs

Failed build logs here: logs

I'll attempt to produce a log file at a later point.

Expected behavior

The tests succeed the same way they do for us with the .NET 7 SDK.

Desktop (please complete the following information):

• OS: Ubuntu 22.04
• Type of project: .NET library
• Framework Version: .NET 8.0.100 RC2 SDK
• Stryker Version: 3.10.0

Additional context

None.

[martincostello]

Yep, still fails with 3.11.0 too.

[rouke-broersma]

Usually using preview versions of sdks is tricky because the roslyn package with support for the next tfm is released with the release of the next tfm. So to support a project trying to use preview versions we would have to update to preview versions of roslyn, which is undesirable for obvious reasons.

I'm not sure if that's the case here, but it's a possibility, we'll have to test if upgrading roslyn to a preview version would fix the issues.

[martincostello]

That makes sense. I wonder if the issue is caused by the default C# language version moving from 11 to 12?

I'll try pinning it to 11 to see if that fixes things. If it does, then we can just workaround the issue in the short term at the expense of not being able to use C# 12.

[dupdob]

could you adjust log levels for Stryker? Ideally, using --logToFile and providing us the resulting log?
Stryker fails at analyzing the projects, clearly failing to capture dependencies.
There is also a warning that net7.0 is requested while there is no configuration supporting it.
SDK resolution is tricky and not handled by Stryker, and this is a crucial step.
Changing C# version will not change a bit, I am afraid.

[martincostello]

Logs can be found in the mutation-report artifact for this GitHub Actions workflow run.

[dupdob]

Thanks. I was looking for an error message I though I added a few months ago, but I failed to see it. And I don't see it in Stryker source files. I guess it is in some yet to be merged PR.
That being said, that error message would not have been super helpful, merely a confirmation of what is happening.
Compilation error messages point toward Stryker/Builadalyzer failing to properly capture project references.
We have no clear cause for this as of now.

[martincostello]

With some further experimentation I have found that I can get it to work on Windows (both locally and in GitHub Actions) by adding the following two settings:

{
+  "language-version": "Preview",
-  "target-framework": "net7.0"
+  "target-framework": "net8.0"
}

Stryker still fails on macOS and Linux with these changes.

Could it be related to #2585 somehow?

[mu88]

Hello fellow Stryker developers 👋🏻

After upgrading my project to .NET 8 and using some of C# 12's latest language features (primary constructor), Stryker fails with:

[14:48:21 WRN] Safe Mode! Stryker will try to continue by rolling back all mutations in method. This should not happen, please report this as an issue on github with the previous error message.
[14:48:21 WRN] Stryker.NET encountered a compile error in /home/runner/work/RaspiFanController/RaspiFanController/RaspiFanController/Logic/RaspiTemperatureController.cs (at 4:39) with message: The feature 'primary constructors' is currently in Preview and *unsupported*. To use Preview features, use the 'preview' language version. (Source code: (ITemperatureProvider temperatureProvider,
                                        IFanController fanController,
                                        ITaskCancellationHelper taskCancellationHelper,
                                        ITaskHelper taskHelper,
                                        ILogger<RaspiTemperatureController> logger,
                                        IOptionsMonitor<AppSettings> settings))

I assume that is also a problem related to this issue? Or shall I better create a new issue?

[rouke-broersma]

Hello fellow Stryker developers 👋🏻

After upgrading my project to .NET 8 and using some of C# 12's latest language features (primary constructor), Stryker fails with:

[14:48:21 WRN] Safe Mode! Stryker will try to continue by rolling back all mutations in method. This should not happen, please report this as an issue on github with the previous error message.
[14:48:21 WRN] Stryker.NET encountered a compile error in /home/runner/work/RaspiFanController/RaspiFanController/RaspiFanController/Logic/RaspiTemperatureController.cs (at 4:39) with message: The feature 'primary constructors' is currently in Preview and *unsupported*. To use Preview features, use the 'preview' language version. (Source code: (ITemperatureProvider temperatureProvider,
                                        IFanController fanController,
                                        ITaskCancellationHelper taskCancellationHelper,
                                        ITaskHelper taskHelper,
                                        ILogger<RaspiTemperatureController> logger,
                                        IOptionsMonitor<AppSettings> settings))

I assume that is also a problem related to this issue? Or shall I better create a new issue?

The stable build of the open source roslyn (compiler) package does not yet have support for dotnet 8, so until Microsoft releases this you'll have to set the language version to preview. Unfortunately the release of the compiler package is never done before the release of the new dotnet version and language version so we're always at least a couple days behind...

[rouke-broersma]

#2773

[rouke-broersma]

3.12.0 is being released and should be available within the hour, please let me know if this fixes both issues @martincostello @mu88

[martincostello]

Thanks - I'll update Polly and check once it lands in NuGet.org.

[martincostello]

3.12.0 appears to resolve the issue 🙇

[rouke-broersma]

Let's hope the Roslyn update is released sooner next language update :)

[mu88]

Yes, works like a charm 🥳 thank you very much!

Let's hope the Roslyn update is released sooner next language update :)

@rouke-broersma are the MSFT guys aware of that, do you have filed an issue that the later release leads to problems in downstream repos? Maybe it's worth pointing their attention to that....

[rouke-broersma]

@mu88 I doubt they're aware of this. But they're usually not this late anyway, usually a couple days at most. Most users don't experience issues with this because they're not going to be updating on release day. I doubt they can do anything for those couple users on the bleeding edge as they have more or less the same issue we have. They have to release the proprietary version first, after which they can update and release the open source SDK packages.