.Net Framework tests fails when .Net 8 is present on system

[tkvalvik]

I have a project that targets both .Net 7 and .Net Framework 4.7.2.

I installed the .Net 8 alpha for an unrelated purpose, and unit tests for .Net Framework will no longer run.

Using:
NUnit 3.13.3
NUnit3TestAdapter 4.3.1
.Net 7.0.102 and .Net Framework 4.7.2

Projects global.json:

{
	"sdk": {
		"allowPrerelease": false,
		"version": "7.0.100",
		"rollForward": "latestMinor"
	}
}

Installed SDKs

PS C:\r\MyProject> dotnet --list-sdks
3.1.426 [C:\Program Files\dotnet\sdk]
6.0.100-rc.2.21505.57 [C:\Program Files\dotnet\sdk]
6.0.103 [C:\Program Files\dotnet\sdk]
6.0.113 [C:\Program Files\dotnet\sdk]
6.0.308 [C:\Program Files\dotnet\sdk]
6.0.405 [C:\Program Files\dotnet\sdk]
7.0.100-rc.2.22477.23 [C:\Program Files\dotnet\sdk]
7.0.101 [C:\Program Files\dotnet\sdk]
7.0.102 [C:\Program Files\dotnet\sdk]
8.0.100-alpha.1.23059.8 [C:\Program Files\dotnet\sdk]

Requested version is correctly used

PS C:\r\MyProject> dotnet --version
7.0.102

NUnit to run with .Net 8 anyway?

PS C:\r\MyProject> dotnet test MyProject\Tests\bin\Tests.dll
Microsoft (R) Test Execution Command Line Tool Version 17.4.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Exception System.TypeInitializationException,    Exception thrown executing tests in C:\r\MyProject\Tests\bin\Tests.dll
The type initializer for 'NUnit.Engine.Services.RuntimeFrameworkService' threw an exception.
   at NUnit.Engine.Services.RuntimeFrameworkService.ApplyImageData(TestPackage package)
   at NUnit.Engine.Services.RuntimeFrameworkService.SelectRuntimeFramework(TestPackage package)
   at NUnit.Engine.Runners.MasterTestRunner.GetEngineRunner()
   at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnitEngine.NUnitEngineAdapter.Explore(TestFilter filter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\NUnitEngineAdapter.cs:line 97
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, IGrouping`2 testCases, TestFilter filter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 282
InnerException: System.ArgumentException: Unknown framework version 8.0
Parameter name: version
   at NUnit.Engine.RuntimeFramework.GetClrVersionForFramework(Version frameworkVersion)
   at NUnit.Engine.RuntimeFramework..ctor(RuntimeType runtime, Version version, String profile)
   at NUnit.Engine.RuntimeFramework.GetNetCoreRuntimesFromDirectoryNames(IEnumerable`1 dirNames)
   at NUnit.Engine.RuntimeFramework.FindDotNetCoreFrameworks()
   at NUnit.Engine.RuntimeFramework.get_AvailableFrameworks()
   at NUnit.Engine.Services.RuntimeFrameworkService..cctor()
No test is available in C:\r\MyProject\Tests\bin\Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

Additionally, path to test adapters can be specified using /TestAdapterPath command. Example  /TestAdapterPath:<pathToCustomAdapters>.

Expectation: Tests run with the sdk and runtime defined by the project, regardless of which unrelated sdks might be present on the system.

[OsirisTerje]

Hi! It is correct that the engine does not support .net 8. That should be fixed with engine version 3.16.2, which is out now. It should not be invoked when targeting net 7, as you do, but when it has been installed, it is picked up by the engine and throws the exception you see above. So it doesn't try to run with net 8. The suppression with global.json has unfortunately no effect either, since the engine enumerates the sdk's. I'll try to get a fix out asap. In the meantime, you'll have to uninstall the net 8.

But thanks a lot for raising this issue!

[madelson]

@OsirisTerje is there a timeline for this?

[OsirisTerje]

We need a release for the engine first, so i am pushing for that. When that's done, the adapter comes the day after.

[rprouse]

This should be fixed once we release an update using the latest version of the NUnit engine.

[OsirisTerje]

@tkvalvik @madelson Version 4.4.0-beta.1 is out now https://www.nuget.org/packages/NUnit3TestAdapter/4.4.0-beta.1 . It includes support for .net 8. Appreciate it if you have the time to verify it works as it should.
Once verification is done, the non-beta will be published.

[tkvalvik]

Ran the test suites on a mixed .Net 7 and .Net Framework 4.7.2-project.

"Works on my machine"

[timcassell]

So, strangely, this works for me with 4.4.0-beta.1, but errors with 4.5.0.

[OsirisTerje]

@timcassell
Can you please upload a repro project?

You can a) Add it as a PR to https://github.com/nunit/nunit3-vs-adapter.issues, add it as a separate folder named IssueXXX where the XXX is the issue number, or b) Add it as a zipped attachment here , or c) Add it to your own repo and just post the link.

Make it as small as you can, it should compile "as is".

[timcassell]

Well, I'm not able to repro anymore. I uninstalled the .Net 8 version I had on my machine (I think it was preview 6). I installed it again to try to repro again, but it stopped erroring (both with preview 6 and the latest RC2).

[Rene-Sackers]

Updating our NUnit3TestAdapter NuGet packages in our test projects from 4.3.1 to 4.5.0 fixed the issue.