4.4.0 breaking change for TestAssemblyLoadContext #1066
Comments
|
I think I'm having the same issue. I've created a test solution at https://github.com/paulhickman-a365/Nunit440Bug I'm using a .Net 6 project with VS2022 17.4.4. If you run the test with 4.4.0 if fails, but 4.3.1 is fine. Foo, the class being tested is: using Microsoft.Extensions.DependencyInjection;
namespace OtherAssembly
{
public class Foo
{
public static void Bar(IServiceCollection serviceCollection)
{
}
}
}The test which loads the assembly that Foo is in dynamically is: public class FooTest
{
[Test]
public void SystemWebServicesTest() => RunTest(
"..\\..\\..\\..\\OtherAssembly\\bin\\Debug\\net6.0",
"OtherAssembly.dll"
);
private void RunTest(
string assemblyFolder,
string assemblyName)
{
//If we are running a release build of the tests, look for release builds of the rest of the code
if (Environment.CurrentDirectory.Contains("\\Release\\", StringComparison.CurrentCulture))
{
assemblyFolder = assemblyFolder.Replace("\\Debug\\", "\\Release\\");
}
string assemblyPath = Path.Combine(assemblyFolder, assemblyName);
if (!File.Exists(assemblyPath))
{
Assert.Fail($"{assemblyPath} does not exist. Have you built the full solution?.");
return;
}
var context = new AssemblyLoadContext("test", true);
context.EnterContextualReflection();
context.Resolving += (context, childAssemblyName) =>
{
var tryPath = Path.Combine(assemblyFolder, childAssemblyName.Name + ".dll");
if (File.Exists(tryPath))
{
return context.LoadFromAssemblyPath(Path.Combine(Environment.CurrentDirectory, tryPath));
}
return null;
};
try
{
var assembly = context.LoadFromAssemblyPath(Path.Combine(Environment.CurrentDirectory, assemblyPath));
var foo = assembly.GetTypes().FirstOrDefault(t => t.Name == "Foo");
var bar = foo.GetMethod("Bar", BindingFlags.Static | BindingFlags.Public | BindingFlags.NonPublic);
var services = new ServiceCollection();
bar.Invoke(null, new[] { services }); //4.4.0 test adaptor fails here, 4.3.1 works
context.Unload();
}
finally
{
AssemblyLoadContext.Default.EnterContextualReflection();
}
}
}The error message is: |
|
@paulhickman-a365 @dave-yotta Does this happen on both Windows and Linux ? We have another issue reported today that seems to only happen on Linux. They look related. #1065 Thanks for the repro @paulhickman-a365 , I'll have a look at it. Can you also just dump the stack traces you see ? |
|
This happens on both Windows and Linux for me. |
|
@Methuselah96 What code do you have where this happens? Can you provide a short repro ? |
|
@paulhickman-a365 Trying your repro on Windows, and it works there. |
I accidentally pushed the repo on 4.3.1. I've updated the project reference to 4.4.0 now. I'm using Windows Server 2022. |
|
@paulhickman-a365 Thanks! Then it failed on Windows! |
|
@paulhickman-a365 The adapter 4.4.0 uses NUnit.Engine 3.16.3. This version of the engine uses Microsoft.Extensions.DependencyModel version 3.1.0 (in order to be compatible with net core 3.1, which we still support). If I change the references you have to the Microsoft.Extensions.DependencyInjection.Abstractions to the same version 3.1.0 (since I assume these are a "package deal"), then your code show compiler errors that look similar to the runtime errors. Since adapter 4.3.1 doesn't use this Microsoft package, it works. So, it looks like we need to rethink package version or possible the way this is solved in the engine |
|
Same issue here, SF reports a lot of failed tests with System.InvalidCastException (inhouse injection system) it happens when casting instances created using reflexion to a certain type. Like "var castedInstance = (T)instance". Works perfectly in 4.3.1. Kaboom in 4.4.0. |
|
It looks like the class ServiceCollection is in the assembly Microsoft.Extensions.DependencyInjection for versions <=5 but was moved to Microsoft.Extensions.DependencyInjection.Abstractions for version 6+ I'm not sure I'm my issue is the same as those others are reporting anymore as it seems specific to ServiceCollection |
|
@paulhickman-a365 It may explain it though, as I do see a lot of different crashes, and they all boil down to the loading of assemblies, one way or another. We have relied on using the same packages for all, keeping them at a low version, but also to avoid taking dependencies on too many external packages. I suspect this is another one of those where it doesn't work. So, we either have to package in for different frameworks, or not use this package at all. The latter seems more tempting. PS: @daviddenis-stx What is SF ? Does the cast error happens also when not using your inhouse system - if so, can you provide a repro? |
|
Sorry, Software Factory. Upgrading the nuget package made the factory unit tests go red in any part using some degree of reflection :) NUnit3TestAdapter 4.3.1 green results, NUnit3TestAdapter 4.4.0 red results. It fails in the region below, basically we just created an instance of something using type lookup, then calling the constructor (the instance is created correctly, that is it finds the assembly, the constructor, and invokes it nicely). Then it tries to cast the received "object" to T and fails with InvalidCastException. For example an instance of class MyThing : IMyThing will fail on the cast line, throwing an exception "Cannot cast to IMyThing". I suspect there's something weird with the reflexive loading of assemblies and the loading of types in 4.4.0 ? |
|
Perhaps the bump to engine 3.16.2 ? I can see this thing is quite new https://github.com/nunit/nunit-console/commits/485153dd0b414f71e2432aadc1028afce114e7cf/src/NUnitEngine/nunit.engine.core/Internal/TestAssemblyLoadContext.cs |
|
@daviddenis-stx Yes, I think that's where it happens, and you see that the MS package was introduced there. |
|
Is it possible to workaround with a binding redirect? I tried adding <?xml version="1.0" encoding="utf-8" ?>
<configuration>
<dependentAssembly>
<assemblyIdentity name="Microsoft.Extensions.DependencyInjestion.Abstractions"
publicKeyToken="adb9793829ddae60"
culture="en-us" />
<bindingRedirect oldVersion="3.1.32.0" newVersion="7.0.0.0" />
</dependentAssembly>
<appSettings>
<add key="foo" value="bar"/>
</appSettings>
</configuration>But I couldn't make it read the app setting from the app.config regardless if I called it that or Nunit440Bug.config as per http://charliepoole.org/technical/how-nunit-finds-config-files.html so I'm not sure if the binding direct won't workaround it or if the repo isn't reading the config file at all |
|
I dont think binding redirects will help in my case: This type: Cannot be casted to this type: So the only difference here is the load context. |
|
@rprouse We need a strategy here. Any thoughts? |
|
@daviddenis-stx Does it happen for any type of cast? |
|
@Methuselah96 @dave-yotta @daviddenis-stx @paulhickman-a365 |
This nuget package seems to solve the issues, good job ! |
|
@OsirisTerje .NET 3.1 is EOL, so we could retire it as a target. Looking at this, nunit/nunit-console@3f854b2#diff-b82806cbe6520a13c308f28d24d01cdbd2639024609d411d529b5022bbe9401bR30, |
|
@rprouse I agree. I did a quick look at retiring both net fw 2.0 and .net core 3.1, but it is quite some job to do that. |
|
@OsirisTerje thanks for the quick fix. any chance of getting that fix deployed? as a beta would suffice |
|
@SimonCropp I'll need to push out a beta (or even alpha) of the NUnit.Engine too. No problem getting the adapter out, as I have all the rights to do that, but I don't think I have similar rights for the Engine project, in particular on the Nuget feed...... hmm... perhaps I do... but would be nice if @rprouse could approve me doing that. |
|
@OsirisTerje should 4.4 be unlisted? it broke approx 50% of projects i used it on |
|
@SimonCropp I normally don't unlist, just add a hotfix and information in the release notes. I'll plan to get a hotfix 4.4.1 out today with the same code as in the dev version enclosed in #1066. It will probably use a locally built version of the engine, based on 3.15.2 but with the fix for the .net 8. Can you test on some of your projects with NUnit.Console 3.16.3. You should get the same crashes there. Imho we should get a hotfix out there too (Issue nunit/nunit-console#1324) . I believe also the 3.16.2 is broken. For anyone using the NUnit.COnsole in addition, they would need to then try the 3.15.2 version to work together with the upcoming 4.4.1. |
|
Hotfix version 4.4.2, nearly identical to the alpha version attached yesterday, is now released. The difference from the attached version is that the Microsoft.Extensions.DependencyModel is not include in the adapter package. It is not needed there, so it was removed. The release notes is here https://docs.nunit.org/articles/vs-test-adapter/AdapterV4-Release-Notes.html (includes some more explanations) and the package is uploaded to nuget https://www.nuget.org/packages/NUnit3TestAdapter/4.4.2. Thanks, everyone for reporting and providing repros! |
|
@OsirisTerje can confirm the new version works. thanks |
|
Thanks for the confirmation @SimonCropp :-) |
|
☝️ different exception, but likely the same cause. I also recommend unlisting 4.4.0. |
|
@SimonCropp @gregsdennis Ok, Ok, you guys win! It's unlisted!! |
|
v4.4.2 is working thanks! |
Looks like it may be specific to roslyn generated types:
When reporting a bug, please provide the following information to speed up triage:
Scriptexecution that comes from the test load context, while the script was compiled in the Default context.dotnet testrunning inside a docker container mcr.microsoft.com/dotnet/sdk:7.0The text was updated successfully, but these errors were encountered: