Migrating from .NET 3.2 to .NET 5.0 - Using DefaultIfEmpty() in Query Returns System.ArgumentNullException: 'Value cannot be null. (Parameter 'other')'

[benhysell]

Repo: https://github.com/benhysell/V.EfCoreIssue

Two projects in the repo, one using .NET Core 3.2 and one using .NET 5.0.

Two entities defined in the system, Organization and Hierarchy. Note these are contrived and pulled from a much larger project, but boiled down to:

• Each one has a Name field and a Guid as the Id.

• Organization has a single Hierarchy

modelBuilder.Entity<Entities.Organization>().HasOne(x => x.Hierarchy).WithMany().HasForeignKey(x => x.HierarchyId).OnDelete(DeleteBehavior.Restrict);

• Attempting to query for all of the Hierarchies that match a given Organization in .NET 5.0 fails but runs without issue in .NET 3.2

var organizationId = Guid.Empty;

var foundOrg = (from organizations in applicationDb.Organizations.Where(x => x.Id == organizationId)
                           select organizations.HierarchyId);

var hiearchies = (
                            from organizationHiearchies in foundOrg.DefaultIfEmpty()
                            from hierarchy in applicationDb.Hierarchies.Where(x => x.Id != organizationHiearchies)
                            select hierarchy).ToList();

Returns System.ArgumentNullException: 'Value cannot be null. (Parameter 'other')'

Stack traces

System.ArgumentNullException
  HResult=0x80004003
  Message=Value cannot be null. (Parameter 'other')
  Source=System.Collections
  StackTrace:
   at System.Collections.Generic.HashSet`1.UnionWith(IEnumerable`1 other)
   at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.ColumnExpressionFindingExpressionVisitor.FindColumns(SelectExpression selectExpression)
   at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.Prune(IReadOnlyCollection`1 referencedColumns)
   at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.Prune()
   at Microsoft.EntityFrameworkCore.Query.Internal.SelectExpressionPruningExpressionVisitor.Visit(Expression expression)
   at Microsoft.EntityFrameworkCore.Query.RelationalQueryTranslationPostprocessor.Process(Expression query)
   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass9_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.GetEnumerator()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at V.EfCoreIssueNet50.Program.<Main>d__0.MoveNext() in C:\jobs\V.EfCoreIssue\V.EfCoreIssueNet50\Program.cs:line 45

Include provider and version information

EF Core version: 5.0.1
Database provider: Microsoft.EntityFrameworkCore.SqlServer
Target framework: .NET 5.0
Operating system: Windows 10
IDE: Visual Studio 2019 16.8.3

[smitpatel]

Fixed in 0b31650

[benhysell]

@smitpatel excellent if this is fixed....can you provide a little bit of insight on what package version will include this fix? I lost the thread trying to figure out if the above fix is going to make it into 5.0.2 or it is a 6.0.0 release.

Thanks!

[smitpatel]

6.0.0 release.

[benhysell]

@smitpatel

1. Is this listed as a breaking change and I misread the release notes?
2. If I want to rewrite all of my queries to avoid this issue is there an easy way to find it? Everything above compiles fine.

[smitpatel]

1. It's a bug, not an intentional breaking change. We will discuss in next team meeting if we want to patch this for 5.0.3
2. I haven't done full investigation to find the root cause what specific part of query causing the error. Though above query can be rewritten in following way

from hierarchy in applicationDb.Hierarchies
where hierarchy.Id != applicationDb.Organizations.Where(x => x.Id == organizationId).Select(e => e.HierarchyId).FirstOrDefault()
select hierarchy).ToList();

Assumptions made

• foundOrg will have either 0 or 1 value since the predicate is on Id which must be a PK.
• After applying DefaultIfEmpty, it will always be 1 result (with actual value or a default value)
• Doing cross join with 1 row result (and also discarding that row) does not require a cross join.

[benhysell]

@smitpatel Thanks for the explanation and taking the time to look at it.

+1 vote for 5.0.3, trying to take advantages of the advancements Blazor made in 5.0.0 when I came across this bug.

[benhysell]

@ajcvickers Is the current timetable for 6.0.0 release November 2021 or is ef core going to move at a different cadence?

[ajcvickers]

@benhysell We discussed this issue in triage and decided that for now it doesn't meet the bar for servicing in a patch release. Specifically, the issue has a reasonable workaround, is in the in-memory database provider only, and is so far reported by a single customer.

EF Core follows the .NET release cadence. The next release will be 6.0 in November 2021.

[benhysell]

@ajcvickers

• I'm hitting this issue with the MS SQL provider, I can't speak for the in-memory provider
• I'm disappointed this is getting pushed out...in my opinion it is a regression from 3.2 and preventing me from moving forward to 5.0 without engineering on my part to track all of these guys down.

Appreciate everyone taking the time to look at this and the responses.

[ajcvickers]

@benhysell Thanks for clarifying; we will discuss again.