Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EF Core joining on the wrong field when using GroupBy #9481

Closed
olivierr91 opened this issue Aug 19, 2017 · 6 comments
Closed

EF Core joining on the wrong field when using GroupBy #9481

olivierr91 opened this issue Aug 19, 2017 · 6 comments
Assignees
Labels
closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. type-bug
Milestone

Comments

@olivierr91
Copy link

olivierr91 commented Aug 19, 2017

Here is probably a big bug introduced in EF Core 2.0. The database is PostgreSQL.

I have this code, which returns a result in EF Core 1.2, and gives no results in EF Core 2.0:

IEnumerable<IGrouping<Sport, Team>> groups = dbContext.Teams
                .Include(t => t.Players)
                .Where(t => t.ID == 3)
                .GroupBy(t => t.Sport)
                .ToList();

I have investigated the produced SQL query. The first SQL query that is produced to join Team on Sport is correct:

SELECT "t"."id", "t"."name", "t"."sport_id"
	FROM "team" AS "t"
	INNER JOIN "sport" AS "t.Sport" ON "t"."sport_id" = "t.Sport"."id"
	WHERE "t"."id" = 3
	ORDER BY "t.Sport"."id", "t"."id"

The second query produced to join Team on Players has an incorrect join:

SELECT "t.Players"."id", "t.Players"."team_id"
	FROM "team" AS "t.Players"
	INNER JOIN (
	    SELECT DISTINCT "t.Sport0"."id", "t0"."id" AS "id0"
	    FROM "team" AS "t0"
	    INNER JOIN "sport" AS "t.Sport0" ON "t0"."sport_id" = "t.Sport0"."id"
	    WHERE "t0"."id" = 3
	) AS "t1" ON "t.Players"."team_id" = "t1"."id"
	ORDER BY "t0"."id"

EF Core makes its outermost join on Player and Sport (t1.id), which does not make any sense and obviously gives no result. Instead, it should join on Player and Team (id0). Running the query manually with the fixed SQL returns the expected record.

Also removing the GroupBy() clause fixes the problem. The following query produces the expected SQL query and returns the expected results:

IEnumerable<Team> groups = dbContext.Teams
                .Include(t => t.Players)
                .Include(t => t.Sport)
                .Where(t => t.ID == 3)
                .ToList();

I can then GroupBy() just fine using LINQ-to-objects,

@olivierr91 olivierr91 changed the title Generated SQL joining on the wrong field when using GroupBy EF Core joining on the wrong field when using GroupBy Aug 19, 2017
@ajcvickers ajcvickers added this to the 2.1.0 milestone Aug 23, 2017
@ajcvickers
Copy link
Contributor

Hi @orobert91. We are gathering information on the use of EF Core pre-release builds. You reported this issue shortly after the release of 2.0.0 RTM. It would be really helpful if you could let us know:

  • Did you consider testing your code against the pre-release builds?
  • Was there anything blocking you from using pre-release builds?
  • What do you think could make it easier for you to use pre-release builds in the future?

Thanks in advance for any feedback. Hopefully this will help us to increase the value of pre-release builds going forward.

@olivierr91
Copy link
Author

olivierr91 commented Sep 12, 2017

Hi @ajcvickers

  • I did not consider, but I will consider. I understand that testing our code against pre-release builds is helpful to you guys and prevent such issues from appearing in the final release. Since we have made .NET Core and EF Core our primary development technology, it would make sense to do it.
  • No.
  • Getting the pre-release builds through NuGet couldn't be easier, really. Maybe if there was a way to get notified when a milestone is reached and you would like us to test the code, that would be great

@ajcvickers
Copy link
Contributor

@orobert91 Thanks for the feedback; much appreciated!

@ajcvickers ajcvickers modified the milestones: 2.1.0-preview1, 2.1.0 Jan 17, 2018
@smitpatel
Copy link
Contributor

Blocked on #10974

@smitpatel smitpatel removed the blocked label Mar 14, 2018
@smitpatel smitpatel assigned maumar and unassigned smitpatel Mar 14, 2018
@smitpatel
Copy link
Contributor

Assigning to @maumar to verify this is fixed while fixing #10974
If there is still invalid sql, feel free to pass me back.

maumar added a commit that referenced this issue Mar 15, 2018
#11288 - Query: Join/GroupJoin where key selectors are qsres or navigations fails during compilation
#10974 - Query: Include Collection group by reference throws exception
#11245 - Query: compilation error for queries with join where the inner key is navigation
#9481 - EF Core joining on the wrong field when using GroupBy

- Added step to the QueryOptimizer that converts entity qsre comparison to key comparisons instead for order by join and group join,
- Improved logic of join/groupjoin inner key navigation rewrite to correctly handle case when navigation is the final operator (i.e. no scalar is projected at the end),
- Improved "requires materialization" logic to correctly mark qsre projected out of subquery that is a result of nav rewrite of join/groupjoin inner key.
maumar added a commit that referenced this issue Mar 16, 2018
#11288 - Query: Join/GroupJoin where key selectors are qsres or navigations fails during compilation
#10974 - Query: Include Collection group by reference throws exception
#11245 - Query: compilation error for queries with join where the inner key is navigation
#11022 - Query: GroupJoin with a complex join key fails during compilation
#9481 - EF Core joining on the wrong field when using GroupBy

- Added step to the QueryOptimizer that converts entity qsre comparison to key comparisons instead for order by join and group join,
- Improved logic of join/groupjoin inner key navigation rewrite to correctly handle case when navigation is the final operator (i.e. no scalar is projected at the end),
- Improved "requires materialization" logic to correctly mark for materialization query sources coming from subquery join keys.
maumar added a commit that referenced this issue Mar 19, 2018
#11288 - Query: Join/GroupJoin where key selectors are qsres or navigations fails during compilation
#10974 - Query: Include Collection group by reference throws exception
#11245 - Query: compilation error for queries with join where the inner key is navigation
#11022 - Query: GroupJoin with a complex join key fails during compilation
#9481 - EF Core joining on the wrong field when using GroupBy

- Added step to the QueryOptimizer that converts entity qsre comparison to key comparisons instead for order by join and group join,
- Improved logic of join/groupjoin inner key navigation rewrite to correctly handle case when navigation is the final operator (i.e. no scalar is projected at the end),
- Improved "requires materialization" logic to correctly mark for materialization query sources coming from subquery join keys.
maumar added a commit that referenced this issue Mar 20, 2018
#11288 - Query: Join/GroupJoin where key selectors are qsres or navigations fails during compilation
#10974 - Query: Include Collection group by reference throws exception
#11245 - Query: compilation error for queries with join where the inner key is navigation
#11022 - Query: GroupJoin with a complex join key fails during compilation
#9481 - EF Core joining on the wrong field when using GroupBy

- Added step to the QueryOptimizer that converts entity qsre comparison to key comparisons instead for order by join and group join,
- Improved logic of join/groupjoin inner key navigation rewrite to correctly handle case when navigation is the final operator (i.e. no scalar is projected at the end),
- Improved "requires materialization" logic to correctly mark for materialization query sources coming from subquery join keys.
maumar added a commit that referenced this issue Mar 20, 2018
#11288 - Query: Join/GroupJoin where key selectors are qsres or navigations fails during compilation
#10974 - Query: Include Collection group by reference throws exception
#11245 - Query: compilation error for queries with join where the inner key is navigation
#11022 - Query: GroupJoin with a complex join key fails during compilation
#9481 - EF Core joining on the wrong field when using GroupBy

- Added step to the QueryOptimizer that converts entity qsre comparison to key comparisons instead for order by join and group join,
- Improved logic of join/groupjoin inner key navigation rewrite to correctly handle case when navigation is the final operator (i.e. no scalar is projected at the end),
- Improved "requires materialization" logic to correctly mark for materialization query sources coming from subquery join keys.
@maumar
Copy link
Contributor

maumar commented Mar 20, 2018

fixed in fdb8ece

@maumar maumar closed this as completed Mar 20, 2018
@maumar maumar added the closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. label Mar 20, 2018
@ajcvickers ajcvickers modified the milestones: 2.1.0-preview2, 2.1.0 Nov 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. type-bug
Projects
None yet
Development

No branches or pull requests

4 participants