Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Fix Spark 3.1 GpuHashJoin since it now requires CodegenSupport #489

Closed
tgravescs opened this issue Jul 31, 2020 · 0 comments · Fixed by #501
Closed

[FEA] Fix Spark 3.1 GpuHashJoin since it now requires CodegenSupport #489

tgravescs opened this issue Jul 31, 2020 · 0 comments · Fixed by #501
Labels
bug Something isn't working

Comments

@tgravescs
Copy link
Collaborator

Describe the bug
apache/spark@ae82768
went into Spark 3.1 to add codegen support to ShuffledHashJoin. This commonized some code which added the CodegenSupport trait to the base HashJoin class. We directly inherit from that class in GpuHashJoin so its expecting us to do codegen now.

note that I made the 3.1.0 build work by adding in the missing functions inputRDDs and prepareRelation but this was just to get it to compile again so it wasn't blocking the build of other versions.

Steps/Code to reproduce bug
Please provide a list of steps or a code sample to reproduce the issue.
Avoid posting private or sensitive data.

Run unit tests for Spark 3.1

Expected behavior
A clear and concise description of what you expected to happen.

Tests shouldn't fail or try to run codegen in GPU version.

Environment details (please complete the following information)

  • Environment location: [Standalone, YARN, Kubernetes, Cloud(specify cloud provider)]
  • Spark configuration settings related to the issue

Additional context
Add any other context about the problem here.

@tgravescs tgravescs added bug Something isn't working ? - Needs Triage Need team to review and classify labels Jul 31, 2020
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label Aug 13, 2020
tgravescs pushed a commit to tgravescs/spark-rapids that referenced this issue Nov 30, 2023
Signed-off-by: spark-rapids automation <[email protected]>

Signed-off-by: spark-rapids automation <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants