[auto-merge] branch-0.4 to branch-0.5 [skip ci] [bot] #1825
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Fix LEAD/LAG failures in Spark 3.1.1 Fixes class-loader related failures of LEAD/LAG operators, which caused errors like: ``` : java.lang.VerifyError: Bad type on operand stack Exception Details: Location: com/nvidia/spark/rapids/OffsetWindowFunctionMeta.<init>(Lorg/apache/spark/sql/catalyst/expressions/OffsetWindowFunction;Lcom/nvidia/spark/rapids/RapidsConf;Lscala/Option;Lcom/nvidia/spark/rapids/ConfKeysAndIncompat;)V @11: invokespecial Reason: Type 'org/apache/spark/sql/catalyst/expressions/OffsetWindowFunction' (current frame, stack[1]) is not assignable to 'org/apache/spark/sql/catalyst/expressions/Expression' ``` Also corrects for the reversal in offset semantics for LAG() expressions in Spark 3.1.1, causing `Lag.offset` to be negative. Signed-off-by: Mithun RK <[email protected]> * Removed dead code. Signed-off-by: Mithun RK <[email protected]> * More specific error handling extracting literal offsets. * Offset checking for pre-3.1 Spark lead/lag window functions. Signed-off-by: Mithun RK <[email protected]> * Missing whitespace. * Added tagExprForGpu() for OffsetWindowFunctionMeta.
|
SUCCESS - auto-merge |
nartal1
pushed a commit
to nartal1/spark-rapids
that referenced
this pull request
Jun 9, 2021
[auto-merge] branch-0.4 to branch-0.5 [skip ci] [bot]
nartal1
pushed a commit
to nartal1/spark-rapids
that referenced
this pull request
Jun 9, 2021
[auto-merge] branch-0.4 to branch-0.5 [skip ci] [bot]
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
auto-merge triggered by github actions on
branch-0.4to create a PR keepingbranch-0.5up-to-date. If this PR is unable to be merged due to conflicts, it will remain open until manually fix.