New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

support explode outer #2047

Closed

sperlingxx wants to merge 42 commits into NVIDIA:branch-0.5 from sperlingxx:explode_outer

Collaborator

sperlingxx commented Mar 30, 2021

Signed-off-by: sperlingxx [email protected]
Current PR is to support explode outer (with/without position) in spark-rapids plugin.

P.S. This PR is marked as W.I.P. because it won't pass the tests until the cuDF fix take effect.


          support explode with GenerateOuter

6653e3f

Signed-off-by: sperlingxx <[email protected]>

revans2 reviewed

View reviewed changes

Collaborator

revans2 left a comment

Overall it is looking really good.

integration_tests/src/main/python/generate_expr_test.py Outdated Show resolved Hide resolved

sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala Show resolved Hide resolved

sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala Show resolved Hide resolved

razajafri marked this pull request as draft

March 30, 2021 17:24


          fix typo

7c3ee85

revans2 reviewed

View reviewed changes

Collaborator

revans2 left a comment

Looks good. The docs still need to be re-generated and added to the PR.

sameerz added the feature request label


          update doc

0b964f8

Signed-off-by: sperlingxx <[email protected]>

sperlingxx changed the title ~~[WIP] support explode outer~~ support explode outer

sperlingxx marked this pull request as ready for review

April 1, 2021 02:17

Collaborator Author

sperlingxx commented Apr 1, 2021

build

Member

jlowe commented Apr 1, 2021

Tests are failing, possibly because of rapidsai/cudf#7787. If that's the case, those tests should be xfailed with a pointer to that cudf issue in the xfail description.

pxLi and others added 16 commits

April 15, 2021 11:42


          init branch-0.6 [skip ci] (NVIDIA#2124)

c5cba83

* init branch-0.6

Signed-off-by: Peixin Li <[email protected]>


          Merge pull request NVIDIA#2136 from NVIDIA/branch-0.5

6f792dd

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge pull request NVIDIA#2137 from NVIDIA/branch-0.5

461dd01

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge pull request NVIDIA#2138 from NVIDIA/branch-0.5

2173c07

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          support truncate cuda version for image build [skip ci] (NVIDIA#2139)

34d8c89

* update integration images to support cuda update version

Signed-off-by: Peixin Li <[email protected]>


          Merge pull request NVIDIA#2141 from NVIDIA/branch-0.5

de2573c

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge pull request NVIDIA#2143 from NVIDIA/branch-0.5

b3a0874

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge pull request NVIDIA#2147 from NVIDIA/branch-0.5

7c9db82

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge pull request NVIDIA#2148 from NVIDIA/branch-0.5

426f229

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge pull request NVIDIA#2150 from NVIDIA/branch-0.5

75cfd17

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Restore CUDA classifier handling to Jenkins scripts (NVIDIA#2142)

543bf47

Signed-off-by: Jason Lowe <[email protected]>


          Merge pull request NVIDIA#2158 from NVIDIA/branch-0.5

ed3e702

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge pull request NVIDIA#2162 from NVIDIA/branch-0.5

116668d

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge pull request NVIDIA#2164 from NVIDIA/branch-0.5

8849be5

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge pull request NVIDIA#2165 from NVIDIA/branch-0.5

3acbd23

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge pull request NVIDIA#2170 from NVIDIA/branch-0.5

76a55c6

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]

nvauto and others added 20 commits

April 19, 2021 23:17


          Merge pull request NVIDIA#2184 from NVIDIA/branch-0.5

1bb3b8f

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge pull request NVIDIA#2186 from NVIDIA/branch-0.5

954fffa

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge pull request NVIDIA#2187 from NVIDIA/branch-0.5

d398e8b

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge pull request NVIDIA#2189 from NVIDIA/branch-0.5

6a3c281

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Branch 0.5 doc update (NVIDIA#2175)

d3775a1

* Update docs for 0.5 release

Signed-off-by: Sameer Raheja <[email protected]>

* Update download page

Signed-off-by: Sameer Raheja <[email protected]>

* Remove duplicate line

Signed-off-by: Sameer Raheja <[email protected]>

* Update upcoming versions of spark testing readme

Signed-off-by: Sameer Raheja <[email protected]>

* Revert changes in FAQ.md so the log lines match the revision

Signed-off-by: Sameer Raheja <[email protected]>

* Removed "SNAPSHOT" verbiage

Signed-off-by: Sameer Raheja <[email protected]>


          Merge pull request NVIDIA#2194 from pxLi/m-0.5-to-0.6

9150a05

merge from 0.5 to 0.6 [skip ci]


          fix merge conflict for 0.5 doc

dbc67cd


          Merge pull request NVIDIA#2197 from pxLi/fix-merge-conflict-from-0.5

72e7814

fix merge conflict from 0.5 [skip ci]


          fix merge conflict for udf doc from 0.5

9831c1e


          Merge pull request NVIDIA#2199 from pxLi/fix-merge-conflict-2198

e6fe915

fix merge conflict for udf doc from 0.5 [skip ci]


          Init scripts to install cuda11 runtime [skip ci] (NVIDIA#2185)

08a5bc1

* Init scripts to install cuda11 runtime

Add the parameter 'init_script' to make init scripts common for Databricks.

Add the 'init_cuda11_runtime.sh' to install cuda11 toolkit on Databricks.

Signed-off-by: Tim Liu <[email protected]>

* init scripts user is root


          Merge pull request NVIDIA#2201 from NVIDIA/branch-0.5

931e6c5

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge pull request NVIDIA#2202 from NVIDIA/branch-0.5

31dc4f6

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge pull request NVIDIA#2205 from NVIDIA/branch-0.5

a974994

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Use CPM to fetch libcudf dependency for native UDF example build (NVI…

c2eaa9e

…DIA#2191)

Signed-off-by: Jason Lowe <[email protected]>


          Merge pull request NVIDIA#2211 from NVIDIA/branch-0.5

575f645

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge pull request NVIDIA#2212 from NVIDIA/branch-0.5

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge pull request NVIDIA#2213 from NVIDIA/branch-0.5

19c43d8

[auto-merge] branch-0.5 to branch-0.6 [skip ci] [bot]


          Merge remote-tracking branch 'origin/branch-0.6' into explode_outer

5cc049b


          sign

f389cbd

Signed-off-by: sperlingxx <[email protected]>

sperlingxx requested review from GaryShen2008, jlowe, NvTimLiu and tgravescs as code owners

April 21, 2021 04:04


          append sign

dbb3a64

Signed-off-by: sperlingxx <[email protected]>

Collaborator

revans2 commented Apr 21, 2021

@sperlingxx this code here was upmerged to 0.6, but this is targeted to 0.5. Also there is a separate new PR for 0.6 #2215 for this same branch. Should we close this one?

Collaborator Author

sperlingxx commented Apr 22, 2021

@sperlingxx this code here was upmerged to 0.6, but this is targeted to 0.5. Also there is a separate new PR for 0.6 #2215 for this same branch. Should we close this one?

Yes, this PR should be closed. I am sorry that I forgot to close it.

sperlingxx closed this

sperlingxx deleted the explode_outer branch

April 22, 2021 01:50

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

revans2 revans2 left review comments

GaryShen2008 Awaiting requested review from GaryShen2008 GaryShen2008 is a code owner

jlowe Awaiting requested review from jlowe jlowe is a code owner

NvTimLiu Awaiting requested review from NvTimLiu NvTimLiu is a code owner

tgravescs Awaiting requested review from tgravescs tgravescs is a code owner

Labels

feature request