Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[gpuCI] Forward-merge branch-0.19 to branch-0.20 [skip ci] #7784

Merged
merged 1 commit into from
Mar 31, 2021

Conversation

GPUtester
Copy link
Collaborator

Forward-merge triggered by push to branch-0.19 that creates a PR to keep branch-0.20 up-to-date. If this PR is unable to be immediately merged due to conflicts, it will remain open for the team to manually merge.

An implementation of get_json_object(). 

Reference:  https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-get_json_object

The fundamental functionality here is running a JSONPath query on each row in an input column of json strings. 

JSONPath spec:  https://tools.ietf.org/id/draft-goessner-dispatch-jsonpath-00.html

For review purposes, the key entry point is `parse_json_path()`.  Each thread of the kernel processes 1 row via this function.  The behavior is recursive in nature but we maintain our own context stack to do it in loop fashion. 

`parse_json_path` is just the high level controlling logic, with most of the heavy lifting happening in the `json_state` parser class. Though the "heavy lifting" is pretty much just traditional string parsing code.

The path to optimization here (I'll open a separate cudf issue for this) is
- Change `parse_json_path` to work on a warp basis.  So each row in the column would be processed by one warp.
- Make the `json_state` parser class thread/warp aware (the class would just store its `tid` and operate accordingly).  I think this is reasonably straightforward to do as most of the cuIO decoding kernels behave like this.

Authors:
  - @nvdbaranec
  - Raza Jafri (@razajafri)

Approvers:
  - Ray Douglass (@raydouglass)
  - Jason Lowe (@jlowe)
  - Jake Hemstad (@jrhemstad)
  - David (@davidwendt)

URL: #7286
@GPUtester GPUtester requested review from a team as code owners March 31, 2021 15:00
@GPUtester GPUtester requested review from harrism and vuule March 31, 2021 15:00
@GPUtester GPUtester merged commit d39cd97 into branch-0.20 Mar 31, 2021
@GPUtester
Copy link
Collaborator Author

SUCCESS - forward-merge complete.

@github-actions github-actions bot added CMake CMake build issue conda Java Affects Java cuDF API. libcudf Affects libcudf (C++/CUDA) code. labels Mar 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CMake CMake build issue Java Affects Java cuDF API. libcudf Affects libcudf (C++/CUDA) code.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants