Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Combine linearizer and ast_plan #8900

Merged
merged 15 commits into from
Aug 2, 2021

Conversation

vyasr
Copy link
Contributor

@vyasr vyasr commented Jul 29, 2021

This PR combines the ast_plan and linearizer classes into the new expression_parser class. Previously the linearizer would parse the expression into some host-side buffers, and the ast_plan simply wrapped a linearizer instance and transferred its contents into an owned device buffer so that expressions could be evaluated in device code. This required either adding lots of accessor methods to otherwise internal components of the linearizer or adding friend relationships. Since we have no use for parsing expressions without making them evaluable on device, this PR combines the two classes to simplify that process.

There's one additional minor change which is the move of the single_dispatch_binary_operator functor from operators.hpp to transform.cuh. It's only being used in one place, so this change reduces the need for includes and makes the code more readable.

Contributes to #8145.

@vyasr vyasr added 3 - Ready for Review Ready for review by team code quality libcudf Affects libcudf (C++/CUDA) code. improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Jul 29, 2021
@vyasr vyasr added this to the Conditional Joins milestone Jul 29, 2021
@vyasr vyasr self-assigned this Jul 29, 2021
@vyasr vyasr requested a review from a team as a code owner July 29, 2021 16:31
@vyasr vyasr requested a review from a team as a code owner July 29, 2021 16:31
@vyasr vyasr requested review from trxcllnt and vuule July 29, 2021 16:31
@github-actions github-actions bot added the CMake CMake build issue label Jul 29, 2021
@github-actions github-actions bot added the conda label Jul 29, 2021
@vyasr vyasr requested a review from a team as a code owner July 29, 2021 17:05
@codecov
Copy link

codecov bot commented Jul 29, 2021

Codecov Report

Merging #8900 (bf3a1e7) into branch-21.10 (18f7c01) will decrease coverage by 0.08%.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff                @@
##           branch-21.10    #8900      +/-   ##
================================================
- Coverage         10.67%   10.59%   -0.09%     
================================================
  Files               110      116       +6     
  Lines             18271    19043     +772     
================================================
+ Hits               1951     2017      +66     
- Misses            16320    17026     +706     
Impacted Files Coverage Δ
python/cudf/cudf/__init__.py 0.00% <ø> (ø)
python/cudf/cudf/core/__init__.py 0.00% <ø> (ø)
python/cudf/cudf/core/column/categorical.py 0.00% <ø> (ø)
python/cudf/cudf/core/column/column.py 0.00% <ø> (ø)
python/cudf/cudf/core/column/lists.py 0.00% <ø> (ø)
python/cudf/cudf/core/column/numerical.py 0.00% <ø> (ø)
python/cudf/cudf/core/column/string.py 0.00% <ø> (ø)
python/cudf/cudf/core/column/struct.py 0.00% <ø> (ø)
python/cudf/cudf/core/dataframe.py 0.00% <ø> (ø)
python/cudf/cudf/core/frame.py 0.00% <ø> (ø)
... and 73 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7032f47...bf3a1e7. Read the comment docs.

Copy link
Contributor

@vuule vuule left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a potential perf impact?

cpp/include/cudf/ast/nodes.hpp Outdated Show resolved Hide resolved
@vyasr
Copy link
Contributor Author

vyasr commented Jul 29, 2021

Is there a potential perf impact?

Nope this change is performance-neutral. It simply converts what was previously an implicit relationship (the linearizer and ast_plan had to be created together, and the sole purpose of the ast_plan was to perform a H2D transfer of the linearizer's internal data) into an explicit one that also reduces some boilerplate.

@vyasr
Copy link
Contributor Author

vyasr commented Jul 29, 2021

rerun tests

@vyasr vyasr requested a review from ttnghia July 30, 2021 05:22
@vyasr
Copy link
Contributor Author

vyasr commented Jul 30, 2021

CC @jlowe you mentioned that you're pulling in the detail::node struct for the JNI. I looked at java/src/main/native/src/CompiledExpression.cpp and it looks like it's being implicitly included via public headers, so I don't think this rename will cause any problems for your implementation, but wanted to ping you just in case there's another file referencing these internals that I missed.

Copy link
Member

@jlowe jlowe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this rename will cause any problems for your implementation

Yep, it should be ok. The good news is that the Java build is now part of the CI, and you can see in the checks whether it fails or not.

cpp/include/cudf/ast/nodes.hpp Show resolved Hide resolved
Copy link
Contributor

@ttnghia ttnghia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still need to convert to east const convention in some places. I didn't point out all those, so please read and check the entire code. Otherwise, it is good.

cpp/include/cudf/ast/detail/expression_parser.hpp Outdated Show resolved Hide resolved
cpp/include/cudf/ast/nodes.hpp Outdated Show resolved Hide resolved
cpp/src/ast/expression_parser.cpp Outdated Show resolved Hide resolved
cpp/src/join/conditional_join_kernels.cuh Outdated Show resolved Hide resolved
Copy link
Contributor

@robertmaynard robertmaynard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CMake changes LGTM

@vyasr
Copy link
Contributor Author

vyasr commented Aug 2, 2021

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 8c9d2be into rapidsai:branch-21.10 Aug 2, 2021
@vyasr vyasr mentioned this pull request Aug 4, 2021
10 tasks
@vyasr vyasr deleted the refactor/ast_linearizer branch January 14, 2022 18:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review Ready for review by team CMake CMake build issue improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants