Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow runtime has_nulls parameter for row operators #9623

Merged
merged 28 commits into from
Dec 6, 2021

Conversation

davidwendt
Copy link
Contributor

@davidwendt davidwendt commented Nov 8, 2021

Closes #6952

This PR allows the has_nulls template parameter for row operators to be used a runtime parameter in places where the null-handling logic has little to no affect on runtime performance.
This can improve compile time as described in #6952.

This will also close #9152 and #9580

@davidwendt davidwendt added 2 - In Progress Currently a work in progress libcudf Affects libcudf (C++/CUDA) code. improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Nov 8, 2021
@davidwendt davidwendt self-assigned this Nov 8, 2021
@davidwendt
Copy link
Contributor Author

Current code reference of template approach for developer (me): https://godbolt.org/z/TbKqq6Tj6

@codecov
Copy link

codecov bot commented Nov 18, 2021

Codecov Report

Merging #9623 (c4f6a08) into branch-22.02 (967a333) will decrease coverage by 0.04%.
The diff coverage is 5.79%.

Impacted file tree graph

@@               Coverage Diff                @@
##           branch-22.02    #9623      +/-   ##
================================================
- Coverage         10.49%   10.44%   -0.05%     
================================================
  Files               119      119              
  Lines             20305    20422     +117     
================================================
+ Hits               2130     2133       +3     
- Misses            18175    18289     +114     
Impacted Files Coverage Δ
python/cudf/cudf/__init__.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/_base_index.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/column/column.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/column/string.py 0.00% <ø> (ø)
python/cudf/cudf/core/frame.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/groupby/groupby.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/index.py 0.00% <ø> (ø)
python/cudf/cudf/core/indexed_frame.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/multiindex.py 0.00% <0.00%> (ø)
python/cudf/cudf/io/json.py 0.00% <ø> (ø)
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7ac8aac...c4f6a08. Read the comment docs.

@davidwendt davidwendt added 3 - Ready for Review Ready for review by team and removed 2 - In Progress Currently a work in progress labels Nov 22, 2021
@davidwendt davidwendt marked this pull request as ready for review November 22, 2021 16:49
@davidwendt davidwendt requested a review from a team as a code owner November 22, 2021 16:49
@davidwendt davidwendt requested a review from jrhemstad November 30, 2021 15:44
@davidwendt
Copy link
Contributor Author

davidwendt commented Dec 1, 2021

Most of the changes in this PR are in headers and the .cu files were merely updated to reflect the new interfaces. There are a few files here where the new runtime (dynamic) check was employed. Here are a list of build time improvements for those files and a few others:

base (ms) new (ms) file
649811 622113 groupby/sort/group_argmax.cu.o
633835 609839 groupby/sort/group_argmin.cu.o
581884 571184 reductions/scan/scan_inclusive.cu.o
529868 518713 search/search.cu.o
507915 368781 groupby/sort/sort_helper.cu.o
499699 324520 groupby/hash/groupby.cu.o
497747 490491 sort/sort_column.cu.o
434132 387221 sort/sort.cu.o
430886 335445 sort/is_sorted.cu.o
406767 327745 stream_compaction/distinct_count.cu.o
392761 388912 sort/stable_sort_column.cu.o
386460 313861 sort/rank.cu.o
300859 199169 stream_compaction/drop_duplicates.cu.o
92297 77487 reductions/scan/rank_scan.cu.o
92059 81777 groupby/sort/group_rank_scan.cu.o
85447 63088 hash/murmur_hash.cu.o
71180 63444 hash/hashing.cu.o
67114 51629 transform/one_hot_encode.cu.o

The libcudf.so file was also reduced by 10MB. My local build is about 200MB right now.
The overall build time improvement was only about 20s on my machine. My local build takes about 15 minutes.

@davidwendt davidwendt added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 3 - Ready for Review Ready for review by team labels Dec 3, 2021
@davidwendt
Copy link
Contributor Author

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 8002cbd into rapidsai:branch-22.02 Dec 6, 2021
@davidwendt davidwendt deleted the fea-row-op-has-nulls branch December 6, 2021 13:38
rapids-bot bot pushed a commit that referenced this pull request Dec 17, 2021
Follow on PR for this comment: #9623 (comment)

The join hasher and equality-comparator were previously hardcoded with `has_nulls=true` (and migrated to `nullate::YES`) to help minimize code size. The new `nullate::DYNAMIC` allows for runtime checking of nulls so this can now be used here instead.

Authors:
  - David Wendt (https://github.com/davidwendt)

Approvers:
  - Jake Hemstad (https://github.com/jrhemstad)
  - Vukasin Milovanovic (https://github.com/vuule)

URL: #9902
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Ready to Merge Testing and reviews complete, ready to merge improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
None yet
6 participants