Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace raw streams with rmm::cuda_stream_view (part 2) #6648

Merged
merged 71 commits into from
Nov 20, 2020

Conversation

harrism
Copy link
Member

@harrism harrism commented Nov 3, 2020

Converting libcudf to use rmm::cuda_stream_view will require a LOT of changes, so I'm splitting it into multiple PRs to ease reviewing. This is the second PR in the series. This series of PRs will

Contributes to #6645 and #5119

Depends on #6646 so this PR will look much bigger until that one is merged.

Also fixes #6706 (to_arrow and to_dlpack are not properly synchronized).

This second PR converts:

  • table.hpp (and source / dependencies)
  • column_factories.hpp (and source / dependencies)
  • headers in include/detail/aggregation (and source / dependencies)
  • include/detail/groupby/sort_helper.hpp (and source / dependencies)
  • include/detail/utilities/cuda.cuh (and dependencies)
  • binary ops
  • concatenate
  • copy_if
  • copy_range
  • fill
  • gather
  • get_value
  • hash groupby
  • quantiles
  • reductions
  • repeat
  • replace
  • reshape
  • round
  • scatter
  • search
  • sequence
  • sorting
  • stream compaction

@harrism harrism added 2 - In Progress Currently a work in progress libcudf Affects libcudf (C++/CUDA) code. RMM labels Nov 3, 2020
@harrism harrism self-assigned this Nov 3, 2020
@GPUtester
Copy link
Collaborator

Please update the changelog in order to start CI tests.

View the gpuCI docs here.

@codecov
Copy link

codecov bot commented Nov 3, 2020

Codecov Report

Merging #6648 (2390e61) into branch-0.17 (99cee1c) will increase coverage by 0.05%.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff               @@
##           branch-0.17    #6648      +/-   ##
===============================================
+ Coverage        81.76%   81.82%   +0.05%     
===============================================
  Files               96       96              
  Lines            16016    16016              
===============================================
+ Hits             13096    13105       +9     
+ Misses            2920     2911       -9     
Impacted Files Coverage Δ
python/cudf/cudf/io/orc.py 89.39% <0.00%> (+6.81%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 99cee1c...2390e61. Read the comment docs.

Copy link
Collaborator

@kkraus14 kkraus14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cmake / conda approval

@harrism
Copy link
Member Author

harrism commented Nov 13, 2020

A little hard to do the review with 1000+ lines of auto-reformatting on the java side, but I got through it and I guess it is better to pull off the bandaid.

So sorry I'm so used to clang-format now I didn't even notice that... Maybe we should turn on clang-format for the java directory in @codereport 's clang-format upgrade PR?

cpp/include/cudf/detail/concatenate.cuh Show resolved Hide resolved
cpp/include/cudf/detail/gather.cuh Outdated Show resolved Hide resolved
cpp/include/cudf/detail/groupby.hpp Outdated Show resolved Hide resolved
cpp/include/cudf/detail/quantiles.hpp Outdated Show resolved Hide resolved
cpp/include/cudf/detail/reduction.cuh Outdated Show resolved Hide resolved
cpp/include/cudf/lists/detail/gather.cuh Outdated Show resolved Hide resolved
cpp/src/copying/scatter.cu Outdated Show resolved Hide resolved
cpp/src/dictionary/set_keys.cu Outdated Show resolved Hide resolved
Copy link
Contributor

@cwharris cwharris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments, but otherwise LGTM! 🔥

cpp/src/dictionary/detail/concatenate.cu Show resolved Hide resolved
cpp/src/groupby/groupby.cu Outdated Show resolved Hide resolved
cpp/src/copying/scatter.cu Outdated Show resolved Hide resolved
cpp/src/interop/dlpack.cpp Outdated Show resolved Hide resolved
cpp/src/join/semi_join.cu Outdated Show resolved Hide resolved
cpp/src/stream_compaction/drop_nulls.cu Outdated Show resolved Hide resolved
cpp/src/transform/transform.cpp Outdated Show resolved Hide resolved
cpp/src/unary/nan_ops.cu Outdated Show resolved Hide resolved
cpp/src/unary/unary_ops.cuh Outdated Show resolved Hide resolved
java/src/main/native/src/map_lookup.cu Outdated Show resolved Hide resolved
@harrism
Copy link
Member Author

harrism commented Nov 18, 2020

OK, I have now addressed all review comments @devavret @cwharris . Thanks for catching lots of easily missed things!

@harrism harrism added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 2 - In Progress Currently a work in progress 3 - Ready for Review Ready for review by team labels Nov 19, 2020
@harrism harrism merged commit 7e51022 into rapidsai:branch-0.17 Nov 20, 2020
@jlowe jlowe mentioned this pull request Nov 20, 2020
jlowe added a commit that referenced this pull request Nov 20, 2020
This fixes the JNI build after #6648. The JNI data_sink was updated to use RMM stream views before the data_sink API was updated.
harrism added a commit to rapidsai/cuspatial that referenced this pull request Nov 24, 2020
This PR converts all usage of cudaStream_t in cuSpatial to rmm::cuda_stream_view, following on from rapidsai/cudf#6646 and rapidsai/cudf#6648

Also reorders stream parameters to occur before MR parameters in all functions.
kkraus14 pushed a commit that referenced this pull request Nov 24, 2020
Converting libcudf to use `rmm::cuda_stream_view` will require a LOT of changes, so I'm splitting it into multiple PRs to ease reviewing. This is the third PR in the series. This series of PRs will

- Replace usage of `cudaStream_t` with `rmm::cuda_stream_view`
- Replace usage of `0` or `nullptr` as a stream identifier with `rmm::cuda_stream_default`
- Ensure all APIs always order the stream parameter before the memory resource parameter. #5119

Contributes to #6645 and #5119

Depends on #6646 and #6648 so this PR will look much bigger until they are merged.

This third PR converts:

 - remaining dictionary functionality
 - cuio
 - lists
 - scalar
 - strings
 - groupby
 - join
 - contiguous_split
 - get_element
 - datetime_ops
 - extract
 - merge
 - partitioning
 - minmax reduction
 - scan
 - byte_cast
 - clamp
 - interleave_columns
 - is_sorted
 - groupby
 - rank
 - tests
 - concurrent map classes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Ready to Merge Testing and reviews complete, ready to merge libcudf Affects libcudf (C++/CUDA) code.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] cudf::to_arrow is not properly synchronized
7 participants