Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use nvCOMP for Snappy compression/decompression #9582

Merged
merged 7 commits into from
Nov 8, 2021
Merged

Use nvCOMP for Snappy compression/decompression #9582

merged 7 commits into from
Nov 8, 2021

Conversation

vuule
Copy link
Contributor

@vuule vuule commented Nov 2, 2021

  • Use nvCOMP for Snappy compression/decompression by default.
  • Change the environment variable naming to be consistent with GDS policy.
  • Add detailed description of the behavior to the docs.

@vuule vuule added cuIO cuIO issue improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Nov 2, 2021
@vuule vuule self-assigned this Nov 2, 2021
@github-actions github-actions bot added the libcudf Affects libcudf (C++/CUDA) code. label Nov 2, 2021
@codecov
Copy link

codecov bot commented Nov 2, 2021 •
edited
Loading

Codecov Report

Merging #9582 (03bc828) into branch-21.12 (ab4bfaa) will decrease coverage by 0.51%.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff                @@
##           branch-21.12    #9582      +/-   ##
================================================
- Coverage         10.79%   10.27%   -0.52%     
================================================
  Files               116      117       +1     
  Lines             18869    20589    +1720     
================================================
+ Hits               2036     2115      +79     
- Misses            16833    18474    +1641
Impacted Files Coverage Δ
python/dask_cudf/dask_cudf/sorting.py 92.90% <0.00%> (-1.21%) ⬇️
python/cudf/cudf/io/csv.py 0.00% <0.00%> (ø)
python/cudf/cudf/io/hdf.py 0.00% <0.00%> (ø)
python/cudf/cudf/io/orc.py 0.00% <0.00%> (ø)
python/cudf/cudf/__init__.py 0.00% <0.00%> (ø)
python/cudf/cudf/_version.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/abc.py 0.00% <0.00%> (ø)
python/cudf/cudf/api/types.py 0.00% <0.00%> (ø)
python/cudf/cudf/io/dlpack.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/frame.py 0.00% <0.00%> (ø)
... and 67 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4cba672...03bc828. Read the comment docs.

@vuule vuule marked this pull request as ready for review November 4, 2021 23:51
@vuule vuule requested a review from a team as a code owner November 4, 2021 23:51
@vuule
Copy link
Contributor Author

vuule commented Nov 4, 2021

rerun tests

elstehle
elstehle reviewed Nov 5, 2021
View reviewed changes
Copy link
Contributor

@elstehle elstehle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good. I just think there's a mismatch between the documented values and the values actually used in the implementation.

cpp/src/io/utilities/config_utils.hpp Outdated Show resolved Hide resolved
cpp/src/io/utilities/config_utils.hpp Outdated Show resolved Hide resolved
rgsl888prabhu
rgsl888prabhu reviewed Nov 5, 2021
View reviewed changes
Copy link
Contributor

@rgsl888prabhu rgsl888prabhu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just need to answer above queries.

@vuule vuule added the 5 - Ready to Merge Testing and reviews complete, ready to merge label Nov 5, 2021
@vuule
Copy link
Contributor Author

vuule commented Nov 8, 2021

rerun tests

@vuule
Copy link
Contributor Author

vuule commented Nov 8, 2021

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 7666d4a into rapidsai:branch-21.12 Nov 8, 2021
@vuule vuule deleted the fea-nvcomp-default-use branch November 8, 2021 21:53
This was referenced Dec 4, 2021
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JohnZed JohnZed JohnZed left review comments

@rgsl888prabhu rgsl888prabhu rgsl888prabhu approved these changes

@elstehle elstehle elstehle approved these changes

@devavret devavret Awaiting requested review from devavret devavret is a code owner automatically assigned from rapidsai/cudf-cpp-codeowners

Assignees

@vuule vuule

Labels
5 - Ready to Merge Testing and reviews complete, ready to merge cuIO cuIO issue improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@vuule @JohnZed @elstehle @rgsl888prabhu