Add minhash support for MurmurHash3_x64_128 #13796
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
davidwendt
added
feature request
davidwendt
changed the title
davidwendt
added
3 - Ready for Review
davidwendt
added
breaking
bdice
reviewed
Aug 10, 2023
bdice
reviewed
Aug 11, 2023
davidwendt
changed the title
davidwendt
changed the title
davidwendt
changed the title
bdice
reviewed
Aug 15, 2023
bdice
approved these changes
Aug 16, 2023
davidwendt
changed the title
vyasr
approved these changes
Aug 18, 2023
| cudf::size_type width = 4, | ||
| cudf::hash_id hash_function = cudf::hash_id::HASH_MURMUR3, | ||
| rmm::mr::device_memory_resource* mr = rmm::mr::get_current_device_resource()); | ||
| cudf::numeric_scalar<uint32_t> seed = 0, |
There was a problem hiding this comment.
Why switch to hardcoding the value here instead of using the constant?
There was a problem hiding this comment.
Because the type may be different. I'd rather be clear that the default seed is actually 0 and would not want to change that if the rest of libcudf decided on a different default. Hopefully that is ok.
| */ | ||
| std::unique_ptr<cudf::column> minhash64( | ||
| cudf::strings_column_view const& input, | ||
| cudf::numeric_scalar<uint64_t> seed = 0, |
There was a problem hiding this comment.
Same question as before, why the different seed choice? Is it because of the potential for unsafe casts depending on the type of DEFAULT_HASH_SEED (currently OK since it's uint32_t)?
There was a problem hiding this comment.
Yes, the type is different here and I'd rather the 2 functions be consistent over any need to use the constant def.
|
/merge |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Adds
nvtext::minhash64to libcudf and the Cython/Python changes to call it.The
MurmurHash3_x64_128is called and only the firstuint64value is used.The libcudf API was changed to remove the
hash_idparameter since it was incompatible with the seed types.Checklist