Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Expose stream parameter in public nvtext tokenize APIs #14317

Merged
merged 4 commits into from
Oct 27, 2023
Merged

Expose stream parameter in public nvtext tokenize APIs #14317

merged 4 commits into from
Oct 27, 2023

Conversation

davidwendt
Copy link
Contributor

@davidwendt davidwendt commented Oct 23, 2023 •
edited
Loading

Description

Add stream parameter to public APIs:

  • nvtext::tokenize() (x2)
  • nvtext::count_tokens() (x2)
  • nvtext::character_tokenize()
  • nvtext::detokenize()

Also cleaned up some of the doxygen comments and added stream gtests.

Reference #13744

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@davidwendt davidwendt added 2 - In Progress Currently a work in progress libcudf Affects libcudf (C++/CUDA) code. strings strings issues (C++ and Python) improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Oct 23, 2023
@davidwendt davidwendt self-assigned this Oct 23, 2023
@github-actions github-actions bot added the CMake CMake build issue label Oct 23, 2023
@davidwendt davidwendt added 3 - Ready for Review Ready for review by team and removed 2 - In Progress Currently a work in progress labels Oct 24, 2023
@davidwendt davidwendt marked this pull request as ready for review October 24, 2023 21:59
@davidwendt davidwendt requested a review from a team as a code owner October 24, 2023 21:59
karthikeyann
karthikeyann requested changes Oct 27, 2023
View reviewed changes
cpp/include/nvtext/tokenize.hpp Outdated Show resolved Hide resolved
cpp/include/nvtext/tokenize.hpp Outdated Show resolved Hide resolved
cpp/tests/streams/text/tokenize_test.cpp Show resolved Hide resolved
@davidwendt
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit 83746a4 into rapidsai:branch-23.12 Oct 27, 2023
54 checks passed
@davidwendt davidwendt deleted the stream-nvtext-tokenize branch October 27, 2023 17:23
@davidwendt davidwendt mentioned this pull request Nov 7, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@karthikeyann karthikeyann karthikeyann approved these changes

@bdice bdice bdice approved these changes

@hyperbolic2346 hyperbolic2346 hyperbolic2346 approved these changes

Assignees

@davidwendt davidwendt

Labels
3 - Ready for Review Ready for review by team CMake CMake build issue improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change strings strings issues (C++ and Python)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@davidwendt @hyperbolic2346 @bdice @karthikeyann