Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use offsetalator in nvtext::tokenize_with_vocabulary #15878

Merged
merged 1 commit into from
May 30, 2024
Merged

Use offsetalator in nvtext::tokenize_with_vocabulary #15878

merged 1 commit into from
May 30, 2024

Conversation

davidwendt
Copy link
Contributor

Description

Updates the token_counts_fn kernel in the nvtext::tokenize_with_vocabulary to use the offsetalator instead of hardcoded size_type for accessing strings offsets.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@davidwendt davidwendt added 3 - Ready for Review Ready for review by team libcudf Affects libcudf (C++/CUDA) code. strings strings issues (C++ and Python) improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels May 29, 2024
@davidwendt davidwendt self-assigned this May 29, 2024
@davidwendt davidwendt requested a review from a team as a code owner May 29, 2024 15:40
@davidwendt davidwendt requested review from karthikeyann and vuule May 29, 2024 15:40
@davidwendt
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit 789cbfd into rapidsai:branch-24.08 May 30, 2024
70 checks passed
@davidwendt davidwendt deleted the vocab-offsetalator branch May 30, 2024 19:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@karthikeyann karthikeyann karthikeyann approved these changes

@bdice bdice bdice approved these changes

@vuule vuule Awaiting requested review from vuule vuule is a code owner automatically assigned from rapidsai/cudf-cpp-codeowners

Assignees

@davidwendt davidwendt

Labels
3 - Ready for Review Ready for review by team improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change strings strings issues (C++ and Python)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@davidwendt @bdice @karthikeyann