Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] need multibyte_split support stream #14383

Closed
weedge opened this issue Nov 9, 2023 · 2 comments
Closed

[FEA] need multibyte_split support stream #14383

weedge opened this issue Nov 9, 2023 · 2 comments
Labels
0 - Backlog In queue waiting for assignment cuIO cuIO issue feature request New feature or request libcudf Affects libcudf (C++/CUDA) code.

Comments

@weedge
Copy link

weedge commented Nov 9, 2023

Is your feature request related to a problem? Please describe.
want to use cuda multi stream (pool) read a big file.

Describe the solution you'd like
just add a new multibyte_split method with rmm::cuda_stream_view stream param to public use.

head: https://github.com/rapidsai/cudf/blob/branch-23.12/cpp/include/cudf/io/text/multibyte_split.hpp

std::unique_ptr<cudf::column> multibyte_split(
  data_chunk_source const& source,
  std::string const& delimiter,
  parse_options options               = {},
  rmm::cuda_stream_view stream = cudf::get_default_stream(),
  rmm::mr::device_memory_resource* mr = rmm::mr::get_current_device_resource());

src: https://github.com/rapidsai/cudf/blob/branch-23.12/cpp/src/io/text/multibyte_split.cu

std::unique_ptr<cudf::column> multibyte_split(cudf::io::text::data_chunk_source const& source,
                                              std::string const& delimiter,
                                              parse_options options,
                                              rmm::cuda_stream_view stream,
                                              rmm::mr::device_memory_resource* mr)
{
  auto result = detail::multibyte_split(
    source, delimiter, options.byte_range, options.strip_delimiters, stream, mr);

  return result;
}

Describe alternatives you've considered

Additional context

@weedge weedge added Needs Triage Need team to review and classify feature request New feature or request labels Nov 9, 2023
@bdice
Copy link
Contributor

bdice commented Nov 9, 2023

Related: #13744. cc: @vuule @shrshi

@GregoryKimball GregoryKimball added this to the Enable streams milestone Dec 14, 2023
@GregoryKimball GregoryKimball added 0 - Backlog In queue waiting for assignment libcudf Affects libcudf (C++/CUDA) code. cuIO cuIO issue and removed Needs Triage Need team to review and classify labels Dec 14, 2023
@davidwendt
Copy link
Contributor

Closed by #16034

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0 - Backlog In queue waiting for assignment cuIO cuIO issue feature request New feature or request libcudf Affects libcudf (C++/CUDA) code.
Projects
Status: No status
Development

No branches or pull requests

4 participants