Code

multibyte_split#8702

Code

Merged

rapids-bot[bot] merged 93 commits intorapidsai:branch-21.10rapidsai/cudf:branch-21.10from cwharris:multibyte-splitcwharris/cudf:multibyte-splitCopy head branch name to clipboardAug 24, 2021

+1,673-2

Commits

Commits on Jun 27, 2021

• multibyte-split scaffolding

  cwharris

  committed
  e1b71e6

• cudf::io::text::input_stream

  cwharris

  committed
  836773a

Commits on Jul 2, 2021

• trie test scaffolding

  cwharris

  committed
  3e06c18

Commits on Jul 7, 2021

• superstate + tests

  cwharris

  committed
  ac14dbd

• added device trie

  cwharris

  committed
  ea8cee2

• add superstate to multibyte_split

  cwharris

  committed
  a4a8dd0

Commits on Jul 8, 2021

• cub block scan superstates

  cwharris

  committed
  094d2d2

Commits on Jul 9, 2021

• block-wide superstate matching

  cwharris

  committed
  1117ab8

• fix superstate constructor bug where only the first 8 states were initialized

  cwharris

  committed
  51b1444

• multibyte_split multiple delimeter support

  cwharris

  committed
  d1f7eb3

• scan output-offsets in multibyte_split

  cwharris

  committed
  a628d73

• printf offsets in multibyte_split

  cwharris

  committed
  e1cc84d

• add match-length to trie to adjust for output offset in multibyte_split

  cwharris

  committed
  c7177bc

• adjust multibyte_split test case to expect delimiters to be retained in output

  cwharris

  committed
  42dc014

• printf match_begin and match_end for multibyte_split

  cwharris

  committed
  5171711

Commits on Jul 10, 2021

• multibyte_split test passing

  cwharris

  committed
  6b62ceb

Commits on Jul 12, 2021

• add multibyte_split comments, break test intentionally to work on multi-block scaling

  cwharris

  committed
  a2c9756

Commits on Jul 13, 2021

• multibyte_split add multi-block support

  cwharris

  committed
  21b8b25

• rename BYTES_PER_TILE to ITEMS_PER_TILE

  cwharris

  committed
  f59a93e

Commits on Jul 14, 2021

• add bounds check to multibyte_split load and flag

  cwharris

  committed
  5fa112a

• multibyte_split benchmark scaffolding

  cwharris

  committed
  cf42fd0

• multibyte_split increase threads per block and adjust test case.

  cwharris

  committed
  e6e9741

Commits on Jul 16, 2021

• use circular buffer in multibyte_split to allow for stream inputs

  cwharris

  committed
  b5c2e05

• update multibyte_split to work with streaming inputs

  cwharris

  committed
  738af48

• consolidate two passes of stream-scanning to a single function

  cwharris

  committed
  0121b22

• add tile_state partial to multibyte_split but dont use yet

  cwharris

  committed
  a233ca2

• add reusable tilestate callback to multibyte_split

  cwharris

  committed
  4946058

• begin working on warp-reduce window aggregation of tile state in multibyte_split

  cwharris

  committed
  d69aeca

Commits on Jul 17, 2021

• fix multibyte_split bug where non-streaming approach would hang

  cwharris

  committed
  079d1ea

Commits on Jul 18, 2021

• interleaved streaming io for multibyte_split

  cwharris

  committed
  970aac2

Commits on Jul 19, 2021

• use no-copy string column construction in multibyte_split

  cwharris

  committed
  fee7ebb

• document multibyte_split minimum tile count requirements

  cwharris

  committed
  e5a5204

• Merge branch 'branch-21.10' into multibyte-split

  cwharris

  committed
  216d620

Commits on Jul 22, 2021

• multibyte_split tunable concurrency via stream pool

  cwharris

  committed
  65af4de

Commits on Jul 23, 2021

• multibyte_split remove device_istream replace with data_chunk_reader

  cwharris

  committed
  a4fe128

• add data_chunk_source factories, nvtx ranges to multibyte_split, use temp dir for benchmark files

  cwharris

  committed
  9bc6c89

• use make_device_uvector_async in trie.hpp

  cwharris

  committed
  08b3069

• rm device_istream

  cwharris

  committed
  7088791

• multibyte_split add some docs, add more test cases

  cwharris

  committed
  b61c14f

• revert CMakeLists ordering

  cwharris

  committed
  017f05d

Commits on Jul 25, 2021

• convert trie storage from SOA to AOS

  cwharris

  committed
  f432e68

Commits on Jul 26, 2021

• Merge branch 'branch-21.10' of github.com:rapidsai/cudf into multibyte-split

  cwharris

  committed
  59a70a9

• fix spelling mistakes

  cwharris

  committed
  f1d3b4a

Commits on Jul 27, 2021

• break multibyte_split by adding queue/multistate support

  cwharris

  committed
  51ac35c

Commits on Jul 28, 2021

• Merge branch 'branch-21.10' of github.com:rapidsai/cudf into multibyte-split

  cwharris

  committed
  3d04556

Commits on Jul 29, 2021

• fix abac pattern matching test, introduce new bug :(

  cwharris

  committed
  1fb36ee

• fix multibyte_split aggregation strategy to avoid assuming T{} is an identity value

  cwharris

  committed
  ecf440a

• Merge branch 'multibyte-split-queue' into multibyte-split

  cwharris

  committed
  9e34efb

• add second host buffer to istream_data_chunk_reader to facilitate overlapping h2d copies~

  cwharris

  committed
  fc014e5

• actually add second buffer to istream_data_chunk_reader

  cwharris

  committed
  896ed31

Commits on Jul 30, 2021

• Merge branch 'branch-21.10' of github.com:rapidsai/cudf into multibyte-split

  cwharris

  committed
  7792521

• clean up multibyte_split code

  cwharris

  committed
  2f75b50

• adjust copyright

  cwharris

  committed
  162e9cf

• remove confusing test case in multibyte_split

  cwharris

  committed
  ade1150

• limit multibyte_split to 32 threads, because of a bug that needs fixing. add overlapping matches test, which also fails

  cwharris

  committed
  8e08012

Commits on Jul 31, 2021

• fix emoji bits documentation

  cwharris

  committed
  5ad2148

Commits on Aug 2, 2021

• style adjustments and documentation update to multibyte_split

  cwharris

  committed
  511ab9f

• move tile-scanning utilites to detail namespace

  cwharris

  committed
  69280e8

• remove "inline" from constexpr members in cudf::io::text

  cwharris

  committed
  2d37dc9

Commits on Aug 3, 2021

• fix large input bug in multibyte_split where offsets were not accounted for correctly

  cwharris

  committed
  9c6bf2a

• improve data_chunk_reader docs

  cwharris

  committed
  ee817b1

• make multibyte_split accept data_chunk_source as a const& arg

  cwharris

  committed
  4cdbee5

• add tile_state.hpp to meta.yaml

  cwharris

  committed
  c3783db

• create bad-case scenario benchmark

  cwharris

  committed
  432399c

Commits on Aug 4, 2021

• remove data_chunk in favor of device_span until it becomes clear an raii type is required

  cwharris

  committed
  ad21c4f

• use std::vector<cuda_stream_view> instread of stream_pool

  cwharris

  committed
  18e0863

• rename ticket to h_ticket

  cwharris

  committed
  45e5b65

• adjust scan_tile_state_view::get_prefix to make the purpose of thread fence more obvious.

  cwharris

  committed
  ee122a8

Commits on Aug 5, 2021

• Merge branch 'branch-21.10' of github.com:rapidsai/cudf into multibyte-split

  cwharris

  committed
  c9d2889

Commits on Aug 6, 2021

• fix UB in multibyte_split concurrent kernel execution, improve perf

  cwharris

  committed
  ca6bbac

• add error messages to multibyte_split to indicate unsupported use cases

  cwharris

  committed
  d68d951

Commits on Aug 9, 2021

• remove __threadfence() in favor of cuda::atomic

  cwharris

  committed
  9684646

Commits on Aug 13, 2021

• improve multibyte_split benchmarks

  cwharris

  committed
  d3de062

• provide explicit memory_order for tile state status stores.

  cwharris

  committed
  d392140

• improve multibyte_split benchmarks

  cwharris

  committed
  42b8c88

• Merge branch 'branch-21.10' of github.com:rapidsai/cudf into multibyte-split

  cwharris

  committed
  b976525

• Merge branch 'branch-21.10' of github.com:rapidsai/cudf into multibyte-split

  cwharris

  committed
  d50f815

Commits on Aug 14, 2021

• add file and host benchmarks for multibyte_split

  cwharris

  committed
  40d81e8

• make use of div_rounding_up_safe

  cwharris

  committed
  3171339

• remove unused temp storage from tile state callback

  cwharris

  committed
  63c4bb0

Commits on Aug 17, 2021

• Merge branch 'branch-21.10' of github.com:rapidsai/cudf into multibyte-split

  cwharris

  committed
  eda265b

• simplify multibyte_split api to accept only a single delimiter

  cwharris

  committed
  05cdecf

Commits on Aug 19, 2021

• add strings column factory which takes device_uvectors

  cwharris

  committed
  a4d4d79

• add docs to cudf::io::text::detail::trie

  cwharris

  committed
  cef897d

Commits on Aug 23, 2021

• Merge branch 'branch-21.10' of github.com:rapidsai/cudf into multibyte-split

  cwharris

  committed
  097cadd

• add more documentation and comments to multibyte_split related code

  cwharris

  committed
  89ce0aa

• adjust multibyte_split benchmark deviation math to be representative of intent.

  cwharris

  committed
  d2735dd

Commits on Aug 24, 2021

• Merge branch 'branch-21.10' of github.com:rapidsai/cudf into multibyte-split

  cwharris

  committed
  5a1e4d6

• Merge branch 'multibyte-split' of github.com:cwharris/cudf into multibyte-split

  cwharris

  committed
  c15b5d2

• multibyte_split: replace typedef with using and replace uint32_t with std::size_t where appropriate

  cwharris

  committed
  615534d

• make data_chunk_reader::get_next_chunk docs more informative.

  cwharris

  committed
  bd67026

• fix style

  cwharris

  committed
  a61fd09

• Merge branch 'branch-21.10' of github.com:rapidsai/cudf into multibyte-split

  cwharris

  committed
  b0d4135