Code

Partial clean up of ORC writer#7324

Code

Merged

rapids-bot[bot] merged 62 commits intorapidsai:branch-0.19rapidsai/cudf:branch-0.19from vuule:refactor-orc-writer-hostvuule/cudf:refactor-orc-writer-hostCopy head branch name to clipboardMar 4, 2021

+965-765

Commits

Commits on Jan 30, 2021

• throw if bool null count would lead to corruption; disable a failing stats test

  vuule

  committed
  220b12a

  Copy full SHA for 220b12a

Commits on Feb 1, 2021

• review feedback

  vuule

  committed
  98aadf1

  Copy full SHA for 98aadf1

• style fix

  vuule

  committed
  439d1da

  Copy full SHA for 439d1da

Commits on Feb 2, 2021

• Merge branch 'branch-0.19' of https://github.com/rapidsai/cudf into bug-orc-null-bool

  vuule

  committed
  fa1428b

  Copy full SHA for fa1428b

Commits on Feb 3, 2021

• Merge branch 'branch-0.19' of https://github.com/rapidsai/cudf into bug-orc-null-bool

  vuule

  committed
  f6aa4f4

  Copy full SHA for f6aa4f4

Commits on Feb 4, 2021

• split encode_columns into separate steps

  vuule

  committed
  9e6f24b

  Copy full SHA for 9e6f24b

• Merge branch 'branch-0.19' of https://github.com/rapidsai/cudf into bug-orc-null-bool

  vuule

  committed
  b394c29

  Copy full SHA for b394c29

• use host_span for orc column array

  vuule

  committed
  c498ebd

  Copy full SHA for c498ebd

Commits on Feb 5, 2021

• add type for stripe boundaries to avoid repeated computation of stripe offsets

  vuule

  committed
  8835a26

  Copy full SHA for 8835a26

• Merge branch 'branch-0.19' of https://github.com/rapidsai/cudf into refactor-orc-writer-host

  vuule

  committed
  de7cfb1

  Copy full SHA for de7cfb1

• fix warning

  vuule

  committed
  9cc193b

  Copy full SHA for 9cc193b

• use uvector; fix merge

  vuule

  committed
  c0c6123

  Copy full SHA for c0c6123

• move stripe comp to separate class

  vuule

  committed
  f465c4c

  Copy full SHA for f465c4c

• group orc streams data into a new type; move encoded_data and chunks to new type

  vuule

  committed
  e76be22

  Copy full SHA for e76be22

Commits on Feb 6, 2021

• use streams instead of streams.streams

  vuule

  committed
  63d95ba

  Copy full SHA for 63d95ba

Commits on Feb 8, 2021

• Merge branch 'branch-0.19' of https://github.com/rapidsai/cudf into refactor-orc-writer-host

  vuule

  committed
  2635da2

  Copy full SHA for 2635da2

• move encode to encoder_chunks (might get reverted); remove redundant parameters

  vuule

  committed
  abec0aa

  Copy full SHA for abec0aa

Commits on Feb 9, 2021

• split stream info from chunks

  vuule

  committed
  306933c

  Copy full SHA for 306933c

• Merge branch 'branch-0.19' of https://github.com/rapidsai/cudf into refactor-orc-writer-host

  vuule

  committed
  17f8902

  Copy full SHA for 17f8902

Commits on Feb 11, 2021

• rename chunk stream type and members

  vuule

  committed
  c7d9efa

  Copy full SHA for c7d9efa

• separate chunk and shunk streams init loops

  vuule

  committed
  ca7a662

  Copy full SHA for ca7a662

• transpose chunks array

  vuule

  committed
  4407745

  Copy full SHA for 4407745

• transpose chunk_streams array

  vuule

  committed
  fa435a4

  Copy full SHA for fa435a4

Commits on Feb 12, 2021

• rename loop iterators

  vuule

  committed
  60956ba

  Copy full SHA for 60956ba

• Merge branch 'branch-0.19' of https://github.com/rapidsai/cudf into refactor-orc-writer-host

  vuule

  committed
  0817bcf

  Copy full SHA for 0817bcf

• fix compiler warning

  vuule

  committed
  990f82f

  Copy full SHA for 990f82f

• add hostdevice_matrix; use it for chunks in encode

  vuule

  committed
  a86428e

  Copy full SHA for a86428e

Commits on Feb 15, 2021

• replace chunk streams with a matrix

  vuule

  committed
  051c078

  Copy full SHA for 051c078

• Merge branch 'branch-0.19' of https://github.com/rapidsai/cudf into refactor-orc-writer-host

  vuule

  committed
  4250dea

  Copy full SHA for 4250dea

Commits on Feb 16, 2021

• stripe boundaries refactor

  vuule

  committed
  2c2e2c6

  Copy full SHA for 2c2e2c6

• extract the null check loop

  vuule

  committed
  f64e27e

  Copy full SHA for f64e27e

Commits on Feb 17, 2021

• reorg chunk streams init

  vuule

  committed
  ddfe1ba

  Copy full SHA for ddfe1ba

• Merge branch 'branch-0.19' of https://github.com/rapidsai/cudf into refactor-orc-writer-host

  vuule

  committed
  2fb3f37

  Copy full SHA for 2fb3f37

Commits on Feb 18, 2021

• namespacing

  vuule

  committed
  30510de

  Copy full SHA for 30510de

• add conversion operators to hostdevice vectors

  vuule

  committed
  c94264e

  Copy full SHA for c94264e

• remove redundant num_columns/rowgroups parameters

  vuule

  committed
  e1a53d1

  Copy full SHA for e1a53d1

• use hostdevice_2dvector for stripe streams; add conversions to spans

  vuule

  committed
  099ba68

  Copy full SHA for 099ba68

• small clean up

  vuule

  committed
  6a33771

  Copy full SHA for 6a33771

Commits on Feb 19, 2021

• minor clean up; add id() to orc_column_view to avoid some raw loops

  vuule

  committed
  4a49773

  Copy full SHA for 4a49773

• docs;naming

  vuule

  committed
  ed47895

  Copy full SHA for ed47895

• small clean up around write_index_stream

  vuule

  committed
  0fe82af

  Copy full SHA for 0fe82af

• 2dspan unit tests

  vuule

  committed
  ccd6d50

  Copy full SHA for ccd6d50

• fix pointer to array syntax in span

  vuule

  committed
  975be44

  Copy full SHA for 975be44

Commits on Feb 20, 2021

• docs

  vuule

  committed
  fedddd7

  Copy full SHA for fedddd7

• docs, naming, param shuffle

  vuule

  committed
  a4b89cf

  Copy full SHA for a4b89cf

• docs and such

  vuule

  committed
  3dcf55e

  Copy full SHA for 3dcf55e

• remove redundant num_rows parameters

  vuule

  committed
  7cefba1

  Copy full SHA for 7cefba1

• minor changes

  vuule

  committed
  e97befa

  Copy full SHA for e97befa

Commits on Feb 22, 2021

• Merge branch 'branch-0.19' of https://github.com/rapidsai/cudf into refactor-orc-writer-host

  vuule

  committed
  15fedfe

  Copy full SHA for 15fedfe

• consolidate redundant *_2dspan code

  vuule

  committed
  c1318bd

  Copy full SHA for c1318bd

• span docs

  vuule

  committed
  167e9bf

  Copy full SHA for 167e9bf

Commits on Feb 23, 2021

• Merge branch 'branch-0.19' of https://github.com/rapidsai/cudf into refactor-orc-writer-host

  vuule

  committed
  3f6632c

  Copy full SHA for 3f6632c

• use template template param in base_2dspan

  vuule

  committed
  ceb5536

  Copy full SHA for ceb5536

Commits on Mar 1, 2021

• Merge branch 'branch-0.19' of https://github.com/rapidsai/cudf into refactor-orc-writer-host

  vuule

  committed
  1922919

  Copy full SHA for 1922919

Commits on Mar 2, 2021

• move string cost computation out for the per-stripe loop

  vuule

  committed
  ca34c29

  Copy full SHA for ca34c29

• Update writer_impl.hpp

  Show description for b74f746
  vuule

  authored
  b74f746

  Copy full SHA for b74f746

Commits on Mar 3, 2021

• Merge branch 'branch-0.19' of https://github.com/rapidsai/cudf into refactor-orc-writer-host

  vuule

  committed
  1fa68ec

  Copy full SHA for 1fa68ec

• add helper function to get total size of a list of stripes

  vuule

  committed
  2b6e11f

  Copy full SHA for 2b6e11f

• Merge branch 'refactor-orc-writer-host' of https://github.com/vuule/cudf into refactor-orc-writer-host

  vuule

  committed
  220a721

  Copy full SHA for 220a721

Commits on Mar 4, 2021

• Merge branch 'branch-0.19' into refactor-orc-writer-host

  vuule

  authored
  7572f7c

  Copy full SHA for 7572f7c

• style fix

  vuule

  committed
  4dbc732

  Copy full SHA for 4dbc732

• merge fix

  vuule

  committed
  a490ff1

  Copy full SHA for a490ff1