Code

Implement groupby apply with JIT#11452

Code

Merged

rapids-bot[bot] merged 157 commits intorapidsai:branch-23.02rapidsai/cudf:branch-23.02from bwyogatama:fea-groupby-apply-jitbwyogatama/cudf:fea-groupby-apply-jitCopy head branch name to clipboardJan 27, 2023

+1,438-190

Commits

Commits on Aug 3, 2022

• Groupby Apply with JIT (First Commit)

  bwyogatama

  committed
  8db918f

Commits on Aug 4, 2022

• Fix error in Pytest

  bwyogatama

  committed
  2d6b4c9

Commits on Aug 5, 2022

• JIT Caching Support

  bwyogatama

  committed
  fd8680e

• Add IdxMax and IdxMin

  bwyogatama

  committed
  9220658

• Add IdxMax and IdxMin

  bwyogatama

  committed
  f4bc7c4

Commits on Aug 14, 2022

• Dynamic Launch Parameter

  bwyogatama

  committed
  8659149

Commits on Aug 19, 2022

• Code cleanup #1

  bwyogatama

  committed
  b7ede43

Commits on Sep 23, 2022

• Merge remote-tracking branch 'origin/branch-22.10' into fea-groupby-apply-jit

  vyasr

  committed
  1dbbb77

• Add support for building the JIT functions with the rest of the build.

  vyasr

  committed
  f98fc63

• Make engine name consistent with tests

  vyasr

  committed
  11edd37

• Generalize compiled PTX selection for CUDA arch.

  vyasr

  committed
  1e12416

• Cleanup of strings_udf PTX detection

  vyasr

  committed
  d348fb8

• Fix tests with some hacks so that we can start validating.

  vyasr

  committed
  795e580

• Standardize the engine argument handling so that we get clear errors.

  vyasr

  committed
  0ce0a90

Commits on Sep 24, 2022

• Update style.

  vyasr

  committed
  3493d49

Commits on Oct 26, 2022

• Refactoring C++ function

  bwyogatama

  committed
  7f9ea1f

Commits on Nov 2, 2022

• Fix bug in C++ and Python Cleanup

  

  Chameleon Cloud User

  committed
  3d76a44

Commits on Nov 21, 2022

• merge latest, resolve conflicts, pass tests

  brandon-b-miller

  committed
  ad878ac

• pass style, cleanup

  brandon-b-miller

  committed
  d876ad7

• start to move files

  brandon-b-miller

  committed
  f600196

Commits on Nov 22, 2022

• starting to refactor

  brandon-b-miller

  committed
  6cbdaf8

Commits on Nov 28, 2022

• continue to refactor typing

  brandon-b-miller

  committed
  3a11fe1

• move lowering to its own file

  brandon-b-miller

  committed
  8161548

• continue refactoring idxmin and idxmax functions

  brandon-b-miller

  committed
  52656ab

• add tests for idxmin and idxmax, not currently passing

  brandon-b-miller

  committed
  b9096f3

• normalize call_cuda_functions keys

  brandon-b-miller

  committed
  d21a099

• continued refactoring

  brandon-b-miller

  committed
  62aad1e

• refactoring lowering

  brandon-b-miller

  committed
  9ff058a

Commits on Nov 29, 2022

• continued refactoring

  brandon-b-miller

  committed
  5f07ca2

Commits on Dec 5, 2022

• Merge branch 'branch-23.02' into groupby-apply-updates

  brandon-b-miller

  committed
  38c3560

Commits on Jan 2, 2023

• CMake changes

  bwyogatama

  committed
  c12a9e3

• C++ changes

  bwyogatama

  committed
  e650c21

Commits on Jan 5, 2023

• Merge branch 'branch-23.02' into fea-groupby-apply-jit

  brandon-b-miller

  committed
  0fd6e22

Commits on Jan 6, 2023

• Merge branch 'branch-23.02' into fea-groupby-apply-jit

  brandon-b-miller

  committed
  aabdc5f

• style

  brandon-b-miller

  committed
  301eea1

Commits on Jan 8, 2023

• found the bug

  brandon-b-miller

  committed
  e50f4a6

• minor refactoring

  brandon-b-miller

  committed
  df1485d

• Merge branch 'branch-23.02' into fea-groupby-apply-jit

  brandon-b-miller

  committed
  353078c

Commits on Jan 9, 2023

• update/add tests

  brandon-b-miller

  committed
  14fe3cb

• continue refactoring tests

  brandon-b-miller

  committed
  f7791b4

Commits on Jan 12, 2023

• add docs, switch to partials

  brandon-b-miller

  committed
  7f63c90

Commits on Jan 17, 2023

• continue addressing reviews

  brandon-b-miller

  committed
  902223a

• Update python/cudf/cudf/core/udf/groupby_typing.py

  Show description for 78f8b6f
  

  brandon-b-miller

  and

  wence-

  authored
  78f8b6f

• Merge branch 'fea-groupby-apply-jit' of github.com:bwyogatama/cudf into fea-groupby-apply-jit

  brandon-b-miller

  committed
  611b864

• merge remote

  brandon-b-miller

  committed
  2849680

• address more reviews

  brandon-b-miller

  committed
  33109f5

• move utilities around

  brandon-b-miller

  committed
  07444eb

Commits on Jan 18, 2023

• template throughout c++

  brandon-b-miller

  committed
  39eb8f9

• Merge branch 'branch-23.02' into fea-groupby-apply-jit

  brandon-b-miller

  committed
  f30cd8b

• cpp code formatting

  PointKernel

  committed
  6158cb7

• Merge branch 'fea-groupby-apply-jit' of github.com:bwyogatama/cudf into fea-groupby-apply-jit

  brandon-b-miller

  committed
  93df707

• style

  brandon-b-miller

  committed
  5ae896a

• Use cudf size_type

  PointKernel

  committed
  a42d307

• Use std limits instead of macros

  PointKernel

  committed
  0e0b750

Commits on Jan 19, 2023

• merge latest, resolve conflicts, don't unset warnings

  brandon-b-miller

  committed
  e891e5f

• Merge branch 'fea-groupby-apply-jit' of github.com:bwyogatama/cudf into fea-groupby-apply-jit

  brandon-b-miller

  committed
  8188508

• remove redundant comments

  brandon-b-miller

  committed
  95fa402

• style fixes

  brandon-b-miller

  committed
  3de3add

Commits on Jan 20, 2023

• debug statements

  brandon-b-miller

  committed
  865bb5d

• patch numba linker based off groupby ptx file

  brandon-b-miller

  committed
  7788944

• Fix idxmin/max bug

  PointKernel

  committed
  bdea84c

• Use static_cast to avoid raw casting

  PointKernel

  committed
  0110075

• Cleanups: reinterpret_cast + remove redundant sync

  PointKernel

  committed
  b039ce7

• Merge branch 'branch-23.02' into fea-groupby-apply-jit

  brandon-b-miller

  committed
  a8c3a75

• Merge branch 'fea-groupby-apply-jit' of github.com:bwyogatama/cudf into fea-groupby-apply-jit

  brandon-b-miller

  committed
  21792c6

• Simplify block min/max logic

  PointKernel

  committed
  14dc674

• Merge branch 'fea-groupby-apply-jit' of github.com:bwyogatama/cudf into groupby

  PointKernel

  committed
  3a48a96

• Merge remote-tracking branch 'upstream/branch-23.02' into groupby

  PointKernel

  committed
  9766233

• Replace custom atomic add with cuda atomic_ref

  PointKernel

  committed
  99af3f2

• Simplify block sum logic

  PointKernel

  committed
  321fdab

• Simplify block var logic

  PointKernel

  committed
  c91a589

• Merge branch 'branch-23.02' into fea-groupby-apply-jit

  brandon-b-miller

  committed
  aa47763

• Refactor block idxmin/idxmax

  PointKernel

  committed
  cbc13e6

• Fix a minor bug

  PointKernel

  committed
  ab20731

• Merge branch 'fea-groupby-apply-jit' of github.com:bwyogatama/cudf into groupby

  PointKernel

  committed
  f721f9c

• Fix the floating point min value bug

  PointKernel

  committed
  a24f09e

• Refactor with CUDA CG

  PointKernel

  committed
  d908621

Commits on Jan 22, 2023

• C++ changes: (1) Addressing more reviewer's comment, (2) Replacing custom atomic with cuda/atomic for max and min (3) New C++ templating style

  bwyogatama

  committed
  595746a

Commits on Jan 23, 2023

• Merge branch 'branch-23.02' into fea-groupby-apply-jit

  brandon-b-miller

  committed
  9a93af7

• Merge branch 'fea-groupby-apply-jit' of github.com:bwyogatama/cudf into fea-groupby-apply-jit

  brandon-b-miller

  committed
  9d3c431

• style

  brandon-b-miller

  committed
  bb8b7c3

• Use proper cuda thread scope

  PointKernel

  committed
  1f475f0

• Merge remote-tracking branch 'upstream/branch-23.02' into groupby

  PointKernel

  committed
  0bb68bc

• Request cpp review for udf_cpp

  PointKernel

  committed
  209188f

• Remove unsupported data types

  PointKernel

  committed
  826ed25

• Use exclusively thread 0 to write var output + minor cleanups

  PointKernel

  committed
  1cf91ea

• Addressing more reviewer's commment and fix mean bug

  bwyogatama

  committed
  381dd00

• error if nulls are present, dont not patch numba linker in strings_udf

  brandon-b-miller

  committed
  ee87548

• Style

  brandon-b-miller

  committed
  1d4edc8

• Merge branch 'fea-groupby-apply-jit' of github.com:bwyogatama/cudf into groupby

  PointKernel

  committed
  4ff80f4

• Replace int with int32_t

  PointKernel

  committed
  4fe21fb

• Remove unused template specializations

  PointKernel

  committed
  3fbe3ff

• Merge remote-tracking branch 'upstream/branch-23.02' into groupby

  PointKernel

  committed
  b59d31e

Commits on Jan 24, 2023

• update utility functions to no longer be strings_udf specific

  brandon-b-miller

  committed
  9af3670

• tweak thread guard logic in groupby template

  brandon-b-miller

  committed
  afd0949

• Merge branch 'branch-23.02' into fea-groupby-apply-jit

  brandon-b-miller

  committed
  c05e889

• Apply suggestions from code review

  Show description for 1828ef7
  

  brandon-b-miller

  and

  vyasr

  authored
  1828ef7

• fix small bug

  brandon-b-miller

  committed
  6489950

• refactor group constructor lowering

  brandon-b-miller

  committed
  9b83d78

• partially address reviews

  brandon-b-miller

  committed
  73a2ba1

• continue addressing reviews

  brandon-b-miller

  committed
  4dfb790

• extraneous copyright

  brandon-b-miller

  committed
  43be944

• fix small comment error in cmake

  brandon-b-miller

  committed
  b5f8f63

• inline _is_jit_supported_type

  brandon-b-miller

  committed
  8bbd725

• adjust logic in maybe_patch_numba_linker

  brandon-b-miller

  committed
  2df3216

• Apply suggestions from code review

  Show description for e8137e3
  

  PointKernel

  and

  bdice

  authored
  e8137e3

• Correct numerical limits

  PointKernel

  committed
  c5e7445

• Apply suggestions from code review

  Show description for 761261c
  

  brandon-b-miller

  and

  bdice

  authored
  761261c

• fix small syntax error

  brandon-b-miller

  committed
  9884897

• add an updater to update-version.sh

  brandon-b-miller

  committed
  41b42c7

• refactor groupby.apply top level impl into separate methods

  brandon-b-miller

  committed
  5855f5c

• GroupType.size_type -> GroupType.group_size_type

  brandon-b-miller

  committed
  d6a3ef2

• introduce group_size_type as a global

  brandon-b-miller

  committed
  9b60a62

• use index_default_type in idxmax/idxmin lowering

  brandon-b-miller

  committed
  f0a9af8

• rename some utility functions and add docs

  brandon-b-miller

  committed
  6708655

• tweak previous function

  brandon-b-miller

  committed
  3e5149d

• Addressing reviewers' comments

  bwyogatama

  committed
  c253b8f

• Merge branch 'fea-groupby-apply-jit' of https://github.com/bwyogatama/cudf into fea-groupby-apply-jit

  bwyogatama

  committed
  51d7ec7

• unused import

  brandon-b-miller

  committed
  bae845d

• Replace std numeric limits with cudf device operators

  PointKernel

  committed
  e91b641

• Merge branch 'fea-groupby-apply-jit' of github.com:bwyogatama/cudf into groupby

  PointKernel

  committed
  0d35554

• add tests for special values

  brandon-b-miller

  committed
  73892e1

• Merge branch 'fea-groupby-apply-jit' of github.com:bwyogatama/cudf into fea-groupby-apply-jit

  brandon-b-miller

  committed
  08cbcb7

Commits on Jan 25, 2023

• Apply suggestions from code review

  PointKernel

  authored
  81bfeb1

• Code formatting

  PointKernel

  committed
  3d76481

• Add more special value tests

  PointKernel

  committed
  97490af

• Fix bugs when all values are nans

  PointKernel

  committed
  43694f7

• Fix a result init bug in are_all_nans

  PointKernel

  committed
  928d404

• separate out idmax and idxmin tests with special values

  brandon-b-miller

  committed
  8079047

• remove redundant tests

  brandon-b-miller

  committed
  11c0eb6

• answer is the offsets not just an array of zeroes

  brandon-b-miller

  committed
  3a5afa6

• dynamically register reductions

  brandon-b-miller

  committed
  4d719b5

• Add corner cases handling to idxmin/idxmax

  PointKernel

  committed
  5c5e37c

• Apply suggestions from code review

  Show description for fac8d70
  

  PointKernel

  and

  bdice

  authored
  fac8d70

• Remove unroll pragma

  PointKernel

  committed
  2f9cc76

• Merge branch 'fea-groupby-apply-jit' of github.com:bwyogatama/cudf into groupby

  PointKernel

  committed
  961b3b9

• Merge remote-tracking branch 'upstream/branch-23.02' into groupby

  PointKernel

  committed
  5e6d4aa

Commits on Jan 26, 2023

• address remaining reviews

  brandon-b-miller

  committed
  6665ef9

• Merge branch 'fea-groupby-apply-jit' of github.com:bwyogatama/cudf into fea-groupby-apply-jit

  brandon-b-miller

  committed
  62b5a99

• fix tests

  brandon-b-miller

  committed
  62a8928

• Code formatting

  PointKernel

  committed
  de6b54c

• go back to index_default_type = types.int64 explicitly

  brandon-b-miller

  committed
  0c3d5a0

• Merge branch 'fea-groupby-apply-jit' of github.com:bwyogatama/cudf into fea-groupby-apply-jit

  brandon-b-miller

  committed
  b0e8c29

• merge resolve conflicts

  brandon-b-miller

  committed
  5a5c0fb

• style

  brandon-b-miller

  committed
  5db0b6c

• Update python/cudf/udf_cpp/groupby/CMakeLists.txt

  Show description for 7e2ca13
  

  brandon-b-miller

  and

  robertmaynard

  authored
  7e2ca13

• Cast mean results to double

  PointKernel

  committed
  40b8ce9

• Merge branch 'fea-groupby-apply-jit' of github.com:bwyogatama/cudf into groupby

  PointKernel

  committed
  bb8e0c3

• Merge remote-tracking branch 'upstream/branch-23.02' into groupby

  PointKernel

  committed
  0952784

• Merge branch 'branch-23.02' into fea-groupby-apply-jit

  PointKernel

  authored
  e0d0230

Commits on Jan 27, 2023

• address reviews

  brandon-b-miller

  committed
  81860c5

• minor edits

  brandon-b-miller

  committed
  2f352bc

• Compute blockstd via blockvar

  PointKernel

  committed
  0b407c8

• Merge branch 'fea-groupby-apply-jit' of github.com:bwyogatama/cudf into groupby

  PointKernel

  committed
  568ab97

• Merge remote-tracking branch 'upstream/branch-23.02' into groupby

  PointKernel

  committed
  83f8d88

• Use atomic operations to avoid concurrent writes

  PointKernel

  committed
  dbd5eeb

• Use int64_t atomic ref

  PointKernel

  committed
  eaa8ff7