Code

Spark list hashing#11292

Code

Merged

rapids-bot[bot] merged 48 commits intorapidsai:branch-22.08rapidsai/cudf:branch-22.08from bdice:spark-list-hashingbdice/cudf:spark-list-hashingCopy head branch name to clipboardJul 28, 2022

+427-22

Commits

Commits on Jun 28, 2022

• Add test for Spark list-of-list hashing.

  bdice

  committed
  ebf079a

Commits on Jul 7, 2022

• Improve test.

  bdice

  committed
  5632e7c

• Merge remote-tracking branch 'upstream/branch-22.08' into spark-list-hashing

  bdice

  committed
  d721597

Commits on Jul 18, 2022

• Merge remote-tracking branch 'upstream/branch-22.08' into spark-list-hashing

  bdice

  committed
  aee8fe0

Commits on Jul 19, 2022

• Copy experimental row hasher for modification.

  bdice

  committed
  6e55198

• Make preprocessed table methods public.

  bdice

  committed
  95f25cb

• Use structured binding to make names map more clearly to the objects.

  bdice

  committed
  ac21705

• Add note about hash types, fix tests.

  bdice

  committed
  4106aef

• Align hashing behavior closer to Spark. Some tests passing, but not all.

  bdice

  committed
  ca8557d

• Remove commented code, fix dispatch for decimal32 and decimal128.

  bdice

  committed
  703956f

• Reorder members to match constructor signature.

  bdice

  committed
  a98c130

• Update nested type hashing to act more like the previous serial hash code.

  bdice

  committed
  659bca9

• Use seed 42 for consistency with Spark.

  bdice

  committed
  fcca18a

Commits on Jul 20, 2022

• Clean up.

  bdice

  committed
  189f3bf

• Fix bug in test (wrong index).

  bdice

  committed
  ee754a7

• Template row_hasher on device_row_hasher class.

  bdice

  committed
  b163b49

• Fix up friends / private methods.

  bdice

  committed
  387390a

Commits on Jul 21, 2022

• Merge remote-tracking branch 'pub/branch-22.08' into bdice/list_hashing

  rwlee

  committed
  931bd33

Commits on Jul 22, 2022

• Enable list types in spark32BitMurmurHash3.

  bdice

  authored
  2218338

• Remove template, always use Spark hash functor.

  bdice

  authored
  bad0b5e

• Add summary of differences in Spark hashing.

  bdice

  authored
  2d30d2c

• clang-format

  bdice

  committed
  4fee0f8

• Switch back to template parameter so the device row hasher matches the template template parameter needed upstream.

  bdice

  committed
  1194723

Commits on Jul 25, 2022

• Merge remote-tracking branch 'pub/pull-request/11292' into bdice/list_hashing

  rwlee

  committed
  bec00fa

• Merge remote-tracking branch 'upstream/branch-22.08' into spark-list-hashing

  bdice

  committed
  ee6cfda

• Use has_nested_nulls instead of has_nulls.

  bdice

  committed
  873452c

• Rename to spark_device_row_hasher.

  bdice

  committed
  2826655

Commits on Jul 26, 2022

• Fix copyright.

  Show description for 3e70ac5
  

  bdice

  and

  vyasr

  authored
  3e70ac5

• Add test for structs of lists.

  bdice

  committed
  c4586f3

• Merge branch 'spark-list-hashing' of github.com:bdice/cudf into spark-list-hashing

  bdice

  committed
  481ac82

• Add failing test for lists of structs.

  bdice

  committed
  1f75acf

• Fail if a table with lists of structs is passed.

  bdice

  committed
  e179a4c

• Use spark_hash_value_type.

  bdice

  committed
  f057bc6

• Change template name to DeviceRowHasher.

  bdice

  committed
  929c589

• Merge remote-tracking branch 'pub/pull-request/11292' into bdice/list_hashing

  rwlee

  committed
  64727c9

• Improve comment.

  bdice

  committed
  248cfaa

• Rename to spark_murmur_device_row_hasher.

  bdice

  committed
  f90af13

• Use spark_hash_value_type more consistently, and enforce that null hash values are always the seed.

  bdice

  committed
  4a80876

• Added developer documentation.

  bdice

  committed
  5be3ea0

• Use list initialization.

  bdice

  committed
  cf6f6cd

Commits on Jul 27, 2022

• add list test

  rwlee

  committed
  6475517

• Merge remote-tracking branch 'pub/pull-request/11292' into bdice/list_hashing

  rwlee

  committed
  0c0a1fb

• nested struct java test

  rwlee

  committed
  028b0bc

• Remove deleted constructor.

  bdice

  committed
  5d05984

• Require SparkMurmurHash3_32 in constructor.

  bdice

  committed
  9281aee

• Remove deleted public constructor because the constructor is declared to be private.

  bdice

  committed
  fa50d2c

• Template only the device_hasher method.

  bdice

  committed
  ef0ad3b

Commits on Jul 28, 2022

• Temporarily remove JNI tests due to an outstanding allocation bug.

  bdice

  authored
  aaadac2