Code

Add SHA-1 and SHA-2 hash functions.#9215

Code

Closed

bdice wants to merge 119 commits intorapidsai:branch-23.12rapidsai/cudf:branch-23.12from bdice:shabdice/cudf:shaCopy head branch name to clipboard

+2,332-29

Commits

Commits on Sep 16, 2021

• Initial SHA structure.

  Show description for 436e652
  

  rwlee

  and

  bdice

  committed
  436e652

• First pass of SHA1 implementation.

  Show description for 7c1b661
  

  rwlee

  and

  bdice

  committed
  7c1b661

• Apply clang-format.

  bdice

  committed
  aac184b

• Update comments and docstrings, enable SHA-1.

  bdice

  committed
  ade59b7

• Add test values for SHA-1.

  bdice

  committed
  ec4673e

• Remove unused seed value from MD5Hash.

  bdice

  committed
  2596297

• Use std::memcpy, add comments.

  bdice

  committed
  f775bbc

• Use __byte_perm to swap endianness.

  bdice

  committed
  955dadf

• Update comments.

  bdice

  committed
  7dd9a9e

• Update for changes to cudf::make_strings_column API.

  bdice

  committed
  77136bc

• Make rotl32 a CUDA_DEVICE_CALLABLE, use __funnelshift_l intrinsic.

  bdice

  committed
  4f7e0d4

• Add comments.

  bdice

  committed
  e42b5a2

• Move MD5 and SHA functions to the only .cu file that uses them.

  bdice

  committed
  070ee5f

• Use named constant for MD5 chunk size.

  bdice

  committed
  252147e

• Fix issue by using named temp variables.

  bdice

  committed
  d998656

• Use intrinsic for swapping endianness.

  bdice

  committed
  f535d27

• Fix bug in message length. Must be stored as a big-endian 64-bit value.

  bdice

  committed
  9d096f3

• Improve comments.

  bdice

  committed
  a7a8e6e

• Expand anonymous namespace.

  bdice

  committed
  dfb1c99

• Clean up SHA implementation and expand tests.

  bdice

  committed
  9735b2f

• Enable all tests.

  bdice

  committed
  4a7d8d2

• Revert changes to MD5 (separated into rapidsai#9212).

  bdice

  committed
  585bea4

• Move endian swaps into functions.

  bdice

  committed
  9ce6f17

• Draft of unified SHA functions - issues runtime error because it uses device virtual functions that cannot be resolved.

  bdice

  committed
  516b857

• Use CRTP design. Also add back missing code to write the output.

  bdice

  committed
  f4e8c70

• Use generic sha_hash function for SHA-1 (can be reused for SHA-256, SHA-512).

  bdice

  committed
  67bf9af

• Intermediate work on SHA-256.

  bdice

  committed
  ead336e

• Fix bugs in SHA-256 implementation (uninitialized hash constants, incorrect types/sizes.)

  bdice

  committed
  5867ebf

• Add SHA-256 tests.

  bdice

  committed
  c94dc8c

• Add SHA-512 draft (one failing test).

  bdice

  committed
  33878d1

• Fix bug in message finalization.

  bdice

  committed
  7abc5ac

• Draft of SHA-224, SHA-384.

  bdice

  committed
  a89f34c

• Use explicit SHA word types, add implementations of SHA-224 and SHA-384.

  bdice

  committed
  d9ecdfa

• Enable SHA-224, SHA-384.

  bdice

  committed
  cc683fc

• Add tests for SHA-224, SHA-384.

  bdice

  committed
  ad65b49

• Use rmm::cuda_stream_view.

  bdice

  committed
  f338217

Commits on Sep 20, 2021

• Small refactorings.

  bdice

  committed
  c46965e

• Use process_key function.

  bdice

  committed
  540f983

• Use process function for string data.

  bdice

  committed
  244f437

• Add comments to process function.

  bdice

  committed
  2023ac1

• Use constexpr.

  bdice

  committed
  502e11a

• Split sha1_hash_step.

  bdice

  committed
  82dd92b

• Move hash step functions above the SHA classes.

  bdice

  committed
  f05a43a

• Use east const, add comments.

  bdice

  committed
  f291c93

• Pass hash state by reference.

  bdice

  committed
  4b49f2f

• Pass hash state by reference in operator and process functions.

  bdice

  committed
  0ffa903

• Update comment.

  bdice

  committed
  84c034d

Commits on Sep 21, 2021

• Store sha_intermediate_data in Hasher instance.

  bdice

  committed
  987992e

• Drop const.

  bdice

  committed
  8e38d61

• Use underlying for hash_step calls.

  bdice

  committed
  56ab37b

• Use memcpy instead of std::memcpy.

  bdice

  committed
  b9a37cd

• Update Java enum.

  bdice

  committed
  5ae53ca

• Update Cython enum.

  bdice

  committed
  1591793

• Rename fixed width process function.

  bdice

  committed
  dbd469c

• Combine unsupported type dispatches.

  bdice

  committed
  48a08c4

• Use const&.

  bdice

  committed
  0d280a5

• Perform memcpys once instead of as a loop.

  bdice

  committed
  952251b

• Move hash constants into sha_hash.cu.

  bdice

  committed
  2474ec9

• Fail on invalid hash functions.

  bdice

  committed
  d7cfbc5

• Use uint32_t consistently to avoid mixing signed/unsigned types.

  bdice

  committed
  8c6983d

• Simplify word size.

  bdice

  committed
  590f8b3

• Minor improvements to constness.

  bdice

  committed
  440583d

• Add tests for empty tables.

  bdice

  committed
  d9ae7f1

Commits on Sep 22, 2021

• Enable debug mode on sha_hash.cu so that CI can compile.

  bdice

  committed
  4e8fdf3

Commits on Sep 27, 2021

• Rename intermediate data to hash state.

  bdice

  committed
  c47945d

• Use HashDispatcher for type dispatch.

  bdice

  committed
  b37e5f0

• Cleanup of templates and class/variable names.

  bdice

  committed
  607aaad

Commits on Sep 28, 2021

• Reorganize headers, update error message.

  bdice

  committed
  2779a96

• Generate null mask for all hash values.

  bdice

  committed
  93b6f54

• Update tests for new null behavior.

  bdice

  committed
  f850e5f

Commits on Oct 4, 2021

• Merge remote-tracking branch 'upstream/branch-21.12' into sha

  bdice

  committed
  c5711c8

• Simplify SFINAE.

  bdice

  committed
  1c6cc2b

Commits on Oct 8, 2021

• Add benchmarks for SHA functions.

  bdice

  committed
  2374df4

• Limit range to reduce memory usage for SHA512.

  bdice

  committed
  c0381b1

Commits on Oct 14, 2021

• Merge remote-tracking branch 'upstream/branch-21.12' into sha

  bdice

  committed
  21b3e26

Commits on Oct 19, 2021

• Add SHA methods to Python.

  bdice

  committed
  d1936f3

• Add tests for SHA hashing.

  bdice

  committed
  a373221

Commits on Oct 20, 2021

• Move result_location to constructor.

  bdice

  committed
  a7773b8

Commits on Oct 22, 2021

• Merge branch 'branch-21.12' into sha

  bdice

  committed
  92466b6

Commits on Oct 25, 2021

• Merge remote-tracking branch 'upstream/branch-21.12' into sha

  bdice

  committed
  27ee762

• Fix SHA constant qualifiers.

  bdice

  committed
  927b419

• Rename and move leaf type check.

  bdice

  committed
  6decb54

• Move shared utility functions to utilities/hash_functions.cuh.

  bdice

  committed
  95b0945

• Clean up use of utility functions.

  bdice

  committed
  327effb

Commits on Oct 26, 2021

• Add includes for column_device_view.

  bdice

  committed
  f2f6715

• Simplify dispatcher construction.

  bdice

  committed
  665a692

• Combine SFINAE templates into one method with if constexpr.

  bdice

  committed
  bce1f65

• Use bitmask_and for MD5 null mask.

  bdice

  committed
  aa14261

• Move base sha_hash function into anonymous namespace.

  bdice

  committed
  adf2a23

Commits on Mar 16, 2022

• Merge remote-tracking branch 'upstream/branch-22.04' into sha

  bdice

  committed
  b8cdbf6

Commits on Mar 18, 2022

• Merge remote-tracking branch 'upstream/branch-22.04' into sha

  bdice

  committed
  4e9ab2c

Commits on May 23, 2022

• Merge remote-tracking branch 'upstream/branch-22.08' into sha

  bdice

  committed
  98595e5

Commits on Jun 25, 2022

• Merge remote-tracking branch 'upstream/branch-22.08' into sha

  bdice

  committed
  ed20e83

• Update copyright.

  bdice

  committed
  c6f6a82

Commits on Jul 18, 2022

• Merge remote-tracking branch 'upstream/branch-22.08' into sha

  bdice

  committed
  3a8d4a6

Commits on Nov 15, 2022

• Merge remote-tracking branch 'upstream/branch-22.12' into sha

  bdice

  committed
  0d2e91f

• Merge remote-tracking branch 'upstream/branch-23.02' into sha

  bdice

  committed
  ae6c58b

Commits on Nov 16, 2022

• Merge remote-tracking branch 'upstream/branch-23.02' into sha

  bdice

  committed
  42beabd

Commits on Nov 17, 2022

• Use __device__ inline instead of CUDA_DEVICE_CALLABLE.

  bdice

  committed
  96cd76f

• Use null count from bitmask_and.

  bdice

  committed
  3b2b618

• Drop constexpr because is_fixed_width is no longer constexpr.

  bdice

  committed
  706c702

• Use CUDF_TEST_EXPECT_COLUMNS_EQUAL.

  bdice

  committed
  9e44661

• Fix namespaces.

  bdice

  committed
  17591c7

• Remove default stream/mr.

  bdice

  committed
  00b172c

• Update includes.

  bdice

  committed
  9689d92

• Try compiling sha_hash.cu in CI without debug flag.

  bdice

  committed
  1dba6b3

Commits on Jan 21, 2023

• Merge branch 'branch-23.02' into sha

  bdice

  authored
  f4e6aef

Commits on Feb 1, 2023

• Merge remote-tracking branch 'upstream/branch-23.04' into sha

  bdice

  committed
  65ee2b9

• Update copyright.

  bdice

  committed
  dfa1dbb

Commits on Oct 17, 2023

• Merge remote-tracking branch 'upstream/branch-23.12' into sha

  bdice

  committed
  b70fa8e

• Remove combined test file.

  bdice

  committed
  143ae6a

• Split into multiple TUs.

  bdice

  committed
  68b23e0

Commits on Oct 18, 2023

• Update tests.

  bdice

  committed
  9fd9e8d

Commits on Oct 20, 2023

• Merge remote-tracking branch 'upstream/branch-23.12' into sha

  bdice

  committed
  84f0527

Commits on Oct 31, 2023

• Merge remote-tracking branch 'upstream/branch-23.12' into sha

  bdice

  committed
  7598247

Commits on Nov 9, 2023

• Merge remote-tracking branch 'upstream/branch-23.12' into sha

  bdice

  committed
  80185b4

• Add alignment to avoid compiler bug.

  bdice

  committed
  1442bfd

• Merge branch 'branch-23.12' into sha

  bdice

  authored
  2cf9c1a

• Remove extra copy of device function.

  bdice

  committed
  47ccdd2