Add support for Hyper Log Log PLus Plus(HLL++)

[res-life]

Description

Spark approx_count_distinct description link
Spark accepts one column(can be nested column) and a double literal relativeSD.
Currently only support TypeSig.cpuAtomics types, for nested types, move to a follow-up task.

closes ##5199

Depending on JNI/cuDF PRs:
rapidsai/cudf#17133
NVIDIA/spark-rapids-jni#2522

TODO

• Add more test cases

Follow-up

Support nested types: move to follow-up tasks. Because for nested type, it's depending on xxhash64 to support nested type.

Perf test

// group by
import org.apache.spark.sql.functions
spark.range(10000000).repartition(5).withColumn("m", functions.expr("id % 10")).createOrReplaceTempView("tab")
spark.time(spark.sql("select m, APPROX_COUNT_DISTINCT(id) from tab group by m").show())

// reduction
spark.range(10000000).repartition(5).createOrReplaceTempView("tab")
spark.time(spark.sql("select APPROX_COUNT_DISTINCT(id) from tab ").show())

num_groups	CPU time(hot runs)	GPU time(hot runs)	speedup
10	1106ms, 1020ms, 1059ms	196ms, 208ms, 188ms	3.53x
1,000,000	5135ms, 5307ms, 5487ms	1447ms, 1565ms, 1497ms	5.38x
reduction	942ms, 1041ms, 973ms	169ms, 165ms, 180ms	5.75x

correctness

For int column, the results are identical between CPU and GPU.

Gpu result:
+---+-------------------------+                                                 
|  m|approx_count_distinct(id)|
+---+-------------------------+
|  0|                  1009779|
|  1|                   912573|
|  2|                   994262|
|  3|                   962191|
|  4|                   957975|
|  5|                   969328|
|  6|                   975973|
|  7|                  1017056|
|  8|                   989262|
|  9|                   954960|
+---+-------------------------+

CPU result:
+---+-------------------------+                                                 
|  m|approx_count_distinct(id)|
+---+-------------------------+
|  0|                  1009779|
|  1|                   912573|
|  2|                   994262|
|  3|                   962191|
|  4|                   957975|
|  5|                   969328|
|  6|                   975973|
|  7|                  1017056|
|  8|                   989262|
|  9|                   954960|
+---+-------------------------+

For the correctness of other types, need more checks.

Signed-off-by: Chong Gao [email protected]

[ttnghia]

Let' call by full name like GpuHyperLogLogPlusPlus to better reflect the CPU version.

[res-life]

Done

[ttnghia]

Not sure if "PlusPlus" is necessary.

Suggested change

	override val name: String = "CudfHLL"
	override val name: String = "CudfHyperLogLogPlusPlus"

[res-life]

Done.

[res-life]

Ready to review except test cases.

[revans2]

Looks good

[revans2]

nit: Using cpuAtomics for a GPU field gets to be kind of confusing. Could you please create a gpuAtomics instead?