[FEA] Allow to run groupby/reduction with externally derived aggregations

[ttnghia]

This idea arose after many times trying to add new aggregations into the libcudf framework to accommodate specific use cases outside of cudf. However, most of the time, the application (Spark plugin) wants very special behaviors that cannot be accommodated. For example, for M2/MERGE_M2 aggregations, we want to output more than just one columns (the main M2 values as well as their intermediate values) for reuse somewhere else.

I would like to refactor the grouby/reduction framework such that it allows runing on aggregations extended outside of libcudf. By doing so, the downstream applications can implement any new, customized aggregations they want and call libcudf code on them. The outside aggregations just need to be implemented from classes derived from cudf base classes (cudf::groupby_aggregation for example).

Allowing extension like this would be very beneficial in the long term, allowing any downstream application to accommodate their needs and maximize performance gain. That would also help reduce maintenance efforts in the libcudf repository.

[davidwendt]

We actually have precedence for custom UDF aggregation

cudf/cpp/include/cudf/aggregation.hpp

Lines 588 to 600 in bf2ee32

	/**
	* @brief Factory to create an aggregation base on UDF for PTX or CUDA
	*
	* @param[in] type: either udf_type::PTX or udf_type::CUDA
	* @param[in] user_defined_aggregator A string containing the aggregator code
	* @param[in] output_type expected output type
	*
	* @return An aggregation containing a user-defined aggregator string
	*/
	template <typename Base = aggregation>
	std::unique_ptr<Base> make_udf_aggregation(udf_type type,
	std::string const& user_defined_aggregator,
	data_type output_type);

Example usage in rolling here:

cudf/cpp/tests/rolling/grouped_rolling_test.cpp

Lines 207 to 208 in bf2ee32

	auto cuda_udf_agg = cudf::make_udf_aggregation<cudf::rolling_aggregation>(
	cudf::udf_type::CUDA, cuda_func, cudf::data_type{cudf::type_id::INT64});