[BUG] groupby on STRUCT / LIST input passes silently, returns incorrect grouping.

[mythrocks]

When a groupby is constructed on a STRUCT or LIST input, it does not currently fail, as one might expect of an unsupported type.

The groupby.aggregate() seems to return the input groupby column unchanged, implying that each row is its own group. I think this also contributes to issues like #8887, where the grouped_rolling_window() returns unexpected output.

It might be worth considering a CUDF_FAIL() for unsupported input types, at least until support is added.

The repro case for STRUCT input follows:

TEST_F(UtilitiesTest, groupby_struct_bork)
{
  using namespace cudf;
  using ints = fixed_width_column_wrapper<int32_t>;

  auto child       = ints{1, 2, 3, 1, 2, 2, 1, 3, 3, 2};
  auto struct_keys = structs_column_wrapper{child};
  ints vals{0, 1, 2, 3, 4, 5, 6, 7, 8, 9};

  auto expect_child       = ints{1, 2, 3};
  auto expect_struct_keys = structs_column_wrapper{expect_child};
  ints expect_vals{9, 19, 17};

  std::cout << "Struct input to groupby (keys)!" << std::endl;
  print(struct_keys);

  std::cout << "Expected output (grouped) struct keys!" << std::endl;
  print(expect_struct_keys);

  auto requests = std::vector<groupby::aggregation_request>{};
  requests.emplace_back(groupby::aggregation_request{});
  auto& agg_request  = requests.front();
  agg_request.values = vals;
  agg_request.aggregations.push_back(cudf::make_sum_aggregation());
  auto gby = groupby::groupby{table_view{{struct_keys}}, null_policy::EXCLUDE, sorted::NO, {}, {}};
  auto result = gby.aggregate(requests);

  std::cout << "Actual output (grouped) struct keys: " << std::endl;
  print(result.first->view().column(0));
}

[mythrocks]

Here's the corresponding repro case for LIST input:

TEST_F(UtilitiesTest, groupby_lists_bork)
{
  using namespace cudf;
  using ints = fixed_width_column_wrapper<int32_t>;
  using int_lists = lists_column_wrapper<int32_t>;

  auto lists = int_lists{ {1,2,3}, {4,5,6}, {1,2,3}, {4,5,6}, {1,2,3}, {4,5,6} };
  auto vals  = ints     {      0,       1,       2,       3,       4,       5};

  auto expect_lists_keys       = int_lists{{1, 2, 3}, {4,5,6}};
  ints expect_vals{9, 19, 17};

  std::cout << "Lists input to groupby (keys)!" << std::endl;
  print(lists);

  std::cout << "Expected output (grouped) lists keys!" << std::endl;
  print(expect_lists_keys);

  auto requests = std::vector<groupby::aggregation_request>{};
  requests.emplace_back(groupby::aggregation_request{});
  auto& agg_request  = requests.front();
  agg_request.values = vals;
  agg_request.aggregations.push_back(cudf::make_sum_aggregation());
  auto gby = groupby::groupby{table_view{{lists}}, null_policy::EXCLUDE, sorted::NO, {}, {}};
  auto result = gby.aggregate(requests);

  std::cout << "Actual output (grouped) lists keys: " << std::endl;
  print(result.first->view().column(0));
}

[mythrocks]

Update: groupby::aggregate() works with STRUCT grouping columns, as evidenced by the structs_tests.cpp added in #9024.

The remaining sticking point is grouping on LISTS.

[mythrocks]

The remaining sticking point is grouping on LISTS.

With a little bit of digging, one sees that groupby's sort-based aggregations fail on LISTS, with the following error:

unknown file: Failure
C++ exception with description "cuDF failure at: /home/mithunr/workspace/dev/cudf/2/cpp/include/cudf/table/row_operators.cuh:362: Attempted to compare elements of uncomparable types." thrown in the test body.

However, hash-based aggregations still pass silently, except in debug-builds:

/home/mithunr/workspace/dev/cudf/2/cpp/include/cudf/table/row_operators.cuh:447: unsigned int cudf:
:element_hasher_with_seed<hash_function, has_nulls>::operator()(cudf::column_device_view, signed in
t) const [with T = cudf::list_view; void *<anonymous> = (void *)nullptr; hash_function = default_ha
sh; __nv_bool has_nulls = false]: block: [0,0,0], thread: [0,0,0] Assertion `false && "Unsupported
type in hash."` failed.

It only fails when it attempts to construct a hash value for a LISTS row.

Rather than depend on the benevolence of flatten_nested_columns() or the hash functions throwing an unsupported exception, it would be preferable to have groupby bail early on LISTS input.

[mythrocks]

hash-based aggregations still pass silently

After #9024, flatten_nested_columns() does throw a visible error, preventing grouping by LIST columns. It would still be preferable to make the changes proposed in #9227:

1. There are tests now, for groupby aggregations with LIST keys.
2. The error is now explicitly a grouping failure.