[BUG] gathering structs can produce valid child values underneath null struct parent

[jlowe]

Describe the bug
Performing a gather on a non-nullable struct column with non-nullable child columns can produce a result where the parent has null rows but the corresponding child rows are not null.

Steps/Code to reproduce bug
Create a non-nullable struct column with non-nullable child columns and perform a gather with some of the gather map indices being out-of-bounds (i.e.: -1) and specify that out-of-bounds gather indices should result in null values during the gather.

Expected behavior
If the gather operation introduces a null struct row, the corresponding child rows should also be null.

[jlowe]

This appears to be fixed with the following patch. I noticed the logic at https://github.com/rapidsai/cudf/blob/branch-21.10/cpp/include/cudf/detail/gather.cuh#L663-L666 was not being applied in the struct column handling of child columns.

diff --git a/cpp/include/cudf/detail/gather.cuh b/cpp/include/cudf/detail/gather.cuh
index 1dd0d472d0..6b1e9b350c 100644
--- a/cpp/include/cudf/detail/gather.cuh
+++ b/cpp/include/cudf/detail/gather.cuh
@@ -470,7 +470,7 @@ struct column_gatherer_impl<struct_view> {
                                                                          mr);
                    });
 
-    auto const nullable = std::any_of(structs_column.child_begin(),
+    auto const nullable = nullify_out_of_bounds || std::any_of(structs_column.child_begin(),
                                       structs_column.child_end(),
                                       [](auto const& col) { return col.nullable(); });
     if (nullable) {

I'm guessing this wasn't caught by the unit tests since they are using nullable child columns.

[ttnghia]

I noticed the logic at https://github.com/rapidsai/cudf/blob/branch-21.10/cpp/include/cudf/detail/gather.cuh#L663-L666 was not being applied in the struct column handling of child columns.

Yeah that is because when implemented it I used the assumption that the children columns have always been superimposed their parent nulls onto.

[jlowe]

that is because when implemented it I used the assumption that the children columns have always been superimposed their parent nulls onto.

In this case the input columns are correctly reflecting the parent nullability. Nothing is nullable in the input, parent struct or otherwise, yet it produces the wrong output. When nullify_out_of_bounds is enabled, nulls can appear in the output when none appear in the input.

[ttnghia]

Thanks Jason. That's definitely my bug. Thanks @mythrocks for taking over this. I'm sorry that it costs you guys time to deal with it 😞

[mythrocks]

That's definitely my bug.

It was a subtle bug. :/ I've been operating under the assumption that it was my doing. :] If my STRUCT tests were better, we would have caught this sooner. A fix is inbound.