[BUG] AST produces non-null result when expression results in null scalar

[jlowe]

Describe the bug
When building an expression to copy a null literal, the resulting column from evaluating the expression contains all non-null values.

Steps/Code to reproduce bug
The following test code will reproduce the problem:

  auto c_0   = column_wrapper<int32_t>{0, 0, 0, 0};
  auto table = cudf::table_view{{c_0}};

  auto literal_value = cudf::numeric_scalar<int32_t>(-123);
  literal_value.set_valid_async(false);
  auto literal       = cudf::ast::literal(literal_value);

  auto expression = cudf::ast::expression(cudf::ast::ast_operator::IDENTITY, literal);

  auto result   = cudf::ast::compute_column(table, expression);
  auto expected = column_wrapper<int32_t>({-123, -123, -123, -123}, {0, 0, 0, 0});

  cudf::test::expect_columns_equal(expected, result->view(), true);

Expected behavior
When the expression evaluation for a row of a column results in a null scalar, the corresponding row of the output column should be null.

[vyasr]

Good catch, this is a bug in the current null handling that incorrectly assumes that nullability only requires special handling for column nodes, not literal nodes. I need to inject the corresponding logic for literals here. I'll tackle this as part of the ongoing refactor of conditional joins and see if it makes sense to make this part of the improved framework for handling nulls that we've discussed.

[vyasr]

This issue turned out to be a bit more involved than my previous comment. That comment was correct, but in addition the nullability of literals contained in an expression has to be considered when determined whether to even descend through the nullable code path. In particular, the change above is insufficient if a null literal is included in an expression for compute_column that is operating on a table that contains no nulls. It's not difficult to add that check, I'm currently just trying to figure out the best way to do so within the current design.