[BUG] Function for Lead/Lag on Struct Type with default values does not work as expected.

[wbo4958]

Describe the bug

Function for Lead/Lag on Struct Type with default values does not work as expected.

Steps/Code to reproduce bug

I didn't dive into libcudf to check how to reproduce it, what I can do is copying java unit tests here for reproducing. You needs to copy the below code into TableTest.java and run the testWindowingLeadOnStruct test. This issue can be reproduced in the newest commit in branch-21.06

 @Test
  void testWindowingLeadOnStruct() {
    try (Table unsorted = new Table.TestBuilder()
        .column(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) // GBY Key
        .column(1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3) // GBY Key
        .column(1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6) // OBY Key
        .column(new StructType(true,
                new BasicType(true, DType.INT32),
                new BasicType(true, DType.STRING)),
            new StructData(1, "s1"), new StructData(null, "s2"), new StructData(2, null), new StructData(3, "s3"),
            new StructData(11, "s11"), null, new StructData(13, "s13"), new StructData(14, "s14"),
            new StructData(111, "s111"), new StructData(null, "s112"), new StructData(2, "s222"), new StructData(3, "s333")) //STRUCT Agg COLUMN
        .build()) {

      try (Table sorted = unsorted.orderBy(OrderByArg.asc(0), OrderByArg.asc(1), OrderByArg.asc(2))) {
        WindowOptions.Builder windowBuilder = WindowOptions.builder().minPeriods(1);
        try (Scalar zero = Scalar.fromInt(0);
             Scalar one = Scalar.fromInt(1);
             WindowOptions options = windowBuilder.window(zero, one).build();
             ColumnVector structDefaultOutput = ColumnVector.fromStructs(
                 new StructType(true,
                     new BasicType(true, DType.INT32),
                     new BasicType(true, DType.STRING)),
                 new StructData(-1, "s1"), new StructData(null, "s2"), new StructData(-2, null), new StructData(-3, "s3"),
                 new StructData(-11, "s11"), null, new StructData(-13, "s13"), new StructData(-14, "s14"),
                 new StructData(-111, "s111"), new StructData(null, "s112"), new StructData(-222, "s222"), new StructData(-333, "s333"));

             Table structWindowAggResults = sorted.groupBy(0, 1).aggregateWindows(
                 Aggregation
                     .lead(1, structDefaultOutput)
                     .onColumn(3) //STRUCT Agg COLUMN
                     .overWindow(options));
             ColumnVector structExpectAggResult = ColumnVector.fromStructs(
                 new StructType(true,
                     new BasicType(true, DType.INT32),
                     new BasicType(true, DType.STRING)),
                 new StructData(null, "s2"), new StructData(2, null), new StructData(3, "s3"), new StructData(-3, "s3"),
                 null, new StructData(13, "s13"), new StructData(14, "s14"), new StructData(-14, "s14"),
                 new StructData(null, "s112"), new StructData(2, "s222"), new StructData(3, "s333"), new StructData(-333, "s333"))) {
          // TODO  this is not gonna work, since libcudf has some issue for lead on struct with default values
           assertColumnsAreEqual(structExpectAggResult, structWindowAggResults.getColumn(0));
        }
      }
    }
  }

Expected behavior
The unit test should pass.

[wbo4958]

@mythrocks pls help to check it

[mythrocks]

Thanks for reporting this, @wbo4958. Looks like I don't have a libcudf test for the Struct case with defaults. Sorry I missed it.

[beckernick]

@wbo4958 @mythrocks was this resolved by #8507 ?

[wbo4958]

Let me check today. Thx

[beckernick]

Thanks Bobby!

[wbo4958]

It worked. thx. close this issue