[BUG] cuDF ORC reader incorrectly reads file written by pyarrow

[vuule]

Elements after the first row group are empty.

Only reproes with a string column.
Does not repro when the file is created by cuDF.

Minimal repro code:

expected = cudf.DataFrame({"str": ["*"] * 10001}, dtype="string")

buffer = BytesIO()
expected.to_pandas().to_orc(buffer)

got = cudf.read_orc(buffer)
assert_eq(expected, got)

[vuule]

Seems to be related to row group index; reading nonsensical offsets for the second row group (larger than the streams):

strm_offset0, strm_offset1: 272 10000
strm_len, strm_len2: 10001 80

Even if we switch the offsets around 272 does not make any sense. Trying to find whether the stream lengths are wrong or the offsets.

[vuule]

Turns out that the row index streams are read in incorrect order.
The values that we read from the pyarrow file are
byte offset, RLE run
272, 0
10000, 72

The correct values are
byte offset, RLE run
10000, 0
72, 272

With a hack to read the streams in inverse order, the file is read correctly.

[vuule]

Now, as to why the row index streams are read in wrong order:
libcudf ORC reader reader these streams in the same order as the order of corresponding data streams in the stripe footer.
When reading the pyarrow file these data streams' descriptors are read from the stripe footer as [LENGTH, DATA], while we read the row index streams in the [DATA, LENGTH] order. This difference in order causes the difference shown in the comment above.

[vuule]

So, why do we read these in inconsistent order? Either libcudf mixes up the order somehow, or the file stores this info in the incorrect order.
With some trace "logging" from the ORC reader, we can inspect the stripe footer and the row index, as they appear in the file (protobuf parsing always marches forward over the input, so the output order much match the input order).

Reading the same data written by cuDF with dictionary encode disabled (RI stands for row index):
StripeFooter begin
Stream begin
enum read: 6
Stream end
Stream begin
enum read: 6
Stream end
Stream begin
enum read: 1 <- DATA stream type
Stream end
Stream begin
enum read: 2 <- LENGTH stream type
Stream end
ColumnEncoding begin
enum read: 0
ColumnEncoding end
ColumnEncoding begin
enum read: 2
ColumnEncoding end
StripeFooter end
RI: 10 RI: 22
RI: 10 RI: 3 RI: 0 RI: 0 RI: 0
RI: 10 RI: 20
RI: 10 RI: 4 RI: 10000 RI: 80 RI: 0

Reading the repro file:
StripeFooter begin
Stream begin
enum read: 6
Stream end
Stream begin
enum read: 6
Stream end
Stream begin
enum read: 2 <- LENGTH stream type!!
Stream end
Stream begin
enum read: 1 <- DATA stream type!!
Stream end
ColumnEncoding begin
enum read: 0
ColumnEncoding end
ColumnEncoding begin
enum read: 2
ColumnEncoding end
StripeFooter end
RI: 10 RI: 24
RI: 10 RI: 3 RI: 0 RI: 0 RI: 0
RI: 10 RI: 23
RI: 10 RI: 5 RI: 10000 RI: 76 RI: 272 <- Same order as cuDF, DATA, then LENGTH

There are tiny differences in the data encode, but in both files the data stream is huge, while the length stream is tiny. The 10000 value should apply to the DATA stream in both cases, but the stripe footer order does not match the index streams.
TL;DR: It looks like the pyarrow file is invalid.

[kkraus14]

@vuule I believe this is unrelated to Arrow and is either a bug in the libORC C++ library or a bug in libcudf. Here's a reproducer without arrow:

import cudf
import pyorc

with open("test_pyorc.orc", "wb") as output:
   with pyorc.Writer(output, "struct<col0:string>", struct_repr=pyorc.StructRepr.DICT) as writer:
      for _ in range(10001):
         writer.write({"col0": "*"})
         
print(cudf.read_orc("test_pyorc.orc"))

[vuule]

Thanks @kkraus14 .I'll try to find what's going on in libORC C++.

Meanwhile, I looked into why we haven't caught the bug in unit tests until recently. Turns out we dodged the issue by chance (?) in multiple tests. With some small changes, I can repro the issue in different tests - the bug is not specific to string data in this or the customer issue.