[BUG] read_orc reads incorrect data on one row

[ayushdg]

Describe the bug
When reading data via cudf.read_orc one of the rows seems to have incorrect data.

Steps/Code to reproduce bug

import cudf

df = cudf.read_orc("row_bug.orc")
df.iloc[3874]

_col0     10100163
_col1         4125
_col2    637000238
Name: 3874, dtype: int32

Expected behavior

import pyarrow.orc as orc
pdf = orc.ORCFile("row_bug.orc").read().to_pandas()
pdf.iloc[3874]

_col0     10100163
_col1         4125
_col2    927000494
Name: 3874, dtype: int32

Value of _col2 at index location 3874 should be 927000494

Environment overview (please complete the following information)

• Method of cuDF install: Conda
  • 0.14 nightly on May 29

Additional context
Data here: row_bug.orc.zip

Thanks @trstovall for pointing out the issue.

cc: @OlivierNV

[OlivierNV]

From the expanded data in the slack thread, it looks like perhaps a corner-case in RLE patched-base or delta mode.

[OlivierNV]

I can reproduce this, extracting stripe #26 (only 331 rows), there is one stray value very different than the others (points at RLE):

[324] 347000041 (0x14aecce9)
[325] 347000011 (0x14aecccb)
[326] 927000463 (0x3740e78f)
[327] 347000002 (0x14aeccc2)
[328] 347000003 (0x14aeccc3)
[329] 347000036 (0x14aecce4)
[330] 347000041 (0x14aecce9)

[OlivierNV]

Weird, it seems to be the 2nd value of a 6-value RLEv2-mode1 sequence.
[Edit] Weirder: I'm seeing the value as 927000463, which appears to be the correct value (not sure if the bug description matches the file). Also tracked this all the way into snappy to exclude decompression errors, which also backreferences these 927000463 values from earlier values in the file (these appear periodically), which seems to be consistent.

[devavret]

Alrighty, we found the reason for the bug. Arrow ORC actually behaves different than their docs. 😡 In case of patched RLE, the size of the patch + patch gap is not actually equal to patch gap width + patch width as defined in the header. It's actually rounded up to some bit size defined here: https://github.com/apache/orc/blob/9c65bc116a2208d5d729687b7744b2ffbd72fea0/c%2B%2B/src/RLEV2Util.cc#L30-L37. This mapping doesn't seem to be in the documentation at all.

[harrism]

Good job investigating. File an Apache issue?