-
Notifications
You must be signed in to change notification settings - Fork 834
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
StructBuilder doesn't handle nulls correctly for empty structs #4842
Comments
The challenge is the null buffer is technically optional. What is your use-case for a StructArray containing no data? |
In this case I was replicating a protobuf structure, where proto allows empty message definitions:
and I was doing this in a reflection-like context where I was just examining descriptors and creating structs for them. In the end I caught this in my code and replaced it with a differently-typed column.
Understood. I think it'd also be sensible to either use this as a fallback with another unwrap_or_default layer, or just bin if you have a) no field columns and b) no null buffer. Alternately if empty structs are unsupported then not allowing you to construct a builder with no fields seems sensible. |
|
Describe the bug
StructBuilder
allows the creation of empty structs, but doesn't handle nulls correctly when callingfinish()
to get the array.To Reproduce
reproducing test:
Expected behavior
The test passes.
Additional context
I think this is because of this line in StructArray:
https://github.com/apache/arrow-rs/blob/master/arrow-array/src/array/struct_array.rs#L116
which uses
unwrap_or_default()
to get the length. A potential fix is theunwrap_or
with the null array length but I don't know the internals well enough to know if that's a correct fix.The text was updated successfully, but these errors were encountered: