Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Docs][Format] Clarify handling of Null values in REE encoding #33830

Closed
zagto opened this issue Jan 22, 2023 · 0 comments · Fixed by #33831
Closed

[Docs][Format] Clarify handling of Null values in REE encoding #33830

zagto opened this issue Jan 22, 2023 · 0 comments · Fixed by #33831
Assignees
@zagto
Labels
Component: Documentation Component: Format Type: bug
Milestone
12.0.0

Comments

@zagto
Copy link
Contributor

zagto commented Jan 22, 2023 •
edited
Loading

In the Columnar format, the handling of null values in the REE type is not described properly.

  • The null count of the parent array should be 0, since the parent has no validity bitmap (and there is a general assumption in arrow code that a non-zero null count means the standard validity bitmap is present)
  • The document only indirectly describes how Null values are encoded

Thanks @tustvold for catching this!

[1] #13333 (comment)

Component(s)

Documentation, Format
@zagto zagto changed the title Clarify handing of Null values in REE encoding Clarify handling of Null values in REE encoding Jan 22, 2023
@github-actions github-actions bot mentioned this issue Jan 22, 2023
Merged
zeroshade pushed a commit that referenced this issue Jan 23, 2023
@zagto @westonpace
GH-33830: Clarify handling of Null values in REE encoding (#33831)
b9d1162 
* Closes: #33830

Lead-authored-by: Tobias Zagorni <[email protected]>
Co-authored-by: zagto <[email protected]>
Co-authored-by: Weston Pace <[email protected]>
Signed-off-by: Matt Topol <[email protected]>
@zeroshade zeroshade added this to the 12.0.0 milestone Jan 23, 2023
@kou kou changed the title Clarify handling of Null values in REE encoding [Docs][Format] Clarify handling of Null values in REE encoding Jan 24, 2023
sjperkins pushed a commit to sjperkins/arrow that referenced this issue Feb 10, 2023
@zagto @westonpace @sjperkins
apacheGH-33830: Clarify handling of Null values in REE encoding (apac…
a4f053d 
…he#33831)

* Closes: apache#33830

Lead-authored-by: Tobias Zagorni <[email protected]>
Co-authored-by: zagto <[email protected]>
Co-authored-by: Weston Pace <[email protected]>
Signed-off-by: Matt Topol <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zagto zagto

Labels
Component: Documentation Component: Format Type: bug
Projects
None yet
Milestone
12.0.0
Development

Successfully merging a pull request may close this issue.

GH-33830: Clarify handling of Null values in REE encoding zagto/arrow
2 participants
@zeroshade @zagto