Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Integration]: Add nanoarrow to IPC integration tests #43680

Closed
bkietz opened this issue Aug 13, 2024 · 1 comment
Closed

[Integration]: Add nanoarrow to IPC integration tests #43680

bkietz opened this issue Aug 13, 2024 · 1 comment
Assignees
@bkietz
Labels
Component: Integration Type: enhancement
Milestone
18.0.0

Comments

@bkietz
Copy link
Member

bkietz commented Aug 13, 2024

Describe the enhancement requested

Nanoarrow can read and write IPC streams and therefore can read IPC files and write them with a little extra flatbuffers manipulation. While these APIs are being made public, it'd be helpful to have the extra test coverage of the integration tests.

Component(s)

Integration
@bkietz bkietz self-assigned this Aug 13, 2024
@bkietz bkietz mentioned this issue Aug 13, 2024
Merged
bkietz added a commit to apache/arrow-nanoarrow that referenced this issue Aug 15, 2024
@bkietz
feat: Add IPC integration test executable (#585)
2c42268 
- Adds a new executable for use with the archery integration tests
  - When reading files, it ignores the Footer entirely
- A few private workarounds are added, most notably the stream writer
tracks Blocks in order to populate the Footer
- Tested against patched archery
apache/arrow@main...bkietz:arrow:nanoarrow-integration-tests
- issue to add nanoarrow to the tests
apache/arrow#43680
@github-actions github-actions bot mentioned this issue Aug 15, 2024
Merged
bkietz added a commit that referenced this issue Sep 24, 2024
@bkietz @pitrou
GH-43680: [Integration] Unskip nanoarrow in IPC integration tests (#4…
242f98c 
…3715)

### Rationale for this change

Nanoarrow can now read and write IPC files as of apache/arrow-nanoarrow#585 so it should no longer be skipped as a producer/consumer

### What changes are included in this PR?

Nanoarrow's tester is updated to point to the new integration executable and to report nanoarrow as a consumer/producer of IPC files.

Notably the `null_trivial` case is skipped even though nanoarrow nominally supports it since it represents a corner case in which nanoarrow's flatbuffers library will not accept some vectors produced by other flatbuffers libraries dvidelabs/flatcc#287

### Are these changes tested?

Yes

### Are there any user-facing changes?

No

* GitHub Issue: #43680

Lead-authored-by: Benjamin Kietzman <[email protected]>
Co-authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Benjamin Kietzman <[email protected]>
@bkietz bkietz added this to the 18.0.0 milestone Sep 24, 2024
@bkietz
Copy link
Member Author

bkietz commented Sep 24, 2024

Issue resolved by pull request 43715
#43715

@bkietz bkietz closed this as completed Sep 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bkietz bkietz

Labels
Component: Integration Type: enhancement
Projects
None yet
Milestone
18.0.0
Development

No branches or pull requests
1 participant
@bkietz