Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve arrow-ipc documentation #6983

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Jan 14, 2025

Which issue does this PR close?

Rationale for this change

The arrow-ipc docs are quite sparse and don't have many examples. Let's fix that with some documentation TLC

What changes are included in this PR?

  1. Add more docs and some basic examples
  2. Fix some typos

Are there any user-facing changes?

Just docs, no functional changes

@github-actions github-actions bot added the arrow Changes to the arrow crate label Jan 14, 2025
@tustvold
Copy link
Contributor

The Stream and File formats are actually different formats... See https://arrow.apache.org/docs/format/Columnar.html#ipc-file-format

@alamb
Copy link
Contributor Author

alamb commented Jan 14, 2025

The Stream and File formats are actually different formats... See https://arrow.apache.org/docs/format/Columnar.html#ipc-file-format

Yes, I am not sure if you are suggesting changes to this PR

https://arrow.apache.org/docs/format/Columnar.html#ipc-file-format says:

IPC File Format

We define a “file format” supporting random access that is an extension of the stream format.

@tustvold
Copy link
Contributor

I found the emphasis on the requirement or not for Seek as a way to distinguish the two to be a little backwards. They're different formats, with one allowing random access, the requirement for Seek to be honest is really an implementation detail. FileDecoder decodes the file format, and does not require seek.

@alamb
Copy link
Contributor Author

alamb commented Jan 15, 2025

I found the emphasis on the requirement or not for Seek as a way to distinguish the two to be a little backwards. They're different formats, with one allowing random access, the requirement for Seek to be honest is really an implementation detail. FileDecoder decodes the file format, and does not require seek.

This is a good point. I tried to clarify in a335701

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants