Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support parsing for parquet writer option #4938

Merged
merged 5 commits into from
Oct 18, 2023

Conversation

fansehep
Copy link
Contributor

Which issue does this PR close?

part #4693

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

@github-actions github-actions bot added the parquet Changes to the parquet crate label Oct 16, 2023
Copy link
Contributor

@tustvold tustvold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we get some tests for this please 🙏

@fansehep
Copy link
Contributor Author

Could we get some tests for this please 🙏

🫣i will add some tests later.

Signed-off-by: fan <[email protected]>
@fansehep fansehep requested a review from tustvold October 17, 2023 10:53
Copy link
Contributor

@tustvold tustvold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly just some minor test nits, but I also wonder if we really want to be case insensitive?

I think it probably also makes sense to derive Display if we are going to define FromStr, but happy for this to be a separate PR.

Comment on lines +2265 to +2272
match "plain_xxx".parse::<Encoding>() {
Ok(e) => {
panic!("Should not be able to parse {:?}", e);
}
Err(e) => {
assert_eq!(e.to_string(), "Parquet error: unknown encoding: plain_xxx");
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
match "plain_xxx".parse::<Encoding>() {
Ok(e) => {
panic!("Should not be able to parse {:?}", e);
}
Err(e) => {
assert_eq!(e.to_string(), "Parquet error: unknown encoding: plain_xxx");
}
}
let err = "plain_xxx".parse::<Encoding>().unwrap_err();
assert_eq!(e.to_string(), "Parquet error: unknown encoding: plain_xxx");

assert_eq!(compress, Compression::LZ4);

// test unknown compression
match "unknown".parse::<Compression>() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These could also make use of unwrap_err()

type Err = ParquetError;

fn from_str(s: &str) -> Result<Self, Self::Err> {
match s.to_owned().to_uppercase().as_str() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should be case sensitive and leave it to the caller to force to upper case if that is what they want?

It seems strange that we would support parsing things like PlAiN

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems strange that we would support parsing things like PlAiN

For users, whose usage is generally unobservable, I also find it odd to support this kind of parsing like PlAiN, but it seems that both cases should be supported?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should just support case sensitive parsing and the user can opt in to case insensitive parsing by converting the input to uppercase should they want this behaviour

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should just support case sensitive parsing and the user can opt in to case insensitive parsing by converting the input to uppercase should they want this behaviour

done. i think it's a good suggestion.

@tustvold tustvold merged commit a94ccff into apache:master Oct 18, 2023
16 checks passed
@tustvold
Copy link
Contributor

Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
parquet Changes to the parquet crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants