Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify ObjectStore configuration pattern #4189

Merged
merged 1 commit into from
May 10, 2023

Conversation

tustvold
Copy link
Contributor

@tustvold tustvold commented May 9, 2023

Which issue does this PR close?

Relates to #4047

Rationale for this change

Originally added in #3436 by @roeap, the API can be made simpler and easier to use

What changes are included in this PR?

Are there any user-facing changes?

This makes the config enumerations non exhaustive, so that options can be added without it being a breaking change

@tustvold tustvold added the api-change Changes to the arrow API label May 9, 2023
@@ -742,18 +727,28 @@ impl AmazonS3Builder {
AmazonS3ConfigKey::UnsignedPayload => {
self.unsigned_payload = str_is_truthy(&value.into())
}
AmazonS3ConfigKey::Checksum => {
let algorithm = Checksum::try_from(&value.into())
.map_err(|_| Error::InvalidChecksumAlgorithm)?;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was problematic as this error would get unwrapped in from_env

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fixed properly in #4192

@github-actions github-actions bot added the object-store Object Store Interface label May 9, 2023
@tustvold tustvold marked this pull request as draft May 10, 2023 10:04
@tustvold tustvold marked this pull request as ready for review May 10, 2023 11:13
@tustvold
Copy link
Contributor Author

@roeap perhaps you might be able to give this a look?

Copy link
Contributor

@roeap roeap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great - looking back, making the config fallible was a big regret :D.

Really looking forward to the deferred parsing as well, which would allow us to pass in impl AsRef<str>'s again? Although that is just a slight convenience upgrade.

@tustvold tustvold merged commit b3a9981 into apache:master May 10, 2023
@tustvold
Copy link
Contributor Author

Thank you for the prompt review 💪

Really looking forward to the deferred parsing as well

#4192 should be ready for review now, it currently only defers the parsing of values, and so the key arguments are still typed. I personally think this is more idiomatic, and allows users to choose how to handle unexpected keys, but appreciate opinions may differ.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api-change Changes to the arrow API object-store Object Store Interface
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants