Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[object_store,gcp] Document GoogleCloudStorage Default Credentials #5187

Closed
justinabrahms opened this issue Dec 7, 2023 · 4 comments · Fixed by #5188
Closed

[object_store,gcp] Document GoogleCloudStorage Default Credentials #5187

justinabrahms opened this issue Dec 7, 2023 · 4 comments · Fixed by #5188
Labels
documentation Improvements or additions to documentation enhancement Any new improvement worthy of a entry in the changelog good first issue Good for newcomers help wanted object-store Object Store Interface

Comments

@justinabrahms
Copy link
Contributor

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

I'd like to authenticate via object_store to GCP, but I don't want to have to pass around a json file of credentials.

Describe the solution you'd like

Ideally, I could use the service principal configured on my compute instance to issue requests. It can be fetched via https://cloud.google.com/compute/docs/metadata/predefined-metadata-keys#instance-metadata

As a less good, but also fine solution.. I'd like access to TokenProvider so that I can just issue the relevant API call myself.

Describe alternatives you've considered

I think I'll have to code my own CredentialProvider, but you already have a higher level abstraction I can't get access to.

Additional context

@justinabrahms justinabrahms added the enhancement Any new improvement worthy of a entry in the changelog label Dec 7, 2023
@tustvold
Copy link
Contributor

tustvold commented Dec 7, 2023

I believe this is the default behaviour if you don't configure any credential provider, is this not working for some reason?

@justinabrahms
Copy link
Contributor Author

We were using credential files for local testing, and there was an error when we went to deploy them. Looking through the docs, I couldn't see anything that mentioned getting creds from instance metadata.

Digging in the source, I finally came across https://github.com/apache/arrow-rs/blob/master/object_store/src/gcp/credential.rs#L322 but I think this needs to be a documentation improvement, not a code change.

This is where I was looking and hoping to find something useful: https://docs.rs/object_store/0.7.1/object_store/gcp/type.GcpCredentialProvider.html or https://docs.rs/object_store/0.7.1/object_store/gcp/struct.GoogleCloudStorage.html#method.credentials

@tustvold
Copy link
Contributor

tustvold commented Dec 7, 2023

but I think this needs to be a documentation improvement, not a code change.

I agree, this is not very clearly documented. I would definitely welcome a PR to improve this.

@tustvold tustvold added documentation Improvements or additions to documentation good first issue Good for newcomers help wanted labels Dec 7, 2023
@tustvold tustvold changed the title [object_store,gcp] Service-principal auth using machine-identity [object_store,gcp] Document GoogleCloudStorage Default Credentials Dec 23, 2023
@tustvold tustvold added the object-store Object Store Interface label Jan 5, 2024
@tustvold
Copy link
Contributor

tustvold commented Jan 5, 2024

label_issue.py automatically added labels {'object-store'} from #5188

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement Any new improvement worthy of a entry in the changelog good first issue Good for newcomers help wanted object-store Object Store Interface
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants