Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Fixes inferring project from service account and get credentials from google.auth.default #57

Merged
merged 12 commits into from
Sep 28, 2022

Conversation

ahuang11
Copy link
Contributor

@ahuang11 ahuang11 commented Sep 21, 2022

Summary

Relevant Issue(s)

I think calling google.auth.default will resolve PrefectHQ/prefect-dbt#56 so the profile.yml can be populated from a credentials object

Checklist

Copy link
Member

@desertaxle desertaxle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of these changes look good, but I don't think that we should add an infer_project attribute. I included my reasoning below, so let me know what you think.

@ahuang11 ahuang11 changed the title Add infer_project, fix inferring project from service account, and ex… Fixes inferring project from service account and get credentials from google.auth.default Sep 26, 2022
@ahuang11
Copy link
Contributor Author

Strange that the tests are failing from:

packages/prefect/testing/standard_test_suites/__init__.py:2: in <module>
    from .blocks import BlockStandardTestSuite
/opt/hostedtoolcache/Python/3.7.14/x64/lib/python3.7/site-packages/prefect/testing/standard_test_suites/blocks.py:7: in <module>
    from PIL import Image

But Prefect has pillow pinned:
https://github.com/PrefectHQ/prefect/blob/main/requirements-dev.txt#L14

Any thoughts?

@ahuang11
Copy link
Contributor Author

I think I may need to mock out google.auth.default()


    @pytest.fixture
    def gcp_credentials():
>       gcp_credentials_mock = GcpCredentials(project="gcp_credentials_project")

tests/conftest.py:154: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/hostedtoolcache/Python/3.7.14/x64/lib/python3.7/site-packages/prefect/blocks/core.py:176: in __init__
    self.block_initialization()
prefect_gcp/credentials.py:119: in block_initialization
    credentials = self.get_credentials_from_service_account()
prefect_gcp/credentials.py:139: in get_credentials_from_service_account
    credentials, _ = google.auth.default()

@ahuang11 ahuang11 requested a review from desertaxle September 27, 2022 18:22
@ahuang11
Copy link
Contributor Author

I just tried logging in through gcloud auth application-default login

It generates a file in:
~/.config/gcloud/application_default_credentials.json

Formatted as follows:

{
  "client_id": "id123",
  "client_secret": "secret123",
  "quota_project_id": "project_id",
  "refresh_token": "refresh_token",
  "type": "authorized_user"
}

So I actually need to search for both project_id and quota_project_id

@ahuang11 ahuang11 merged commit 068d6a0 into main Sep 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add the capability to use the Pod service account instead of using .gcp credential file
2 participants