Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] Multiple instances of Google connectors cannot be used simultaneously with different credentials #1039

Closed
austinweisgrau opened this issue Apr 22, 2024 · 0 comments
Closed

[Bug] Multiple instances of Google connectors cannot be used simultaneously with different credentials #1039

austinweisgrau opened this issue Apr 22, 2024 · 0 comments
Labels
bug Impact - something is currently broken in Parsons and needs to be fixed

Comments

@austinweisgrau
Copy link
Collaborator

When multiple instances of Google connectors are initialized, each successive connector overwrites the same GOOGLE_APPLICATION_CREDENTIALS environment variable, thereby overwriting the authentication of the prior connectors.

Detailed Description

parsons.google.utitities.setup_google_application_credentials() takes a dictionary of application credentials as an argument and stores them generically in the GOOGLE_APPLICATION_CREDENTIALS environment variable. This environment variable is then read by instances of GCP clients rather than passed directly to them. This makes it impossible to initialize multiple connectors with different credentials in the same code.

To Reproduce

from utilities.google import GoogleBigQuery

first_client = GoogleBigQuery(app_creds=first_app_creds, project="FIRST_PROJECT")
second_client = GoogleBigQuery(app_creds=second_app_creds, project="SECOND_PROJECT")

first_client.query("select session_user()")
>> Forbidden: 403 POST https://bigquery.googleapis.com/bigquery/v2/projects/FIRST_PROJECT/jobs?prettyPrint=false: Access Denied: Project FIRST_PROJECT: User does not have bigquery.jobs.create permission in project FIRST_PROJECT.

Your Environment

  • Version of Parsons used (if you know):
  • Environment name and version (e.g. Chrome 39, node.js 5.4):
  • Operating System and version (desktop or mobile):

Additional Context

Add any other context about the problem here.

Priority

Please indicate whether fixing this bug is high, medium, or low priority for you. If the issue is time-sensitive for you, please let us know when you need it addressed by.
@austinweisgrau austinweisgrau added the bug Impact - something is currently broken in Parsons and needs to be fixed label Apr 22, 2024
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue Apr 22, 2024
@austinweisgrau
Pass credentials explicitly to GoogleBigQuery client
89a876a 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
@austinweisgrau austinweisgrau mentioned this issue Apr 22, 2024
Merged
9 tasks
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue Apr 22, 2024
@austinweisgrau
Pass credentials explicitly to GoogleBigQuery client
a1c5267 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue Apr 22, 2024
@austinweisgrau
Pass credentials explicitly to GoogleBigQuery client
4e42fac 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue Apr 22, 2024
@austinweisgrau
Pass credentials explicitly to GoogleBigQuery client
b69a7e1 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue Apr 22, 2024
@austinweisgrau
Pass credentials explicitly to GoogleBigQuery client
3b65834 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue Apr 22, 2024
@austinweisgrau
Pass credentials explicitly to Google clients
e90de92 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue Apr 22, 2024
@austinweisgrau
Pass credentials explicitly to Google clients
f3bfbd3 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue Apr 22, 2024
@austinweisgrau
Pass credentials explicitly to Google clients
6527c20 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue Apr 24, 2024
@austinweisgrau
Pass credentials explicitly to Google clients
92b3514 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue May 21, 2024
@austinweisgrau
Pass credentials explicitly to Google clients
27f58b2 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue May 21, 2024
@austinweisgrau
Pass credentials explicitly to Google clients
e280f3a 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue May 21, 2024
@austinweisgrau
Pass credentials explicitly to Google clients
6d5b6f2 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue May 23, 2024
@austinweisgrau
Pass credentials explicitly to Google clients
6fbc31e 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue May 23, 2024
@austinweisgrau
Pass credentials explicitly to Google clients
ff9415e 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue May 29, 2024
@austinweisgrau
Pass credentials explicitly to Google clients
c3814e8 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue Jun 3, 2024
@austinweisgrau
Pass credentials explicitly to Google clients
44c6757 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue Jun 21, 2024
@austinweisgrau
Pass credentials explicitly to Google clients
d1bcb9e 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue Jun 21, 2024
@austinweisgrau
Pass credentials explicitly to Google clients
7f3e7b3 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue Jun 21, 2024
@austinweisgrau
Pass credentials explicitly to Google clients
c8e4250 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue Jun 21, 2024
@austinweisgrau
Pass credentials explicitly to Google clients
f8bcf74 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue Jun 28, 2024
@austinweisgrau
Pass credentials explicitly to Google clients
87002a3 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit to austinweisgrau/parsons that referenced this issue Jul 11, 2024
@austinweisgrau
Pass credentials explicitly to Google clients
2f7d641 
Avoids issue documented in move-coop#1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.
austinweisgrau added a commit that referenced this issue Jul 12, 2024
@austinweisgrau
Pass credentials directly to GCP connectors rather than through envir…
6dfaaec 
…onment variable (#1040)

* Fix typo in filename utitities > utilities

* New method to pass credentials directly to GCP clients

* Pass credentials explicitly to Google clients

Avoids issue documented in #1039 where credentials for all GCP clients
are stored in the same environment variable, leading to overwrites if
multiple clients are initialized in the same environment.

* Mock credential parsing in tests

Avoids mock credentials needing to match Google Service Account
credential parsing

* Refactor Google Admin using new authed request session

* Fix tests on GoogleAdmin, expect new response structure

* Small changes to get BigQuery tests working
@austinweisgrau austinweisgrau mentioned this issue Jul 24, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Impact - something is currently broken in Parsons and needs to be fixed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@SorenSpicknall @austinweisgrau