-
Notifications
You must be signed in to change notification settings - Fork 177
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve run times when using key pair auth by caching the private key #1110
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Caching LGTM, with a couple of nits.
I'll let someone from Adapters provide a final review.
@@ -63,6 +72,58 @@ | |||
} | |||
|
|||
|
|||
@cache |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: can we move these to their own module? Just want to keep connections.py from getting bigger if we don't need to
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I actually want to move them into dbt-adapters
long term. I thought about creating a separate module, but thought it would be unnecessary if we were going to move it later. I can add that here though if you think it's worth it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's worth it as it should make it cleaner to move later
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I moved over only the two methods that are generic. I left the one that's Snowflake-specific (converting the key to DER format).
Along the same lines of making connections.py
smaller, I actually think the Credentials
class should be in its own module (probably named auth
, even though that's what I moved these methods to). That's a separate PR, but it shouldn't take long to do if we want to do that.
resolves #1082
Problem
We evaluate and validate the private key on every connection when using key pair auth despite the fact that the key does not change during a dbt run.
Solution
Credentials
class and pass the parameters to this functionfunctools.lru_cache
if we're using Python 3.8functools.cache
if we're using Python 3.9Checklist