Add KubernetesPodOperatorAsync
#28523
Conversation
boring-cyborg
bot
added
provider:cncf-kubernetes
phanikumv
force-pushed
the
kubernetes_async
branch
from
93cd391 to
d6e9eac
Compare
phanikumv
changed the title
KubernetesPodOperatorAsync
phanikumv
force-pushed
the
kubernetes_async
branch
from
af23ad3 to
5c88103
Compare
| await config.load_kube_config( | ||
| config_file=kubeconfig_path, | ||
| client_configuration=self.client_configuration, | ||
| context=cluster_context, | ||
| ) |
There was a problem hiding this comment.
Is this function really one that requires async? I imagine loading a kube config would be pretty quick?
There was a problem hiding this comment.
you're referring to async def _load_config(self) right?
well it does do file operations which are technically blocking (even though they might be fast)
so you might as well handle them asyncly ... just cus... why not?
but now that you mention it one thing that is a bit curious is ... how are we able to await await config.load_kube_config becaus it look like that is not an async method... i must be missing something
There was a problem hiding this comment.
yup, @phanikumv this config should come from from kubernetes_asyncio import config instead of from kubernetes import config like it is doing right now.
| "You can only use one option at a time." | ||
| ) | ||
| if in_cluster: | ||
| self.log.debug("loading kube_config from: in_cluster configuration") |
There was a problem hiding this comment.
May be place more descriptive comment here to describe what does actually in_cluster parameter mean?
| await config.load_kube_config( | ||
| config_file=kubeconfig_path, | ||
| client_configuration=self.client_configuration, | ||
| context=cluster_context, | ||
| ) |
There was a problem hiding this comment.
you're referring to async def _load_config(self) right?
well it does do file operations which are technically blocking (even though they might be fast)
so you might as well handle them asyncly ... just cus... why not?
but now that you mention it one thing that is a bit curious is ... how are we able to await await config.load_kube_config becaus it look like that is not an async method... i must be missing something
kaxil
force-pushed
the
kubernetes_async
branch
from
375bae2 to
2c78f81
Compare
kaxil
force-pushed
the
kubernetes_async
branch
from
0ed246e to
bbd2254
Compare
|
#28230 was created first, lets get that one merged and than rebase this on top of it to add more features |
phanikumv
force-pushed
the
kubernetes_async
branch
from
bbd2254 to
30f12a2
Compare
phanikumv
force-pushed
the
kubernetes_async
branch
2 times, most recently
from
36d7187 to
04e81e2
Compare
kaxil
force-pushed
the
kubernetes_async
branch
from
04e81e2 to
373cb18
Compare
I am working to rebase it and add more features |
phanikumv
force-pushed
the
kubernetes_async
branch
from
373cb18 to
7cf6740
Compare
phanikumv
force-pushed
the
kubernetes_async
branch
from
7cf6740 to
5ac8b9d
Compare
|
Closing as the additional feature on periodic fetching of logs has been implemented in #27758 |
This PR donates the
KubernetesPodOperatorAsyncfrom astronomer-providers repo to Airflow.Astronomer and its customers have been running this in production since last 6 months
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.