Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KubernetesPodOperator new callbacks and allow multiple callbacks #44357

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

johnhoran
Copy link


I would like to have multiple callbacks in the kubernetes pod operator, and add two new callbacks. on_manifest_finalization would allow the callback to make changes just before the manifest is turned into a pod.
on_pod_wrapup would happen after the calls to on_pod_completion but just before the pod is deleted.

Adding both of these plus allowing multiple callbacks would allow you to do things in the kubernetes pod operator akin to how it approaches XComs but in a modular way. My use case here is I'm running DBT in kpo, and I want to do multiple things to the DBT artefacts after the DBT job has run, I could use on_manifest_finalization to insert an alpine sidecar with volumes mounted with the same intention as the XCom sidecar, where the sidecar keeps the volumes alive as files are extracted from it. on_pod_wrapup would allow me to insert a single sidecar and have multiple callbacks run their on_pod_completion before on_pod_wrapup kills the sidecar.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers provider:cncf-kubernetes Kubernetes provider related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant