[bug] Kubeflow workflow runs are in "Pending Execution" state #10901
Comments
sureshmol
changed the title
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
github-actions
bot
added
the
lifecycle/stale
|
This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it. |
|
@sureshmol Are you able to fix this issue? I am facing same. |
|
/reopen |
|
@imranrazakhan: You can't reopen an issue/PR unless you authored it or you are a collaborator. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
@sureshmoligi: You can't reopen an issue/PR unless you authored it or you are a collaborator. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
@imranrazakhan |
|
@sureshmol: Reopened this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
github-actions
bot
removed
the
lifecycle/stale
|
I also have this issue. Kubernetes version: 1.29 You can see the run finished successfully, but hangs in the "Pending Execution" stage: The same happens with all runs, even with the failed ones: |
|
I also have the same issue, how can this be debugged? |
|
This might be related: kubeflow/manifests#2850 After querying the .well-known issuer and setting that in the m2m requestauth (and restarting ml-persistence), the pipeline run statusses are updated correctly. However, the cron-job for refreshing the JWKS keys for the m2m issuer fails due to a faulty |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
github-actions
bot
added
the
lifecycle/stale
Environment
git clone https://github.com/kubeflow/manifests.git
cd manifests
while ! kustomize build example | kubectl apply -f -; do echo "Retrying to apply resources"; sleep 10; done
Kubeflow version: 1.8
Kubernetes version: 1.27
Steps to reproduce
After installation of kubeflow, All the pods are in running state:
I have created a pipeline from UI using a yaml file and pipeline is successfully created. See image below of the pipeline:
When i create a Run for this pipeline. Run are going into Pending Execution State and when i open the run, graphs are not loading yet. See image below:
But when I check for the workflow status using kubectl. Workflows are running started and completed successfully. Please see image below:
Expected result
I should be seeing the actual status of the workflows in the UI.
Can you please help me with this Bug, I am unable to figure out a solution for this
Impacted by this bug? Give it a 👍.
The text was updated successfully, but these errors were encountered: