fix: only cleanup agent pod if exists

[tooptoop4]

Fixes #12659

[jswxstw]

The agent pod deletion logic is currently performed in markWorkflowPhase and the PodGC strategy is invalid to the agent pod, so I think it is more appropriate to put it into queuePodsForCleanup.

argo-workflows/workflow/controller/operator.go

Line 2291 in 5c8062e

func (woc *wfOperationCtx) markWorkflowPhase(ctx context.Context, phase wfv1.WorkflowPhase, message string) {

argo-workflows/workflow/controller/pod_cleanup.go

Line 13 in 5c8062e

func (woc *wfOperationCtx) queuePodsForCleanup() {

[jswxstw]

The agent pod deletion logic is currently performed in markWorkflowPhase and the PodGC strategy is invalid to the agent pod, so I think it is more appropriate to put it into queuePodsForCleanup.

argo-workflows/workflow/controller/operator.go

Line 2291 in 5c8062e

func (woc *wfOperationCtx) markWorkflowPhase(ctx context.Context, phase wfv1.WorkflowPhase, message string) {

argo-workflows/workflow/controller/pod_cleanup.go

Line 13 in 5c8062e

func (woc *wfOperationCtx) queuePodsForCleanup() {

One thing to note is that the agent pod cleanup policy depends on workflowPhase rather than nodePhase.

condition with podGC strategy and workflowPhase	pod cleanup action
(OnPodCompletion || OnPodSuccess || OnWorkflowCompletion) && workflowPhase.Completed()	deletePod
OnWorkflowSuccess && WorkflowSucceeded	deletePod
[Default]workflowPhase.Completed()	labelPodCompleted&&terminateContainers

[agilgur5]

Superseded by #13016. EDIT: Split to #13294