[CI] ok-to-test race condition

[VannTen]

Apparently, if an org member create a PR and include /ok-to-test in the initial message, failfast will occasionnaly create two gitlab pipelines, both considered latest, but one fails the 'ci-authorized' test and prevent the PR status to be successful:

See the example here : #11489
/assign @ant31
/kind flake

[VannTen]

/kind flake

[tico88612]

FYI: https://gitlab.com/kargo-ci/kubernetes-sigs-kubespray/-/jobs/7715052922

[VannTen]

Not sure how, but I managed to clear the stuck ci-not-authorized by restarting the whole pipeline from github by using "Re-run all checks" on the Check page of the PR (top right)

[ant31]

then doing a /retest would also work

[VannTen]

Ok, I hadn't realized this was integrated 👍 I spoke too soon, though. While a new pipeline was triggered and succeeded, a failing context still appears on the PR page (only in at the end of the "Conversation" pane, though, not in the "Checks") : "Kubespray CI/pipeline" -> which links to the old failed PR with the failed ci-not-authorzed. I'll try a /retest on the PR to see if it changes anything

[VannTen]

(Don't think it will though, @tico88612 tried that last week with no resolution)

[VannTen]

Yep, no changes :/

[tico88612]

Another PR, #11540

[ant31]

are the /retest /retest-failed not working or they are working but don't resolve the issue ?

[ant31]

On the next occurrence, ping me on slack without trying to retest/retry first. I'll check the logs

[VannTen]

Working, but failing to resolve the issue.

I'm not completely sure, but it's possible that the trigger was using /ok-to-test in the PR description right away.

[VannTen]

On the next occurrence, ping me on slack without trying to retest/retry first. I'll check the logs

Will do 👍

[ant31]

FYI: the Gitlab Sync job shows the logs of the sync between github / gitlab.
Those Logs are coming from the failfast-ci service directly.
e.g.
https://github.com/kubernetes-sigs/kubespray/pull/11540/checks?check_run_id=30415855924

one of the line is showing the variables sent to gitlab:

[INFO] Setting variables: {
'PR_LABELS': 'cncf-cla: yes,size/XS,ok-to-test,approved,release-note-none,dependencies', 
'EVENT': 'pull_request', 
'PR_ID': '11540',
 'SHA': '0f97c7a334c1811e4f7fdf6d83e3cd06ba3776e9', 
'SHA8': '0f97c7a3',
'FAILFASTCI_STATUS_API': 'https://ffci.kubespray.com/api/v1/github_status', 
'SOURCE_REF': 'dependabot/pip/ansible-9.10.0', 
'REF_NAME': 'dependabot/pip/ansible-9.10.0', 
'CI_REF': 'pr-11540-dependabot/pip/ansible-9.10.0', 
'GITHUB_REPO': 'kubernetes-sigs/kubespray'}