Windows tests should not checkout main branch and merge the PR branch

[rm3l]

/kind bug
/area testing
/area Windows

What versions of software are you using?

Output of odo version:

odo v3.0.0-rc2 (f78096d26)

Server: https://10.10.10.105:6443
Kubernetes: v1.24.6

Actual behavior

Currently, Windows tests checkout the main branch and then merge the PR branch into the local main branch:

odo/.ibm/pipelines/windows-test-script.ps1

Lines 31 to 34 in f78096d

	Shout "Checkout to $GIT_PR_NUMBER"
	git fetch -v origin pull/${GIT_PR_NUMBER}/head:pr${GIT_PR_NUMBER}
	git checkout main
	git merge pr${GIT_PR_NUMBER} --no-edit

This can lead to potential issues for several reasons:

• it does not guarantee that the code tested is the one from the PR branch. At the time the test is run, if the PR branch is behind main, with main containing changes not conflicting with the PR branch, the merge operation would succeed. But this would give a local branch with changes coming from both main and the PR branch. I think this explains the behavior I observed and reported in check if namespace is created instead of project #6108 (comment) (where a given test spec had run, but the PR branch had no reference to that test spec) and which led me to file Display Git commit ID in output of odo commands where the version is shown #6131 (for better traceability).
• At the time the test is run, if there are conflicts between main and the PR branch, the whole Windows test would not pass.
• This might prevent from testing PRs targeting branches other than main, like release/v3.0.0.

To me, the test should run with what is exactly in the PR branch, and not attempt to merge anything (or if this intended, it should be a dedicated stage in the Pipeline). We already have the OpenShift CI bot reporting potential conflicts and requesting to rebase PRs.

Expected behavior

Just like what is done in Kubernetes and OpenShift tests, Windows tests should rely on the code already checked out by IBM Cloud.
We could scp the current folder to the Windows machine, as we do with the PowerShell script: https://github.com/redhat-developer/odo/blob/main/.ibm/pipelines/windows-test.sh#L13.
Or if the scp operation is too slow, the Windows test script could perform a shallow checkout of the right PR branch (rather than merging it into the local main branch).

[rm3l]

Fixed by #6177 (and merged into release/v3.0.0). This will get merged into main right after the v3.0.0 tag.

/close

[openshift-ci]

@rm3l: Closing this issue.

In response to this:

Fixed by #6177, which will get merged into main right after the v3.0.0 tag.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.