ppc64le libvirt ci job #8327
ppc64le libvirt ci job #8327
Conversation
Signed-off-by: Christy Norman <[email protected]>
openshift-ci-robot
added
the
do-not-merge/work-in-progress
openshift-ci-robot
removed
the
do-not-merge/work-in-progress
|
/lgtm |
|
/cc @jaypoulz |
These tests contains the tcp connection to google.com and seen connection drops to google.com, hence skipping them
|
/lgtm |
|
/test pj-rehearse |
|
/lgtm |
| @@ -550,6 +550,8 @@ objects: | |||
|
|
|||
| wait "${openshift_install}" | |||
|
|
|||
| mock-nss.sh openshift-install --dir=/tmp/artifacts/installer --log-level=debug wait-for install-complete 2>&1 & | |||
There was a problem hiding this comment.
Isn't this already covered by the wait "${openshift_install}" above?
There was a problem hiding this comment.
Like all commits, a description for why this is needed should be included in the commit message.
There was a problem hiding this comment.
The reason for adding this is - install is taking a little longer than the default timeout(30mins), so added this wait condition to add that extra wait time for all the cluster operators to come online.
There was a problem hiding this comment.
What we are finding is that the 30 minute timeout in the installer is not sufficient in our CI environment for ppc64le. We are running just over or into around 35 minutes. So this is an additional wait.
There was a problem hiding this comment.
The 30 timeout missing the install completion by a few minutes is also something we've run into ~20% of recent nightly builds on s390x. We should definitely include an explanation in the commit description though.
There was a problem hiding this comment.
If a timeout increase is needed, that change belongs in the installer itself; not CI. Our CI doesn't do us much good if it isn't reflective of what our customers are doing. Can we quickly figure out why the increased timeout is necessary and then figure out where the fix belongs?
There was a problem hiding this comment.
From what I can tell, this wouldn't be fixed quickly. The discussion @mkumatag linked to led me to another PR: #7568 -- which you obviously saw b/c you approved it (@crawford) :D -- but it's new to me, and I do see that it was an attempt to fix setup and teardown slowness:
https://github.com/openshift/release/pull/7568/files#diff-1c02c7c476c95d66377474c542fdfb33R274-R276 -- which gets us to the same thing, IIUC: https://github.com/ironcladlou/openshift4-libvirt-gcp/blob/rhel8/tools/create-cluster#L77
But it looks like that's only if libvirt is running in gcp? And this is on a system in our lab (meaning: not in GCP). And we don't have any specialized scripts. :/
We're okay with taking that wait out, but it would be nice if we could leave it in, and just open an issue with the installer in the meantime. I'll definitely update the commit message with more detail if you're okay with leaving this. LMK!
There was a problem hiding this comment.
Also, @manojnkumar said that "in theory" no customers should see this version of the installer. It's very much unsupported (dev only), just to make sure that's clear. That makes me okay with this approach.
There was a problem hiding this comment.
Oh right, the IPI flow is only going to be used in CI. Duh. Sorry everyone, my mind has been scrambled this week. 😞
This is fine as is, but let's not forget to investigate this further.
There was a problem hiding this comment.
Thanks @crawford! I think we can all say the same thing about our minds these past couple of weeks. :)
|
all passed again. looking better on wednesday. :) |
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: crawford, jaypoulz, manojnkumar, mkumatag The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci-robot
added
the
approved
|
@mkumatag: The following test failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
/test generated-config |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
@mkumatag: Updated the following 5 configmaps:
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Carry forward from #8110