-
Notifications
You must be signed in to change notification settings - Fork 60
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Disabling flaky test #429
Disabling flaky test #429
Conversation
Disabling the `testCancellingTaskThroughClientDoesNotCallCompletion` for now because it is flaky and failing inconsistently.
✅ Deploy Preview for eclectic-pie-88a2ba canceled.
|
✅ Deploy Preview for apollo-ios-docc canceled.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We currently have two ways of skipping tests:
- Using
XCTSkip()
. - Disabling the test in the test configuration.
This PR does 2 but I think we should switch it to using 1 instead. It looks like we use that method (1) for tests that should be skipped everywhere and the configuration-based method (2) when differentiating between tests that should be run on CI vs. local. XCTSkip()
is also easily searchable when we go back to resolve them.
Maybe we shouldn't be doing things this way; worth discussing?
Is the test flaky locally as well, or just on CI? If it's just flaky on CI, we should just disable it in the CI test config but not in the unit test config. |
So that's a good point. I haven't had it fail for me locally so I guess the intention of this PR is just to skip it for CI then - my bad. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should just disable the test in the Apollo-CITestPlan
config then.
git-subtree-dir: apollo-ios git-subtree-mainline: 304b53d git-subtree-split: 06c1a89127ba5655243a6a2cd014e418899cd5c7
Disabling the
testCancellingTaskThroughClientDoesNotCallCompletion
for now because it is flaky and failing inconsistently.