-
Notifications
You must be signed in to change notification settings - Fork 9.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tests: unify --strict-reconfig-check
config of common framework
#14360
tests: unify --strict-reconfig-check
config of common framework
#14360
Conversation
af77669
to
1c84082
Compare
Codecov Report
@@ Coverage Diff @@
## main #14360 +/- ##
==========================================
- Coverage 75.26% 75.19% -0.08%
==========================================
Files 457 457
Lines 37140 37140
==========================================
- Hits 27953 27926 -27
- Misses 7428 7444 +16
- Partials 1759 1770 +11
Flags with carried forward coverage won't be shown. Click here to find out more.
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
Good catch and thanks for raising this discussion. Comments:
|
Thanks for your analysis! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, leaving for @ahrtr to approve and merge
@clarkfw Are you going to do this in this PR or in a separate PR? |
I'm going to do it in this PR, still working on it. |
f71afd0
to
d44bb5d
Compare
A new commit was force pushed. It enables The checks are expected to fail. The integration testing on my own computer shows that 15 test cases failed due to this commit. Most of them use I'm gonna push another commit to fix or discuss these failed test cases. Before that, I would like to know if changes in this commit are appropriate @ahrtr . |
func withDisableStrictReconfig() ctlOption { | ||
return func(cx *ctlCtx) { cx.disableStrictReconfigCheck = true } | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
func withDisableStrictReconfig() ctlOption { | |
return func(cx *ctlCtx) { cx.disableStrictReconfigCheck = true } | |
} | |
func withStrictReconfig() ctlOption { | |
return func(cx *ctlCtx) { cx.disableStrictReconfigCheck = true } | |
} |
@clarkfw yes, overall the change looks good to me. Please continue to finish the task. Thanks. I see that the e2e configuration is a bit mess. There are three ways to set the configuration.
I would suggest to remove all FYI. I just submitted a PR to cleanup this. #14389 |
+1 to config cleanup (not needed to be addressed in this PR). For me whole usage |
1eced78
to
e9a26a9
Compare
…ommon framework Signed-off-by: Clark <[email protected]>
e9a26a9
to
93a17b9
Compare
With |
There are still some check failures. I'm working on it. |
93a17b9
to
7f7ccd8
Compare
…onfig-check` by default Signed-off-by: Clark <[email protected]>
7f7ccd8
to
3f3149b
Compare
Thank God, all checks have passed. |
PeerTLS TLSConfig | ||
ClientTLS TLSConfig | ||
QuotaBackendBytes int64 | ||
DisableStrictReconfigCheck bool |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The name DisableStrictReconfigCheck
seems to be anti-pattern because users (test cases) need to use double negative to enable it, not disabled == enabled
. It's also a tradeoff, please see #14389 (comment) .
You just follow the same naming as the existing e2e configuration NoStrictReconfig
, so it isn't your fault. So we can keep it as it's for now, and feel free to change it to StrictReconfigCheck
so as to be consistent with the definition in the production code in a separate PR.
Overall looks good to me. I see that you disabled the StrictReconfigCheck for some cases, did you intentionally disable them to increase the test coverage or fix the test failures? |
Just to fix the test failures. After enabling I'm also working on another thread(#14281) that migrates |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Thank you @clarkfw
Purpose
This PR is opened to discuss the different policies of `--strict-reconfig-check' between e2e test and integration test.
What went wrong?
As described in #14281 (comment), the
MemberAdd
function works fine in integration test but encounters anunhealthy cluster
error in e2e test.Why?
Integration cluster has `--strict-reconfig-check' disabled by default:
etcd/tests/framework/integration/cluster.go
Line 723 in d05c894
in the meantime, e2e cluster has `--strict-reconfig-check' enabled by default:
etcd/tests/framework/e2e/cluster.go
Lines 313 to 315 in d05c894
therefore, in e2e cluster, functions like
MemberAdd
,MemberRemove
andTransferLeadership
would encounter anunhealthy cluster
error before membersisConnectedFullySince
, which will take 5 seconds:etcd/server/etcdserver/server.go
Lines 1350 to 1358 in d05c894
Solution
I've come up with some solutions. But I'm not sure if they have more hidden risks.
MemberAdd
function could wait 5 seconds before all membersisConnectedFullySince
. Apparently it'll make our code difficult to maintain.--strict-reconfig-check
to common framework, in order to give test cases control of whether to enable/disable--strict-reconfig-check
in the process of cluster creation.--strict-reconfig-check
both in e2e cluster and integration cluster.I tend to go with the 3rd solution. Though the 2nd solution may be required to do some specific tests in the future, we can skip it now.
What have I done?
I made some changes to disable
--strict-reconfig-check
in e2e cluster of common framework. At the same time, I kept it enabled as it was in the old framework, to avoid breaking existing test cases.