-
Notifications
You must be signed in to change notification settings - Fork 502
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tidb graceful upgrade #112
Conversation
15a3eaa
to
a7de64c
Compare
pkg/controller/tidb_control.go
Outdated
// TiDBControlInterface is the interface that knows how to manage tidb peers | ||
type TiDBControlInterface interface { | ||
// GetHealth returns tidb's health info | ||
GetHealth(tc *v1alpha1.TidbCluster) map[string]bool | ||
// ResignDDLOwner resign the ddl owner of tidb |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
resigns
pkg/controller/tidb_control.go
Outdated
ns := tc.GetNamespace() | ||
|
||
hostName := fmt.Sprintf("%s-%d", TiDBMemberName(tcName), ordinal) | ||
url := fmt.Sprintf("http://%s.%s-tidb-peer.%s:10080/ddl/owner/resign", hostName, tcName, ns) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
url := fmt.Sprintf("http://%s.%s.%s:10080/ddl/owner/resign", hostName, TiDBPeerMemberName(tcName), ns)
pkg/manager/member/tidb_upgrader.go
Outdated
upgradeOrdinal := tc.Status.TiDB.StatefulSet.CurrentReplicas - 1 | ||
if member, exist := tc.Status.TiDB.Members[tidbPodName(tc, upgradeOrdinal)]; exist && member.Health { | ||
err := tdu.tidbControl.ResignDDLOwner(tc, upgradeOrdinal) | ||
if err != nil && tc.Status.TiDB.ResignDDLOwnerFailCount < MaxResignDDLOwnerCount { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If err != nil && ResignDDLOwnerFailCount >= ResignDDLOwnerFailCount
then
this method will return nil.
This is the expected behavior?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Until the tidb resign ddl owner fail count is more then MaxResignDDLOwnerCount , we can think the tidb is unavailable, then upgrade it; otherwise, we must continue to try
pkg/manager/member/tidb_upgrader.go
Outdated
|
||
upgradeOrdinal := tc.Status.TiDB.StatefulSet.CurrentReplicas - 1 | ||
if member, exist := tc.Status.TiDB.Members[tidbPodName(tc, upgradeOrdinal)]; exist && member.Health { | ||
err := tdu.tidbControl.ResignDDLOwner(tc, upgradeOrdinal) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the TiDB replicas is 1, you do not need to resign.
/run-e2e-tests |
return controller.RequeueErrorf("tidbcluster: [%s/%s]'s tidb doesn't have old version pod to upgrade", ns, tcName) | ||
} | ||
|
||
if !tc.TiDBAllPodsStarted() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to wait all TiDB ready to upgrade TiDB?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
/run-e2e-tests |
* tidb graceful upgrade
this PR add the tidb graceful upgrade feature and fixes #89
it mainly logic is to resign ddl owner before upgrade a tidb instance