Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tidb graceful upgrade #112

Merged
merged 7 commits into from
Oct 12, 2018
Merged

Conversation

xiaojingchen
Copy link
Contributor

this PR add the tidb graceful upgrade feature and fixes #89
it mainly logic is to resign ddl owner before upgrade a tidb instance

@xiaojingchen xiaojingchen force-pushed the tidb-graceful-upgrade branch from 15a3eaa to a7de64c Compare October 10, 2018 06:46
// TiDBControlInterface is the interface that knows how to manage tidb peers
type TiDBControlInterface interface {
// GetHealth returns tidb's health info
GetHealth(tc *v1alpha1.TidbCluster) map[string]bool
// ResignDDLOwner resign the ddl owner of tidb
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resigns

ns := tc.GetNamespace()

hostName := fmt.Sprintf("%s-%d", TiDBMemberName(tcName), ordinal)
url := fmt.Sprintf("http://%s.%s-tidb-peer.%s:10080/ddl/owner/resign", hostName, tcName, ns)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

url := fmt.Sprintf("http://%s.%s.%s:10080/ddl/owner/resign", hostName, TiDBPeerMemberName(tcName), ns)

upgradeOrdinal := tc.Status.TiDB.StatefulSet.CurrentReplicas - 1
if member, exist := tc.Status.TiDB.Members[tidbPodName(tc, upgradeOrdinal)]; exist && member.Health {
err := tdu.tidbControl.ResignDDLOwner(tc, upgradeOrdinal)
if err != nil && tc.Status.TiDB.ResignDDLOwnerFailCount < MaxResignDDLOwnerCount {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If err != nil && ResignDDLOwnerFailCount >= ResignDDLOwnerFailCount
then
this method will return nil.

This is the expected behavior?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Until the tidb resign ddl owner fail count is more then MaxResignDDLOwnerCount , we can think the tidb is unavailable, then upgrade it; otherwise, we must continue to try


upgradeOrdinal := tc.Status.TiDB.StatefulSet.CurrentReplicas - 1
if member, exist := tc.Status.TiDB.Members[tidbPodName(tc, upgradeOrdinal)]; exist && member.Health {
err := tdu.tidbControl.ResignDDLOwner(tc, upgradeOrdinal)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the TiDB replicas is 1, you do not need to resign.

@xiaojingchen xiaojingchen changed the title [WIP] tidb graceful upgrade tidb graceful upgrade Oct 11, 2018
@xiaojingchen
Copy link
Contributor Author

/run-e2e-tests

return controller.RequeueErrorf("tidbcluster: [%s/%s]'s tidb doesn't have old version pod to upgrade", ns, tcName)
}

if !tc.TiDBAllPodsStarted() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to wait all TiDB ready to upgrade TiDB?

Copy link
Member

@tennix tennix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xiaojingchen
Copy link
Contributor Author

/run-e2e-tests

@weekface weekface merged commit 2189d69 into pingcap:master Oct 12, 2018
queenliuxx pushed a commit to queenliuxx/tidb-operator that referenced this pull request Dec 19, 2018
yahonda pushed a commit that referenced this pull request Dec 27, 2021
fgksgf pushed a commit to fgksgf/tidb-operator that referenced this pull request Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Graceful upgrade TiDB pod
3 participants