Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scheduler: reduce GetStore in hot-region-scheduler #3870

Merged
merged 4 commits into from
Jul 27, 2021

Conversation

HunDunDM
Copy link
Member

Signed-off-by: HunDunDM [email protected]

What problem does this PR solve?

What is changed and how it works?

Reduce GetStore in hot-region-scheduler. In addition to preventing panic caused by data inconsistency, it can also reduce the use of BasicCluster locks.

Check List

Tests

  • Unit test

Code changes

Side effects

Related changes

  • Need to cherry-pick to the release branch

Release note

Fix the bug that PD may panic during scaling out TiKV.

@HunDunDM HunDunDM added component/scheduler Scheduler logic. type/bugfix This PR fixes a bug. needs-cherry-pick-release-4.0 The PR needs to cherry pick to release-4.0 branch. needs-cherry-pick-release-5.0 The PR needs to cherry pick to release-5.0 branch. needs-cherry-pick-release-5.1 Type: Need cherry pick to release-5.1 labels Jul 14, 2021
@HunDunDM HunDunDM requested review from nolouch and lhy1024 July 14, 2021 02:22
@ti-chi-bot ti-chi-bot added the release-note Denotes a PR that will be considered when it comes time to generate release notes. label Jul 14, 2021
@codecov
Copy link

codecov bot commented Jul 14, 2021

Codecov Report

Merging #3870 (e703e79) into master (d02f0a9) will increase coverage by 0.03%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3870      +/-   ##
==========================================
+ Coverage   74.49%   74.52%   +0.03%     
==========================================
  Files         246      246              
  Lines       24846    24854       +8     
==========================================
+ Hits        18509    18523      +14     
+ Misses       4682     4676       -6     
  Partials     1655     1655              
Flag Coverage Δ
unittests 74.52% <100.00%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
server/schedulers/utils.go 93.06% <ø> (ø)
server/schedulers/hot_region.go 85.38% <100.00%> (+0.87%) ⬆️
server/schedulers/shuffle_hot_region.go 65.30% <100.00%> (+10.56%) ⬆️
server/tso/local_allocator.go 64.86% <0.00%> (-13.52%) ⬇️
pkg/etcdutil/etcdutil.go 82.35% <0.00%> (-5.89%) ⬇️
server/tso/tso.go 60.22% <0.00%> (-3.32%) ⬇️
server/server.go 72.23% <0.00%> (-0.92%) ⬇️
server/config/persist_options.go 91.50% <0.00%> (-0.78%) ⬇️
server/tso/allocator_manager.go 66.77% <0.00%> (-0.67%) ⬇️
server/member/member.go 64.70% <0.00%> (-0.54%) ⬇️
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d02f0a9...e703e79. Read the comment docs.

Signed-off-by: HunDunDM <[email protected]>
@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jul 14, 2021
@rleungx
Copy link
Member

rleungx commented Jul 14, 2021

Do we need unit tests?

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • lhy1024
  • nolouch

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jul 18, 2021
@nolouch
Copy link
Contributor

nolouch commented Jul 27, 2021

/merge

@ti-chi-bot
Copy link
Member

@nolouch: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: ff84f4e

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jul 27, 2021
@ti-chi-bot ti-chi-bot merged commit a2ea41c into tikv:master Jul 27, 2021
ti-chi-bot pushed a commit to ti-chi-bot/pd that referenced this pull request Jul 27, 2021
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #3909.

ti-chi-bot pushed a commit to ti-chi-bot/pd that referenced this pull request Jul 27, 2021
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #3910.

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #3911.

HunDunDM added a commit to HunDunDM/pd that referenced this pull request Jul 28, 2021
* scheduler: reduce GetStore in hot-region-scheduler

Signed-off-by: HunDunDM <[email protected]>

* tiny fix

Signed-off-by: HunDunDM <[email protected]>

Co-authored-by: Ti Chi Robot <[email protected]>
@HunDunDM HunDunDM deleted the reduce-getstore branch July 28, 2021 11:47
HunDunDM added a commit to ti-chi-bot/pd that referenced this pull request Aug 16, 2021
ti-chi-bot added a commit that referenced this pull request Aug 16, 2021
HunDunDM added a commit to ti-chi-bot/pd that referenced this pull request Sep 6, 2021
ti-chi-bot added a commit that referenced this pull request Sep 6, 2021
ti-chi-bot added a commit that referenced this pull request Sep 13, 2021
* scheduler: reduce GetStore in hot-region-scheduler

Signed-off-by: HunDunDM <[email protected]>

* tiny fix

Signed-off-by: HunDunDM <[email protected]>

Co-authored-by: HunDunDM <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/scheduler Scheduler logic. needs-cherry-pick-release-4.0 The PR needs to cherry pick to release-4.0 branch. needs-cherry-pick-release-5.0 The PR needs to cherry pick to release-5.0 branch. needs-cherry-pick-release-5.1 Type: Need cherry pick to release-5.1 release-note Denotes a PR that will be considered when it comes time to generate release notes. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/bugfix This PR fixes a bug.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

hot scheduler causes pd panic when scale out
5 participants