Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Provide PD self-protection mechanism #4373

Closed
6 of 7 tasks
rleungx opened this issue Nov 24, 2021 · 1 comment · Fixed by #8645
Closed
6 of 7 tasks

Provide PD self-protection mechanism #4373

rleungx opened this issue Nov 24, 2021 · 1 comment · Fixed by #8645
Assignees
@CabinfeverB
Labels
type/feature-request Categorizes issue or PR as related to a new feature.

Comments

@rleungx
Copy link
Member

rleungx commented Nov 24, 2021 •
edited
Loading

Background

We have met many cases which crush down the PD leader or make the PD leader overloaded. Under these circumstances, PD cannot provide important services such as TSO. It has a large impact on high availability and performance.

In the current implementation, PD doesn't have any self-protection mechanism. In other words, any component which accesses etcd directly or PD API may lead to the above things happening, like TiDB, TiFlash, TiCDC, etc. Even we manually request to PD through the raw HTTP API or pd-ctl may cause PD unavailable.

Development task

Some optional tasks:

Testing

TODO
@rleungx rleungx added the type/feature-request Categorizes issue or PR as related to a new feature. label Nov 24, 2021
@rleungx rleungx pinned this issue Nov 24, 2021
@CabinfeverB
Copy link
Member

/assign

CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 10, 2021
@CabinfeverB
add Self Protection Middleware Framework(tikv#4373)
3761aae 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 10, 2021
@CabinfeverB
add Self Protection Middleware Framework(tikv#4373)
3c20341 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 10, 2021
@CabinfeverB
add Self Protection Middleware Framework(tikv#4373) and fix some typo…
89d2eb5 
… in last commit

Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 10, 2021
@CabinfeverB
adjust Self Protection Middleware Framework(tikv#4373)
1e7d7c7 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 15, 2021
@CabinfeverB
Fix tikv#4373 : add rate limiter
3713e62 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 15, 2021
@CabinfeverB
Fix tikv#4373 : add rate limiter
e55b021 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 15, 2021
@CabinfeverB
Fix tikv#4373 : add audit logger
4b37f31 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 15, 2021
@CabinfeverB
Fix tikv#4373 : add config
7bd6353 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 15, 2021
@CabinfeverB
Fix tikv#4373 : change HTTP service source
16bbac9 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 15, 2021
@CabinfeverB
Fix tikv#4373 : change HTTP service source
fdcee1e 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 15, 2021
@CabinfeverB
Fix tikv#4373 : fix some lint problem
4461424 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 15, 2021
@CabinfeverB
Fix tikv#4373 : delete code for review
07b7fed 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 15, 2021
@CabinfeverB
Fix tikv#4373 : delete code for review
e4f30a8 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 15, 2021
@CabinfeverB
Fix tikv#4373 : delete code for review
4a33c9e 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 15, 2021
@CabinfeverB
Fix tikv#4373 : delete code for review
e7ae2bf 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 15, 2021
@CabinfeverB
Fix tikv#4373 : add middleware in server and add pdctl component
098acb7 
Signed-off-by: Cabinfever_B <[email protected]>
@CabinfeverB CabinfeverB mentioned this issue Dec 15, 2021
Closed
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 16, 2021
@CabinfeverB
Fix tikv#4373 : fix defailt config
49d4ec2 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 20, 2021
@CabinfeverB
Fix tikv#4373 : add comment
1c186dd 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 20, 2021
@CabinfeverB
Fix tikv#4373 : add comment
2a10ed3 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 20, 2021
@CabinfeverB
Fix tikv#4373 : add comment
fb3bf37 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 20, 2021
@CabinfeverB
Fix tikv#4373 : add comment
112aac3 
Signed-off-by: Cabinfever_B <[email protected]>
@CabinfeverB CabinfeverB mentioned this issue Dec 20, 2021
Closed
18 tasks
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 21, 2021
@CabinfeverB
close tikv#4373 : add framework
01fb757 
Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 22, 2021
@CabinfeverB
Revert "close tikv#4373 : add framework"
45b95d8 
This reverts commit 01fb757.

Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 28, 2021
@CabinfeverB
Revert "close tikv#4373 : add framework"
b899fa2 
This reverts commit 01fb757.

Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 28, 2021
@CabinfeverB
Revert "close tikv#4373 : add framework"
3647df8 
This reverts commit 01fb757.

Signed-off-by: Cabinfever_B <[email protected]>
CabinfeverB added a commit to CabinfeverB/pd that referenced this issue Dec 28, 2021
@CabinfeverB
Revert "close tikv#4373 : add framework"
f4b62fe 
This reverts commit 01fb757.

Signed-off-by: Cabinfever_B <[email protected]>
@rleungx rleungx reopened this Jan 4, 2022
@rleungx rleungx unpinned this issue Jul 4, 2022
@CabinfeverB CabinfeverB mentioned this issue Sep 27, 2023
Open
12 tasks
This was referenced Sep 23, 2024
Merged
Merged
@ti-chi-bot ti-chi-bot bot closed this as completed in 76dc560 Sep 26, 2024
ti-chi-bot bot added a commit that referenced this issue Sep 30, 2024
@rleungx @ti-chi-bot
pkg: delete config when QPS and concurrency are both deleted (#8653)
76bc34a 
ref #4373

Signed-off-by: Ryan Leung <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@rleungx rleungx mentioned this issue Jan 14, 2025
Merged
ti-chi-bot bot pushed a commit that referenced this issue Jan 15, 2025
@rleungx
server: use rateLimitCheck to reduce code (#8995)
dd8df1a 
ref #4373

Signed-off-by: Ryan Leung <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@CabinfeverB CabinfeverB

Labels
type/feature-request Categorizes issue or PR as related to a new feature.
Projects
None yet
Milestone
No milestone
2 participants
@CabinfeverB @rleungx