Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

schema(cdc): refactor schema snapshot to reduce memory usage #5144

Merged
merged 34 commits into from
Apr 26, 2022
Merged

schema(cdc): refactor schema snapshot to reduce memory usage #5144

merged 34 commits into from
Apr 26, 2022

Conversation

hicqu
Copy link
Contributor

@hicqu hicqu commented Apr 11, 2022 •
edited
Loading

What problem does this PR solve?

Issue Number: close #1386

What is changed and how it works?

Ticdc keeps all versions of schema snapshot for each DDL, and it can only GC them after its checkpoint is advanced. If there are too many DDLs, the checkpoint advancing can be very slow so that all versions of schema snapshot are kept in memory.

Without the PR schema snapshots are deep cloned from the previous one. With the patch all schema snapshots share one MVCC tree to avoid deep clone. This is why memory usage is reduced in the PR.

Check List

Tests

  • Unit test

Create 12K tables WITH THE PATCH:
图片

Create 12K tables WITHOUT THE PATCH:
图片

Code changes

Side effects

Related changes

Release note 

None

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Apr 11, 2022 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • asddongmen
  • liuzix

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Apr 11, 2022
@hicqu hicqu force-pushed the schema-snapshot-cow-1 branch from d31a705 to d436465 Compare April 12, 2022 07:15
@hicqu
fix a bug
3eea023 
Signed-off-by: qupeng <[email protected]>
@hicqu hicqu force-pushed the schema-snapshot-cow-1 branch from d436465 to 3eea023 Compare April 12, 2022 07:47
@hicqu
fix lint
1d4d0d9 
Signed-off-by: qupeng <[email protected]>
@3AceShowHand 3AceShowHand changed the title refactor schema snapshot to reduce memory usage schema(cdc): refactor schema snapshot to reduce memory usage Apr 12, 2022
@hicqu hicqu requested a review from asddongmen April 12, 2022 09:34
CharlesCheung96
CharlesCheung96 reviewed Apr 13, 2022
View reviewed changes
go.mod Outdated Show resolved Hide resolved
@hicqu
address comments
ac1f151 
Signed-off-by: qupeng <[email protected]>
@ti-chi-bot ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 25, 2022
@overvenus
Copy link
Member

/run-all-tests

overvenus
overvenus reviewed Apr 25, 2022
View reviewed changes
cdc/entry/schema/snapshot.go Outdated Show resolved Hide resolved
cdc/entry/schema/snapshot.go Outdated
id int64
tag uint64
// the associated entity pointer.
target unsafe.Pointer
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can it be interface{}?

liuzix
liuzix reviewed Apr 26, 2022
View reviewed changes
cdc/entry/schema/snapshot.go Show resolved Hide resolved
cdc/entry/schema/snapshot.go Outdated Show resolved Hide resolved
cdc/entry/schema/snapshot.go Outdated Show resolved Hide resolved
cdc/entry/schema/snapshot.go Show resolved Hide resolved
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Apr 26, 2022
@hicqu
Copy link
Contributor Author

hicqu commented Apr 26, 2022

/merge

@ti-chi-bot
Copy link
Member

@hicqu: /merge is only allowed for the committers, you can assign this pull request to the committer in list by filling /assign @committer in the comment to help merge this pull request.

In response to this:

/merge

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@hicqu
Copy link
Contributor Author

hicqu commented Apr 26, 2022

@overvenus please take a look again and /merge it if you are free.

@overvenus
Copy link
Member

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 826f9e4

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 26, 2022
@hicqu
Copy link
Contributor Author

hicqu commented Apr 26, 2022

/run-leak-test

@ti-chi-bot ti-chi-bot merged commit be428b7 into pingcap:master Apr 26, 2022
@hicqu hicqu deleted the schema-snapshot-cow-1 branch April 26, 2022 15:31
@nkorange nkorange mentioned this pull request Jul 18, 2022
Closed
@hicqu hicqu mentioned this pull request Dec 13, 2022
Merged
ti-chi-bot pushed a commit that referenced this pull request Dec 14, 2022
@hicqu
cdc: cherry pick #5144 into release-5.1 (#7894)
231a90b 
close #1386
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@overvenus overvenus overvenus left review comments

@3AceShowHand 3AceShowHand 3AceShowHand left review comments

@CharlesCheung96 CharlesCheung96 CharlesCheung96 left review comments

@nongfushanquan nongfushanquan nongfushanquan left review comments

@liuzix liuzix liuzix approved these changes

@asddongmen asddongmen asddongmen approved these changes

@maxshuang maxshuang Awaiting requested review from maxshuang

Assignees
No one assigned
Labels
release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Owner OOM when upstream TiDB has too many tables
9 participants
@hicqu @ti-chi-bot @codecov-commenter @overvenus @3AceShowHand @liuzix @asddongmen @CharlesCheung96 @nongfushanquan