executor, ddl: support database placement option and partition placement option #28025

zhaoxugang · 2021-09-14T00:33:52Z

What problem does this PR solve?

Issue Number: refer to #26581

Depends on parser PR pingcap/parser#1342

This closes #27973 .

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

None

ti-chi-bot · 2021-09-14T00:33:53Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

mjonss
xhebox

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

zhaoxugang · 2021-09-14T16:14:31Z

/cc @xhebox @AilinKid

xhebox

Overall LGTM

ddl/db_test.go

ddl/ddl_api.go

ddl/placement_policy_test.go

ddl/ddl_api.go

mjonss

Oops, I missed this and did some duplicated work (but good for my own understanding) here:
#27969

mjonss · 2021-09-15T07:21:20Z

ddl/ddl_api.go

+	var opt *ast.CharsetOpt
+	if len(dbOps) != 0 {
+		opt = &ast.CharsetOpt{}
+		for _, val := range dbOps {


This looks like it is duplicated and would benefit from a common function, similar to this:
https://github.com/pingcap/tidb/pull/27969/files#diff-b7c54f7dad7257cbe4fa04b8d76c9bed694e2e2e7c5599f45eb2172d8d4ff8a7R2289

This looks like it is duplicated and would benefit from a common function, similar to this:
https://github.com/pingcap/tidb/pull/27969/files#diff-b7c54f7dad7257cbe4fa04b8d76c9bed694e2e2e7c5599f45eb2172d8d4ff8a7R2289

I will revoke the part of schema and use the common func that named SetDirectPlacementOpt to simple code after your pr be merged .

xhebox

Rest LGTM

ddl/ddl_api.go

xhebox

pingcap/parser#1342 has been merged, you could update the parser and remove replace directives.

zhaoxugang · 2021-09-22T15:51:03Z

/rebuild

zhaoxugang · 2021-09-22T23:31:47Z

/run-check_dev_2

mjonss

LGTM with one suggestion below.

mjonss · 2021-09-28T22:40:34Z

ddl/table.go

+		job.State = model.JobStateCancelled
+		return 0, errors.Trace(table.ErrUnknownPartition.GenWithStackByArgs("drop?", tblInfo.Name.O))
+	}
+	for idx, ptDef := range ptInfo.Definitions {


Looking at (*PartitionInfo) GetNameByID, it looks like this could be done similarly, including the check for existence above.
I.e. could you remove the if ptInfo.GetNameByID(partitionID) ... and replace the loop here with for i := range ptInfo.Definitions { and then check afterwards if the partition was found and if not return the error? Then you will only iterate over the array once, and according to the note in GetNameByID it is also faster.

ti-chi-bot · 2021-09-28T22:48:24Z

@mjonss: Thanks for your review. The bot only counts approvals from reviewers and higher roles in list, but you're still welcome to leave your comments.

In response to this:

LGTM with one suggestion below.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

morgo · 2021-09-28T22:52:10Z

LGTM also, but I will wait for @mjonss ' comment to be answered before approving/merging.

mjonss

LGTM

tisonkun · 2021-10-05T23:21:56Z

@mjonss & @xhebox does this PR need a release note? At least current release note is not a proper one. Please also take care of the release note during a review.

tisonkun · 2021-10-05T23:23:18Z

Also it closes a closed issue #26581, but why?

cc @morgo

xhebox · 2021-10-06T02:48:17Z

@mjonss & @xhebox does this PR need a release note? At least current release note is not a proper one. Please also take care of the release note during a review.

We could a release note in other PRs. There are too many PRs related to the feature.

Also it closes a closed issue #26581, but why?

I think it is a mistake by @zhaoxugang , he may want to say 'refer' or 'related'.

zhaoxugang · 2021-10-06T15:11:45Z

@xhebox @tisonkun

@mjonss & @xhebox does this PR need a release note? At least current release note is not a proper one. Please also take care of the release note during a review.

We could a release note in other PRs. There are too many PRs related to the feature.

Also it closes a closed issue #26581, but why?

I think it is a mistake by @zhaoxugang , he may want to say 'refer' or 'related'.

I have fixed it

tisonkun · 2021-10-06T23:26:33Z

@zhaoxugang thanks. I've updated the release note part to None for you. @xhebox please check whether we write the release note somewhere so that we don't miss this feature in 5.3.0 release.

@zhaoxugang please fix go.mod conflict.

/assign @xhebox

Can we merge this PR later or we still wait for another committer's approval?

xhebox · 2021-10-08T05:48:16Z

@zhaoxugang thanks. I've updated the release note part to None for you. @xhebox please check whether we write the release note somewhere so that we don't miss this feature in 5.3.0 release.

@zhaoxugang please fix go.mod conflict.

/assign @xhebox

Can we merge this PR later or we still wait for another committer's approval?

No, it can be merged now. I believe the release note is included in #27969

/merge

ti-chi-bot · 2021-10-08T05:48:18Z

This pull request has been accepted and is ready to merge.

Commit hash: c3d6792

ti-chi-bot · 2021-10-08T05:48:28Z

@zhaoxugang: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Sep 14, 2021

ti-chi-bot requested review from AilinKid and xhebox September 14, 2021 16:14

xhebox reviewed Sep 15, 2021

View reviewed changes

mjonss reviewed Sep 15, 2021

View reviewed changes

mjonss mentioned this pull request Sep 15, 2021

ddl: add schema placement rules #27969

Merged

10 tasks

xhebox reviewed Sep 17, 2021

View reviewed changes

ddl/ddl_api.go Outdated Show resolved Hide resolved

ddl/ddl_api.go Outdated Show resolved Hide resolved

ddl/ddl_api.go Outdated Show resolved Hide resolved

ti-chi-bot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Sep 17, 2021

xhebox reviewed Sep 22, 2021

View reviewed changes

zhaoxugang added 4 commits September 22, 2021 23:13

support database placement option and partition placement option

0e96e86

add parser replace

18a9379

remove part about of schema

8f44cd9

fix

483b82b

ti-chi-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Sep 22, 2021

zhaoxugang force-pushed the support_database_placement_option_ branch from bfeedc2 to 483b82b Compare September 22, 2021 15:37

ti-chi-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Sep 22, 2021

fix go.mod

214e69f

zhaoxugang added 4 commits September 23, 2021 00:21

fix

c7dce67

fix

457ea5f

fix test

0dc5a08

fix test

8ad82f4

ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 23, 2021

mjonss approved these changes Sep 28, 2021

View reviewed changes

zhaoxugang added 3 commits September 29, 2021 08:11

fix

1fc2ef0

Merge branch 'master' into support_database_placement_option_

f612a51

Merge branch 'master' into support_database_placement_option_

485830b

mjonss approved these changes Oct 1, 2021

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Oct 1, 2021

zhaoxugang added 2 commits October 2, 2021 12:16

Merge branch 'master' into support_database_placement_option_

2f48a14

fix test

6183fab

Merge branch 'master' into support_database_placement_option_

8b70a24

ti-chi-bot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. release-note-none Denotes a PR that doesn't merit a release note. and removed release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Oct 6, 2021

ti-chi-bot assigned xhebox Oct 6, 2021

Merge branch 'master' into support_database_placement_option_

c3d6792

ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 7, 2021

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Oct 8, 2021

Merge branch 'master' into support_database_placement_option_

2e8d0fa

ti-chi-bot merged commit 6c44982 into pingcap:master Oct 8, 2021

xhebox mentioned this pull request Oct 8, 2021

Ensure the partitionDef options for placement are persisted in TiDB DDL #27973

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

executor, ddl: support database placement option and partition placement option #28025

executor, ddl: support database placement option and partition placement option #28025

zhaoxugang commented Sep 14, 2021 •

edited by tisonkun

Loading

ti-chi-bot commented Sep 14, 2021 •

edited

Loading

zhaoxugang commented Sep 14, 2021

xhebox left a comment

mjonss left a comment

mjonss Sep 15, 2021

zhaoxugang Sep 15, 2021

xhebox left a comment

xhebox left a comment

zhaoxugang commented Sep 22, 2021

zhaoxugang commented Sep 22, 2021

mjonss left a comment

mjonss Sep 28, 2021

zhaoxugang Sep 29, 2021

ti-chi-bot commented Sep 28, 2021

morgo commented Sep 28, 2021

mjonss left a comment

tisonkun commented Oct 5, 2021

tisonkun commented Oct 5, 2021

xhebox commented Oct 6, 2021

zhaoxugang commented Oct 6, 2021

tisonkun commented Oct 6, 2021

xhebox commented Oct 8, 2021 •

edited

Loading

ti-chi-bot commented Oct 8, 2021

ti-chi-bot commented Oct 8, 2021

executor, ddl: support database placement option and partition placement option #28025

executor, ddl: support database placement option and partition placement option #28025

Conversation

zhaoxugang commented Sep 14, 2021 • edited by tisonkun Loading

What problem does this PR solve?

Check List

Release note

ti-chi-bot commented Sep 14, 2021 • edited Loading

zhaoxugang commented Sep 14, 2021

xhebox left a comment

Choose a reason for hiding this comment

mjonss left a comment

Choose a reason for hiding this comment

mjonss Sep 15, 2021

Choose a reason for hiding this comment

zhaoxugang Sep 15, 2021

Choose a reason for hiding this comment

xhebox left a comment

Choose a reason for hiding this comment

xhebox left a comment

Choose a reason for hiding this comment

zhaoxugang commented Sep 22, 2021

zhaoxugang commented Sep 22, 2021

mjonss left a comment

Choose a reason for hiding this comment

mjonss Sep 28, 2021

Choose a reason for hiding this comment

zhaoxugang Sep 29, 2021

Choose a reason for hiding this comment

ti-chi-bot commented Sep 28, 2021

morgo commented Sep 28, 2021

mjonss left a comment

Choose a reason for hiding this comment

tisonkun commented Oct 5, 2021

tisonkun commented Oct 5, 2021

xhebox commented Oct 6, 2021

zhaoxugang commented Oct 6, 2021

tisonkun commented Oct 6, 2021

xhebox commented Oct 8, 2021 • edited Loading

ti-chi-bot commented Oct 8, 2021

ti-chi-bot commented Oct 8, 2021

zhaoxugang commented Sep 14, 2021 •

edited by tisonkun

Loading

ti-chi-bot commented Sep 14, 2021 •

edited

Loading

xhebox commented Oct 8, 2021 •

edited

Loading