Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VSchema Validation on ReshardWorkflow Creation #7977

Merged
merged 1 commit into from
Apr 30, 2021

Conversation

makinje16
Copy link
Contributor

Signed-off-by: Malcolm Akinje [email protected]

Description

This PR adds in vschema validation during creation of a ReshardWorkflow. If any of the source shards have tables in their _vt database that are not in the vschema then the creation of the ReshardWorkflow will fail. We will then return which shards failed their vschema checks and what tables caused their respective failures.

Related Issue(s)

#7620

Checklist

  • Tests were added or are not required
  • Documentation was added or is not required

Deployment Notes

Copy link
Member

@rafael rafael left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work Malcolm!

This is looking good. One thing that I was discussing with @rohit-nayak-ps is that we should make sure to ignore ghost schema changes from these validations.

However, there are a few other places we need to update before fully adopting those conventions (e.g VDiff should ignore tables that are related to ghost).

We can work on that as separate PR.

@deepthi
Copy link
Member

deepthi commented Apr 29, 2021

Great work Malcolm!

This is looking good. One thing that I was discussing with @rohit-nayak-ps is that we should make sure to ignore ghost schema changes from these validations.

However, there are a few other places we need to update before fully adopting those conventions (e.g VDiff should ignore tables that are related to ghost).

We can work on that as separate PR.

Does this mean that in the presence of gh-ost artifact tables workflow creation will always fail (with the changes in this PR)?

@rafael
Copy link
Member

rafael commented Apr 29, 2021

Does this mean that in the presence of gh-ost artifact tables workflow creation will always fail (with the changes in this PR)?
@deepthi that is correct. But I think that is better right now than the existent behavior. Today, we will create the resharding workflow, but then VDiff will consistently fail because it doesn't know how to handle this case yet.

Copy link
Contributor

@rohit-nayak-ps rohit-nayak-ps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Contributor

@ajm188 ajm188 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. One comment that I think can be done as a follow-up change.

@@ -199,7 +217,7 @@ func (wr *Wrangler) ValidateSchemaKeyspace(ctx context.Context, keyspace string,
}
sort.Strings(shards)
if len(shards) == 1 {
return wr.ValidateSchemaShard(ctx, keyspace, shards[0], excludeTables, includeViews)
return wr.ValidateSchemaShard(ctx, keyspace, shards[0], excludeTables, includeViews, false /*includeVSchema*/)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a follow-up, I think it would be great to add the VSchema validation to ValidateSchemaKeyspace as well

@ajm188 ajm188 merged commit 92da561 into vitessio:master Apr 30, 2021
@ajm188 ajm188 deleted the vschema-validation-checks branch April 30, 2021 17:41
@makinje16 makinje16 mentioned this pull request Apr 30, 2021
2 tasks
@askdba askdba added Component: VReplication Type: Enhancement Logical improvement (somewhere between a bug and feature) labels May 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: VReplication Type: Enhancement Logical improvement (somewhere between a bug and feature)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants