Schema tracking in vtgate #8074
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
systay
added
Component: Query Serving
Type: Enhancement
systay
force-pushed
the
schema-tracking-vtgate
branch
from
5c1c251 to
9f7a7ff
Compare
Signed-off-by: Harshit Gangal <[email protected]>
Signed-off-by: Andres Taylor <[email protected]>
Signed-off-by: Andres Taylor <[email protected]>
Signed-off-by: Andres Taylor <[email protected]>
Signed-off-by: Harshit Gangal <[email protected]>
harshit-gangal
force-pushed
the
schema-tracking-vtgate
branch
from
8e6f5a3 to
410cd4c
Compare
Signed-off-by: Harshit Gangal <[email protected]>
harshit-gangal
force-pushed
the
schema-tracking-vtgate
branch
from
36a4841 to
31294da
Compare
…vtopo.Server directly Signed-off-by: Harshit Gangal <[email protected]>
Signed-off-by: Harshit Gangal <[email protected]>
…rom schema tracker Signed-off-by: Harshit Gangal <[email protected]>
Signed-off-by: Andres Taylor <[email protected]>
Signed-off-by: Andres Taylor <[email protected]>
Signed-off-by: Andres Taylor <[email protected]>
Signed-off-by: Andres Taylor <[email protected]>
…ema.Tracker Signed-off-by: Andres Taylor <[email protected]>
Signed-off-by: Andres Taylor <[email protected]>
Co-authored-by: Andres Taylor <[email protected]> Signed-off-by: Florent Poinsard <[email protected]>
…chema when errors Signed-off-by: Florent Poinsard <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Signed-off-by: Andres Taylor <[email protected]>
systay
force-pushed
the
schema-tracking-vtgate
branch
from
0c52ebe to
3c11a1c
Compare
Signed-off-by: Florent Poinsard <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Signed-off-by: Andres Taylor <[email protected]>
Signed-off-by: Andres Taylor <[email protected]>
systay
force-pushed
the
schema-tracking-vtgate
branch
from
7cada71 to
369ad4a
Compare
Co-authored-by: Andres Taylor <[email protected]> Signed-off-by: Florent Poinsard <[email protected]>
Co-authored-by: Andres Taylor <[email protected]> Signed-off-by: Florent Poinsard <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
…tgate Signed-off-by: Andres Taylor <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
Signed-off-by: Florent Poinsard <[email protected]>
harshit-gangal
requested review from
deepthi,
GuptaManan100,
shlomi-noach and
systay
as code owners
Signed-off-by: Florent Poinsard <[email protected]>
2 tasks
harshit-gangal
commented
Jun 1, 2021
Comment on lines
+106
to
+114
| clusterInstance.VtTabletExtraArgs = []string{"-queryserver-config-schema-change-signal"} | ||
| err = clusterInstance.StartUnshardedKeyspace(*keyspace, 1, false) | ||
| require.NoError(t, err) | ||
|
|
||
| // teardown vttablets | ||
| for _, vttablet := range clusterInstance.Keyspaces[0].Shards[0].Vttablets { | ||
| err = vttablet.VttabletProcess.TearDown() | ||
| require.NoError(t, err) | ||
| } |
There was a problem hiding this comment.
instead of starting tablets and shutting them down, we can test this by just creating the keyspace and shard in topo by calling vtctld APIs (CreateKeyspace and CreateShard). That will mimic the same behavior.
go/test/endtoend/vtgate/schematracker/loadkeyspace/schema_load_keyspace_test.go
Show resolved
Hide resolved
go/test/endtoend/vtgate/schematracker/restarttablet/schema_restart_test.go
Show resolved
Hide resolved
Comment on lines
+161
to
+165
| if v == nil { | ||
| // We encountered an error, we should always have a current vschema | ||
| log.Warning("got a schema changed signal with no loaded vschema. if this persist, something is wrong") | ||
| vschema, _ = vindexes.BuildVSchema(&vschemapb.SrvVSchema{}) | ||
| } else { |
There was a problem hiding this comment.
if currentSrvSchema is nil we do not need to rebuild the VSchema as the empty VSchema would have been published to the subscriber through VSchemaUpdate
This should only be done in the case when we are using updated schema to enhance the VSchema
Comment on lines
+1160
to
+1162
| if vschema != nil { | ||
| e.vschema = vschema | ||
| } |
There was a problem hiding this comment.
This can create an issue when a user wants to explicitly clear out the VSchema.
This check will ignore the update and will keep the old version.
systay
approved these changes
Jun 1, 2021
This was referenced Jun 22, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This adds a schema tracker that will update the vschema with schema information about tables and columns, fetched from the underlying MySQL instance.
Related Issue(s)
#7995
#7994
Checklist