When handling a scatter query vtgate should ensure consistent schema

[setassociative]

Overview of the Issue

If a table is in the middle of a schema migration it's possible for a scattered SELECT * query to return an inconsistent schema depending on the state of each tablet. When aggregating results we should ensure that each result contains the same set of fields and return an error if not.

Caveat: I'm not positive this is not already handled but based on commentary in this comment I don't believe it is.

Reproduction Steps

1. Begin making a schema change
2. Issue a scatter select * before the schema is applied to all shards

[indera-shsp]

@setassociative Did you guys deploy this patch to production?

tinyspeck@3d3f660
I am encountering this issue when trying to connect to the Vstream

panic: runtime error: index out of range [4] with length 4

goroutine 30 [running]:
vitess.io/vitess/go/sqltypes.MakeRowTrusted(0xc0001be220, 0x4, 0x4, 0xc000169e50, 0x8, 0xdd0840, 0x1)
	/go/pkg/mod/vitess.io/[email protected]/go/sqltypes/result.go:170 +0x178
vitess.io/vitess/go/sqltypes.proto3ToRows(0xc0001be220, 0x4, 0x4, 0xc0001b4520, 0x1, 0x1, 0xc000cc7820, 0x415c78, 0x8)

[setassociative]

@indera-shsp yes! we've been running with that patch in prod since Feb 2019; it never made it upstream because, iirc, there was some concern that we were essentially trading one case of panic for another. A call out though is that this case is generally extremely transient -- if you're seeing it every time you try to connect to a vstream there is likely something else going on and it suggests that the connecting endpoint doesn't have the right schema.

My suggestion would be to drop into the Vitess slack and you can get some more targeted advice.

[indera-shsp]

Thanks for the reply... this issue seems to have two fixes, one patch and one PR in progress 🤣
(at least that how it seems to me)

• https://github.com/vitessio/vitess/pull/5572/files - -enable-query-plan-field-caching=false
• VStreamer: Schema versioning #6164 - -watch_replication_stream and -track_schema_versions
• fix query panic when fields and row length is not matching #7325