backport upstream patch 12178 and 9106

[pbibra]

Signed-off-by: Priya Bibra [email protected]

Description

This PR backports two upstream patches to resolve some vtgate health cache issues.

1. When tablets were removed from the topo, vtgate's secondary cache was not properly being cleared. This is fixed in upstream patch 9106: Remove tablet healthcheck cache record on error vitessio/vitess#9106
2. When tablets' type changes from their original creation type, and then they are removed from the topo, vtgate's secondary cache was not properly being cleared. This is fixed in patch 12178: VTGate: Ensure HealthCheck Cache Secondary Maps Stay in Sync With Authoritative Map on Tablet Delete vitessio/vitess#12178

Both problems affected the vtgate's /debug/status page and the show vitess_tablets queries. VTAdmin also periodically fails to load schemas because it relies on these commands to get up to date tablet information.

Testing

pbibra@vtctld-dev-iad-wc-cqxf:~$ sudo vtops-go status -k loadtest -s 80-
alias: , keyspace/shard: loadtest/80-, tablet: , detailed: false

 alias			hostname				type	hardware	SBS / SQL / IO
 us_east_1d-0303146168	tablet-iad-dev-loadtest-80-00-a4fh:3306	PRIMARY	i3.2xlarge
 us_east_1a-0303059067	tablet-iad-dev-loadtest-80-00-6y6d:3306	REPLICA	i3.2xlarge	  0 /   Y /  Y
 us_east_1e-0303243144	tablet-iad-dev-loadtest-80-00-l097:3306	REPLICA	i3.2xlarge	  0 /   Y /  Y
 us_east_1e-0303209134	tablet-iad-dev-loadtest-80-00-emqi:3306	REPLICA	i3.2xlarge	  0 /   Y /  Y
 us_east_1e-0303202038	tablet-iad-dev-loadtest-80-00-s8jj:3306	REPLICA	i3.2xlarge	  0 /   Y /  Y
 us_east_1e-0303232236	tablet-iad-dev-loadtest-80-00-1yxz:3306	REPLICA	i3.2xlarge	  0 /   Y /  Y

SCENARIO 1:

pbibra@vtctld-dev-iad-wc-cqxf:~$ sudo vtops-go deprovision-tablet --tablet tablet-iad-dev-loadtest-80-00-s8jj

before:

[~] ssh vtgate-loadtest-dev-iad-zxpt "mysql -ugod -e \"SHOW vitess_tablets\"" | grep "tablet-iad-dev-loadtest-80-00-s8jj"                                                                                                                           13:43:16
us_east_1e	loadtest	80-	REPLICA	NOT_SERVING	us_east_1e-0303202038	tablet-iad-dev-loadtest-80-00-s8jj

after:

[~] ssh vtgate-loadtest-dev-iad-kwvy "mysql -ugod -e \"SHOW vitess_tablets\"" | grep "tablet-iad-dev-loadtest-80-00-s8jj"                                                                                                                           14:14:16
[~]

SCENARIO 2:

pbibra@vtctld-dev-iad-wc-cqxf:~$ vtctl-slack ChangeTabletType us_east_1e-0303209134 drained
pbibra@vtctld-dev-iad-wc-cqxf:~$ sudo vtops-go deprovision-tablet --tablet tablet-iad-dev-loadtest-80-00-emqi

before:

[~] ssh vtgate-loadtest-dev-iad-zxpt "mysql -ugod -e \"SHOW vitess_tablets\"" | grep "tablet-iad-dev-loadtest-80-00-emqi"                                                                                                                           14:13:52
us_east_1e	loadtest	80-	DRAINED	NOT_SERVING	us_east_1e-0303209134	tablet-iad-dev-loadtest-80-00-emqi

after

[~] ssh vtgate-loadtest-dev-iad-kwvy "mysql -ugod -e \"SHOW vitess_tablets\"" | grep "tablet-iad-dev-loadtest-80-00-emqi"                                                                                                                           14:14:31
[~]

Related Issue(s)

Checklist

• "Backport to:" labels have been added if this change should be back-ported
• Tests were added or are not required
• Documentation was added or is not required

Deployment Notes

[tanjinx]

Nice work, thanks Priya!!

[tanjinx]

this debug code can be removed along with the line below?