-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
upgrades: remove redundant ensureSQLSchemaTelemetrySchedule
permanent upgrade
#139632
upgrades: remove redundant ensureSQLSchemaTelemetrySchedule
permanent upgrade
#139632
Conversation
…nt upgrade During startup, `CreateSchemaTelemetrySchedule`, which creates the scheduled job to collect schema telemetry, is redundantly invoked twice concurrently: once via `schematelemetrycontroller.Controller.Start` and again through the `ensureSQLSchemaTelemetrySchedule` permanent upgrade. This was identified in cockroachdb#119340, where the permanent upgrade encounters contention and is resolved nearly 30 seconds later causing slow startup of the tenant. Although the exact reason for the prolonged transaction deadlock is unclear, we can still benefit by removing this redundant upgrade. Informs: cockroachdb#119340 Closes: cockroachdb#130931 Release note: None Epic: none
5b4225a
to
f04bdb7
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice find! It seems like this test also confirms that the schedule is created upon startup:
func TestSchemaTelemetrySchedule(t *testing.T) { |
Thanks for taking this on! Let's keep an eye on this post merge. At test cluster startup, we still have something of a thundering herd of servers trying to create the telemetry schedule. Moving it out of the permanent upgrades means the contention won't delay startup, but it is still a little unfortunate that we have all of these processes spinning trying to create this simple record. |
Sure! I'll keep an eye on this. bors r=rafiss |
blathers backport 25.1 |
Based on the specified backports for this PR, I applied new labels to the following linked issue(s). Please adjust the labels as needed to match the branches actually affected by the issue(s), including adding any known older branches. Issue #130931: branch-release-25.1. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
During startup,
CreateSchemaTelemetrySchedule
, which creates the scheduled job to collect schema telemetry, is redundantly invoked twice concurrently: once viaschematelemetrycontroller.Controller.Start
and also through theensureSQLSchemaTelemetrySchedule
permanent upgrade.This was identified in #119340, where the permanent upgrade encounters contention and is resolved nearly 30 seconds later causing slow startup of the tenant. Although the exact reason for the prolonged transaction deadlock is unclear, we can still benefit by removing this redundant upgrade.
Informs: #119340
Closes: #130931
Release note: None
Epic: none