backupccl: make checkpoint interval configurable #83151
Conversation
|
This change is |
stevendanna
requested review from
adityamaru and
benbardin
and removed request for
livlobo
|
@benbardin I think we need to backport the setting as well since we plan on shipping the setting to a customer and we want them to be able to upgrade to a real release once it is available. |
stevendanna
force-pushed
the
configurable-checkpoint-interval
branch
from
99d286a to
d235fe1
Compare
| defer leaktest.AfterTest(t)() | ||
| defer log.Scope(t).Close(t) | ||
|
|
||
| skip.WithIssue(t, 33357) |
There was a problem hiding this comment.
This test has been skipped since 2019 and this issue has been closed.
This PR makes the interval between checkpoints configurable and also excludes the processing time of the checkpoint itself from that interval. The goal of this change is to potentially address issues we've seen in large clusters that we currently believe can be attributed to the backup process slowing down substantially once it takes a minute or longer to marshall, compress, and write the progress checkpoint. Release note (ops change): A new setting `bulkio.backup.checkpoint_interval` controls the minimum interval between writes of progress checkpoints to external storage.
stevendanna
force-pushed
the
configurable-checkpoint-interval
branch
from
d235fe1 to
76f5a09
Compare
|
TFTR! bors r=adityamaru,benbardin |
|
Build succeeded: |
|
Encountered an error creating backports. Some common things that can go wrong:
You might need to create your backport manually using the backport tool. error creating merge commit from 76f5a09 to blathers/backport-release-22.1-83151: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict [] you may need to manually resolve merge conflicts with the backport tool. Backport to branch 22.1.x failed. See errors above. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan. |
This PR makes the interval between checkpoints configurable and also
excludes the processing time of the checkpoint itself from that
interval.
The goal of this change is to potentially address issues we've seen in
large clusters that we currently believe can be attributed to the
backup process slowing down substantially once it takes a minute or
longer to marshall, compress, and write the progress checkpoint.
Release note (ops change): A new setting
bulkio.backup.checkpoint_intervalcontrols the minimum intervalbetween writes of progress checkpoints to external storage.