Skip to content

Commit

Permalink
jobs: bump default progress log time to 30s
Browse files Browse the repository at this point in the history
The previous code allowed updates to be performed every 1s, which could
cause the MVCC row to be very large causing problems with splits. We
can update much more slowly by default. In the case of a small backup
job, the 5% fraction threshold will allow a speedier update rate.

Remove a note that's not useful anymore since the referred function
can now only be used in the described safe way.

See #25770. Although this change didn't fix that bug, we still think
it's a good idea.

Release note: None
  • Loading branch information
maddyblue committed May 31, 2018
1 parent e6bbb2e commit 1faebfa
Showing 1 changed file with 1 addition and 9 deletions.
10 changes: 1 addition & 9 deletions pkg/sql/jobs/progress.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ import (
// last logged fractionCompleted and the current fractionCompleted is more than
// progressFractionThreshold.
var (
progressTimeThreshold = time.Second
progressTimeThreshold = 30 * time.Second
progressFractionThreshold float32 = 0.05
)

Expand Down Expand Up @@ -53,14 +53,6 @@ type ProgressLogger struct {
// chunkFinished marks one chunk of the job as completed. If either the time or
// fraction threshold has been reached, the progress update will be persisted to
// system.jobs.
//
// NB: chunkFinished is not threadsafe. A previous implementation that was
// threadsafe occasionally led to massive contention. One 2TB restore on a 15
// node cluster, for example, had 60 goroutines attempting to update the
// progress at once, causing massive contention on the row in system.jobs. This
// inadvertently applied backpressure on the restore's import requests and
// slowed the job to a crawl. If multiple threads need to update progress, use a
// channel and a dedicated goroutine that calls loop.
func (jpl *ProgressLogger) chunkFinished(ctx context.Context) error {
jpl.completedChunks++
fraction := float32(jpl.completedChunks) / float32(jpl.TotalChunks)
Expand Down

0 comments on commit 1faebfa

Please sign in to comment.