Interruptable garbage collection

[Stebalien]

What version of Go are you using (go version)?

$ go version
go version go1.14.2 amd64/linux

What operating system are you using?

Linux

What version of Badger are you using?

v1.6.1

Description

RunValueLogGC cannot be interrupted and can potentially run for minutes on a large enough datastore. This can be a problem if garbage collection is running on shutdown.

It would be nice to be able to pass a context to RunValueLogGC to cancel the process early. Alternatively, allowing a concurrent call to Close to interrupt a GC run would fix this. However, at the moment, close can't be called concurrently with anything else.

Would a new RunValueLogGCWithContext function be considered if a patch were proposed?

[jarifibrahim]

This looks like a useful enhancement since GC could take minutes at times to complete.The GC has two steps. The first sampling step can be exited at any time.

badger/value.go

Line 1628 in 536fed1

func (vlog *valueLog) doRunGC(lf *logFile, discardRatio float64, tr trace.Trace) (err error) {

The second step is where we rewrite the value log file. I'm wondering if there would be any side effects of stopping the rewrite while it's running

badger/value.go

Line 499 in 536fed1

func (vlog *valueLog) rewrite(f *logFile, tr trace.Trace) error {

[minhaj-shakeel]

Github issues have been deprecated.
This issue has been moved to discuss. You can follow the conversation there and also subscribe to updates by changing your notification preferences.