Skip to content

Commit

Permalink
GC: Consider size of value while rewriting (#1357)
Browse files Browse the repository at this point in the history
The existing code doesn't consider the size of the value while
calculating the size of the entry batch in rewrite method.
This PR fixes it.

Fixes #1292

(cherry picked from commit 14386ac)
  • Loading branch information
Ibrahim Jarif authored and NamanJain8 committed Sep 8, 2020
1 parent f76b64d commit 969cacb
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions value.go
Original file line number Diff line number Diff line change
Expand Up @@ -411,6 +411,11 @@ func (vlog *valueLog) rewrite(f *logFile, tr trace.Trace) error {

ne.Value = append([]byte{}, e.Value...)
es := int64(ne.estimateSize(vlog.opt.ValueThreshold))
// Consider size of value as well while considering the total size
// of the batch. There have been reports of high memory usage in
// rewrite because we don't consider the value size. See #1292.
es += int64(len(e.Value))

// Ensure length and size of wb is within transaction limits.
if int64(len(wb)+1) >= vlog.opt.maxBatchCount ||
size+es >= vlog.opt.maxBatchSize {
Expand Down

0 comments on commit 969cacb

Please sign in to comment.