custombuild-v20.1.6-1-g63f61aa24e
jordanlewis
tagged this
29 Sep 14:40
This commit adds a memory monitor to the kv fetcher infrastructure that's initialized by its users. When kv fetches occur, the new infrastructure ensure that there's always at least 1 kilobyte allocated for the fetch before it happens. Once the fetch returns, the accounting is adjusted to include the entire size of the fetch. Subsequent fetches that return less memory do *not* ratchet down the allocation size to preserve safety and reduce some pointless allocation adjustment churning. This memory monitoring is gated behind a new cluster setting called `sql.scan_memory_accounting.enabled` in 20.1. In 20.2, this memory monitoring will always be on, and no such cluster setting will exist. Release note (sql change): the memory used by disk scans is accounted for, reducing the likelihood of out of memory conditions that result in process crashes as opposed to SQL out of memory error messages. This new behavior is off by default, and is gated behind the sql.scan_memory_accounting.enabled cluster setting. Release justification: fixes for high-priority bugs in existing functionality