Skip to content

Commit

Permalink
Channel program may spuriously fail with "memory limit exhausted"
Browse files Browse the repository at this point in the history
ZFS channel programs (invoked by `zfs program`) are executed in a LUA
sandbox with a limit on the amount of memory they can consume.  The
limit is 10MB by default, and can be raised to 100MB with the `-m` flag.
If the memory limit is exceeded, the LUA program exits and the command
fails with a message like `Channel program execution failed: Memory
limit exhausted.`

The LUA sandbox allocates memory with `vmem_alloc(KM_NOSLEEP)`, which
will fail if the requested memory is not immediately available.  In this
case, the program fails with the same message, `Memory limit exhausted`.
However, in this case the specified memory limit has not been reached,
and the memory may only be temporarily unavailable.

This commit changes the LUA memory allocator `zcp_lua_alloc()` to use
`vmem_alloc(KM_SLEEP)`, so that we won't spuriously fail when memory is
temporarily low.  Instead, we rely on the system to be able to free up
memory (e.g. by evicting from the ARC), and we assume that even at the
highest memory limit of 100MB, the channel program will not truly
exhaust the system's memory.

External-issue: DLPX-71924
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: Ryan Moeller <[email protected]>
Signed-off-by: Matthew Ahrens <[email protected]>
Closes #11190
  • Loading branch information
ahrens authored Nov 12, 2020
1 parent c08d442 commit a724db0
Showing 1 changed file with 1 addition and 6 deletions.
7 changes: 1 addition & 6 deletions module/zfs/zcp.c
Original file line number Diff line number Diff line change
Expand Up @@ -722,8 +722,6 @@ static void *
zcp_lua_alloc(void *ud, void *ptr, size_t osize, size_t nsize)
{
zcp_alloc_arg_t *allocargs = ud;
int flags = (allocargs->aa_must_succeed) ?
KM_SLEEP : (KM_NOSLEEP | KM_NORMALPRI);

if (nsize == 0) {
if (ptr != NULL) {
Expand All @@ -746,10 +744,7 @@ zcp_lua_alloc(void *ud, void *ptr, size_t osize, size_t nsize)
return (NULL);
}

allocbuf = vmem_alloc(allocsize, flags);
if (allocbuf == NULL) {
return (NULL);
}
allocbuf = vmem_alloc(allocsize, KM_SLEEP);
allocargs->aa_alloc_remaining -= allocsize;

*allocbuf = allocsize;
Expand Down

0 comments on commit a724db0

Please sign in to comment.