Avoid memory allocations in the ARC eviction thread

[markjdb]

This change mitigates a low-memory deadlock observed on FreeBSD, where the ARC eviction thread fails to allocate memory and sleeps, blocking the page reclamation thread.

Motivation and Context

The change removes some memory allocations from the ARC eviction thread, thus helping ensure that arc_wait_for_eviction() won't block a thread that's attempting to reclaim pages on behalf of the rest of the system.

Description

The change modifies arc_init() to pre-allocate a set of marker buffer headers for use as placeholders in arc_evict_state(). When the ARC eviction thread calls arc_evict_state(), it uses the pre-allocated markers rather than performing a blocking memory allocation.

How Has This Been Tested?

I was able to reproduce the deadlock occasionally using parallel pgbench and FreeBSD kernel builds to incur memory pressure. With this patch I haven't been able to trigger any hangs.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.

[amotin]

Thanks. I agree that memory allocations in evict thread is a bad idea. I don't see any problems in this patch, but generally in perspective I'd really like eviction to be multi-threaded, since single thread is a known bottleneck. That would require some changes to this code, but that may be handled later.

[behlendorf]

If we can avoid the allocation here, I agree that would be a good thing. On a related note, we may have avoided this particular deadlock on Linux because of the spl_fstrans_mark() / spl_fstrans_unmark() calls in arc_evict_cb(). They're wired up on Linux to allow us to mark critical sections where we need to ensure the memory allocator won't ask the VFS layer to free up memory to satisfy the allocation (which might deadlock). I'm not sure if there's a need for this on FreeBSD, but I thought it worth mentioning.

[gamanakis]

I like the proposed change, don't see any problems with it, thank you. As mentioned in #12987, it seems to help with the arc_evict and arc_prune contention when there is pressure on the ARC.

[markjdb]

If we can avoid the allocation here, I agree that would be a good thing. On a related note, we may have avoided this particular deadlock on Linux because of the spl_fstrans_mark() / spl_fstrans_unmark() calls in arc_evict_cb(). They're wired up on Linux to allow us to mark critical sections where we need to ensure the memory allocator won't ask the VFS layer to free up memory to satisfy the allocation (which might deadlock). I'm not sure if there's a need for this on FreeBSD, but I thought it worth mentioning.

I think the larger difference is that FreeBSD doesn't do direct reclaim: if a page allocation fails, the allocating thread must either handle the failure or sleep waiting for system threads (the "page daemon") to reclaim memory. That approach has some advantages - it makes system behaviour in low memory conditions much easier to reason about for one - but it does mean that threads responsible for reclaiming memory have to be disciplined and avoid memory allocations (or pre-allocate memory reserves to avoid deadlocks). This is easy to handle within the page daemon, but gets tricky once the page daemon has to request work from other threads (the ARC eviction thread in this case; system I/O threads if the page daemon is writing dirty pages to swap; etc.). For memory reclamation purposes, FreeBSD doesn't maintain a real distinction between anonymous pages and file-backed pages the way Linux does, so there's no real way to implement spl_fstrans_mark() in any case.