Refactor Lifetime Event Loop #14996
Refactor Lifetime Event Loop #14996
Conversation
|
While trying to integrate the evloop with the experimental shard for RFC 2, I noticed a race condition between I solved it with the requirement that to resume an IO event with a timeout we must successfully dequeue the event from both queues (poll descriptor waiters and timers) and a bias on timers: they always win. This allowed to keep the Notes:
|
|
This pull request has been mentioned on Crystal Forum. There might be relevant details there: https://forum.crystal-lang.org/t/new-event-loop-unix-call-for-reviews-tests/7207/1 |
|
I just noticed one difference in I'm not sure if this has any impact on this evloop. If the |
|
I'm unable to build this on Debian Sid with full logFWIW, I can build up to 2821fbb just fine and I have plenty of available memory while building this branch
edit: (it builds successfully with |
|
Sadly the log doesn't tell much because this is a runtime error in a macro run (to generate Crystal code from ECR). The only new |
Can't remember if I set it that high or if it was the default :/ I did some debugging of the macros if it helps at all, new ev, evloop_libevent edit: here's the diff (though it'd be easier to view with Meld) |
|
The large number of fds is the problem. The new evloop is trying to virtually allocate ~40GB of memory and refuses to. What's weird is that this is only reserved memory, not allocated memory 🤔 It's also working on macos despite the hardware limit being infinite (capped to Int32::MAX). What's the software limit for |
|
|
@GeopJr I made a change to select the soft limit instead of the hard limit. That should workaround the mmap issue —until we improve the arena to allocate individual blocks dynamically, instead of a single region at once. |
|
Can confirm that it compiles successfully, thanks! (Can also confirm that I didn't face any issues with the new ev + GTK) |
|
While integrating with the execution_context shard I identified a race condition in |
Keeps information about the event that a fiber is waiting on, can be a time event and/or an IO event.
Keeps waiting reader and writer events for an IO. The event themselves keep the information about the event and the associated fiber.
A simple, unoptimized, data structure to keep a list of timed events (IO timeout, sleep or select timeout).
The foundation for the system specific epoll and kqueue event loops.
Specific to Linux and Android. It might be working on Solaris too through their Linux compatibility layer.
For BSD and Darwin.
This is now only required for the libevent event loop, and the wasi pseudo event loop.
Also includes the `:evloop_libevent` flag to fallback to libevent.
Avoids an issue with the timecop shard that overrides Time.monotonic.
We must cast the actual LibEvent types because the type signatures return the abstract Crystal::EventLoop interface that don't implement the necessary methods.
|
@straight-shoota the check format CI action fails, but I can't reproduce on my local (neither with crystal 1.14.0 nor a compiler from this branch) 😕 |
|
The failure happens on the |
|
This pull request has been mentioned on Crystal Forum. There might be relevant details there: https://forum.crystal-lang.org/t/new-event-loop-unix-call-for-reviews-tests/7207/22 |
Replaces the static `mmap` that must accommodate for as many file descriptors as allowed by ulimit/rlimit. Despite being virtual memory, not really allocated in practice, this led to out-of-memory errors in some situations. The arena now dynamically allocates individual blocks as needed (no more virtual memory). For simplicity reasons it will only ever grow, and won't shrink (we may think of a solution for this later). The original safety guarantees still hold: once an entry has been allocated in the arena, its pointer won't change. The event loop still limits the arena capacity to the hardware limit (ulimit: open files). **Side effect:** the arena don't need to remember the maximum fd/index anymore; that was only needed for `fork`; we can simply iterate the allocated blocks now. Co-authored-by: Johannes Müller <[email protected]>
Refactors the internals of the epoll/kqueue event loop to `yield` the fiber(s) to be resumed instead of blindly calling `Crystal::Scheduler.enqueue`, so the `#run` method becomes the one place responsible to enqueue the fibers. The current behavior doesn't change, the `#run` method still enqueues the fiber immediately, but it can now be changed in a single place. For example the [execution context shard](https://github.com/ysbaddaden/execution_context) monkey-patches an alternative `#run` method that collects and returns fibers to avoid parallel enqueues from an evloop run to interrupt the evloop run (:sob:). Note that the `#close` method still directly enqueues waiting fibers one by one, for now.
Related to [RFC #12](crystal-lang/rfcs#12). Replaces the `Deque` used in #14996 for a min [Pairing Heap] which is a kind of [Mergeable Heap] and is one of the best performing heap in practical tests when arbitrary deletions are required (think cancelling a timeout), otherwise a D-ary Heap (e.g. 4-heap) will usually perform better. See the [A Nearly-Tight Analysis of Multipass Pairing Heaps](https://epubs.siam.org/doi/epdf/10.1137/1.9781611973068.52) paper or the Wikipedia page for more details. The implementation itself is based on the [Pairing Heaps: Experiments and Analysis](https://dl.acm.org/doi/pdf/10.1145/214748.214759) paper, and merely implements a recursive twopass algorithm (the auxiliary twopass might perform even better). The `Crystal::PointerPairingList(T)` type is generic and relies on intrusive nodes (the links are into `T`) to avoid extra allocations for the nodes (same as `Crystal::PointerLinkedList(T)`). It also requires a `T#heap_compare` method, so we can use the same type for a min or max heap, or to build a more complex comparison. Note: I also tried a 4-heap, and while it performs very well and only needs a flat array, the arbitrary deletion (e.g. cancelling timeout) needs a linear scan and its performance quickly plummets, even at low occupancy, and becomes painfully slow at higher occupancy (tens of microseconds on _each_ delete, while the pairing heap does it in tens of nanoseconds). Follow up to #14996 [Mergeable Heap]: https://en.wikipedia.org/wiki/Mergeable_heap [Pairing Heap]: https://en.wikipedia.org/wiki/Pairing_heap [D-ary Heap]: https://en.wikipedia.org/wiki/D-ary_heap Co-authored-by: Linus Sellberg <[email protected]> Co-authored-by: Johannes Müller <[email protected]>
Almost identical to #14959 but with cleaner history and more documentation that led me to identify issues in the arena.
RFC: https://github.com/crystal-lang/rfcs/blob/main/text/0009-lifetime-event_loop.md
Overall design
The logic of the event loop doesn't change much from libevent: we try to execute an operation (e.g. read, write, connect, ...) on nonblocking file descriptors; if the operation would block (EAGAIN) we create an event that references the operation along with the current fiber; we eventually rely on the polling system (epoll or kqueue) to report when an fd is ready, which will dequeue a pending event and resume its associated fiber (one at a time).
Unlike libevent that will add and remove the fd to and from the polling system every time we'd block, this event loop adds it once and will only remove it when closing the fd. The theory is that being notified for readiness (once thanks to edge-triggered) is less expensive than always modifying the polling system. In practice a purely IO-bound benchmark with long running sockets, we notice up to 20% performance improvement. Actual applications should see less improvements.
Implementation details
Unlike the previous attempts to integrate epoll and kqueue directly that required a global events object and a global lock to protect it, this PR only needs fine-grained locks for each IO object and operation (read, write) to add the events to the waiting lists. In practice, the locks should never be contented (unless you share a fd for the same read or write operation in multiple fibers).
Caveat: timers are still global to the event loop, and we need a global lock to protect it. This means that IO with timeout will still see contention. Improving the timers data structure will come later (e.g. lock free, more precise, faster operations).
To avoid keeping pointers to the IO object that could prevent the GC from collecting lost IO objects, this PR introduces "poll descriptor" objects (the name comes from Go's netpoll) that keep the list of readers and writers and don't point back to the IO object. The GC collecting an IO object is fine: the finalizer will close the fd and tell the event loop to cleanup the
fd resourcesassociated poll descriptor (so we can safely reuse the fd).To avoid pushing raw pointers into the kernel data structures, and to quickly retrieve the poll descriptor from a mere fd, but also to avoid programming errors that would segfault the program, this PR introduces a Generational Arena to store the "Poll Descriptors" (the name is inherited from Go's netpoll) so we only store an index into the polling system. Another benefit is that we can reuse the existing allocation when a fd is reused. If we try to retrieve an outdated index (the allocation was freed or reallocated) the arena will raise an explicit exception.
The poll descriptors associate a fd to an event loop instance, so we can still have multiple event loops per processes, yet make sure that an fd is only ever in one event loop. When a fd will block on another event loop instance, the fd will be transferred automatically (i.e. removed from the old one & added to the new one). The benefits are numerous: this avoids having multiple event loops being notified at the same time; this avoids having to close/remove the fd from each event loop instances; this avoids cross event loop enqueues that are much slower than local enqueues in RFC 2.
A limitation is that trying to move a fd from one evloop to another while there are pending waiters will raise an exception. We can't move timeout events along with the fd from one event loop instance to another one, but that would also break the "always local enqueues" benefit.
Most application shouldn't notice any impact because of this design choice, since a fd is usually not shared across fibers (concurrency issues), except maybe a server socket with multiple accepting fibers? In that case you'll need to make sure the fibers are on the same thread (
preview_mt) or same context (RFC 2).Availability
We may want to have a compile time flag to enable the new event loop before we merge? For the time being this PR uses the new shiny evloop by default (to have CI runs) and introduces the
:evloop_libeventflag to fallback to libevent.TODO
A couple changes before merge (but not blocking review):
-Devloop=libeventto return to libevent (in case of regressions or to test/benchmark);-Devloop=epollto useepoll(e.g. Solaris);-Devloop=kqueueto usekqueue(on *BSD);epoll_waitorkeventwith infinite timeout;#try_run?and#try_lock?methods that are no longer needed.REGRESSIONS/ISSUES
Timers are noticeably slower than libevent (especially
Fiber.yield): we should consider a minheap (4-heap) or a skiplist;DragonFlyBSD: running
std_specis eons slower than libevent; it regularly hangs on evloop.run until the stack pool collector timeout kicks in (i.e. loops on 5s pauses);OpenBSD: running
std_specis noticeably slower (4:15 minutes) compared to libevent (1:16 minutes); it appears that the main fiber keeps re-enqueueing itself from the evloop run (10us on each run);NetBSD: the evloop doesn't work with
keventreturning ENOENT for the signal loopfdand EINVAL when trying to set anEVFILT_TIMER.For the reasons above the kqueue evloop is disabled by default on DragonFly(BSD), OpenBSD and NetBSD.