Skip to content

Commit

Permalink
Enable chpl_comm_regMemHeapTouch() for more configurations (#25787)
Browse files Browse the repository at this point in the history
### Summary:

This pull request addresses something which I noticed while trying to
understand what code actually runs in Chapel's "no-op" test. As
explained in the commit message (reproduced below), I observed that the
hook to touch heap memory between allocation and static registration
with the NIC was only ever enabled for `FAST` segment mode, even though
`LARGE` is identical to `FAST` in all registration-based conduits other
than ibv. This single-commit pull request resolves this.

This is anticipated to have minimal impact on users, since my
understanding of Chapel's documentation is that it does not recommend
any of aries, ofi or ucx-conduits (except where ucx is mentioned as an
alternative to ibv in "rare cases").

I have taken an approach that splits the preprocessor logic into two
groups of conduits, to produce what I found to be the most intelligible
option. If there is a desire to avoid replicating the assignment to
`gasnet_client_attach_hook`, or to group by segment instead of conduit,
let me know and I will revise as requested.

### Commit(s)

- Enable chpl_comm_regMemHeapTouch() for more configurations. 
  
This commit updates the preprocessor logic which enables use of
`chpl_comm_regMemHeapTouch()` to more accurately reflect the set of
"configurations that register a fixed heap at startup".
  
Specifically, for registration-based conduits other than ibv, the code
for LARGE and FAST segment modes are identical. So, LARGE should be
excluded *only* for ibv-conduit, not for aries, ofi or ucx.
  • Loading branch information
jhh67 authored Aug 29, 2024
2 parents c32ec45 + 7f361ed commit e629baa
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion runtime/src/comm/gasnet/comm-gasnet-ex.c
Original file line number Diff line number Diff line change
Expand Up @@ -884,10 +884,14 @@ void chpl_comm_init(int *argc_p, char ***argv_p) {
// For configurations that register a fixed heap at startup use a gasnet hook
// to allow us to fault and interleave in the memory in parallel for faster
// startup and better NUMA affinity.
#if defined(GASNET_CONDUIT_IBV) || defined(GASNET_CONDUIT_UCX) || defined(GASNET_CONDUIT_ARIES) || defined(GASNET_CONDUIT_OFI)
#if defined(GASNET_CONDUIT_IBV)
#if defined(GASNET_SEGMENT_FAST)
gasnet_client_attach_hook = &chpl_comm_regMemHeapTouch;
#endif
#elif defined(GASNET_CONDUIT_UCX) || defined(GASNET_CONDUIT_ARIES) || defined(GASNET_CONDUIT_OFI)
#if defined(GASNET_SEGMENT_FAST) || defined(GASNET_SEGMENT_LARGE)
gasnet_client_attach_hook = &chpl_comm_regMemHeapTouch;
#endif
#endif

set_max_segsize();
Expand Down

0 comments on commit e629baa

Please sign in to comment.