Set high water mark for GC at 70% of available memory and only then start performing old generation gcs.

[chflood]

…tely.

This should address some of the recent OOM errors we have been seeing.

[Keno]

This isn't the calculation you want. This only checks what the OS imposes. Instead, we should look at the maxrss of the current process and compare that to the heap size we know about and use that to make an informed decision about how much memory is being stolen by non-managed allocators. It is of course possible for the OS configuration to change over the lifetime of a process, but it's somewhat unlikely and certainly not the cause of observed issues.

[chflood]

Yes, I switched that to uv_get_free_memory which does sort of what we want, but apparently not quite because it says I have 300m available but I get an OOM when allocating 40m. I suspect I'm going to have to do some approximating to figure out what a reasonable allocation upper bound is.

…

On Tue, May 24, 2022 at 3:37 PM Keno Fischer ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In src/gc.c <#45442 (comment)>: > @@ -3170,6 +3170,20 @@ static int _jl_gc_collect(jl_ptls_t ptls, jl_gc_collection_t collection) } } + // Readjust the max_total_memory based on what is currently available. + // LLVM or C may have allocated some of the memory since the last time + // we checked. + + uint64_t total_mem = uv_get_total_memory(); + uint64_t constrained_mem = uv_get_constrained_memory(); This isn't the calculation you want. This only checks what the OS imposes. Instead, we should look at the maxrss of the current process and compare that to the heap size we know about and use that to make an informed decision about how much memory is being stolen by non-managed allocators. It is of course possible for the OS configuration to change over the lifetime of a process, but it's somewhat unlikely and certainly not the cause of observed issues. — Reply to this email directly, view it on GitHub <#45442 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE3CLBFABJXA3EOUSC75MXTVLUVY5ANCNFSM5W2LAIUA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[KristofferC]

I'll merge this now since it improves things in some cases I have seen where previously the OOM killer would otherwise kill the process.