Investigate ghc.exe: getMBlocks: VirtualAlloc MEM_COMMIT failed: The paging file is too small for this operation to complete CI failures

[RyanGlScott]

The SAWScript.Intepreter file is one of the largest in the SAW codebase, currently clocking in at 4765 lines of code. What's more, most of this file consists of the monolithic primitives table, which consists of thousands of string literals that document all of the SAWScript commands. While no single entry in primitives is likely that taxing to compile times in isolation, each little bit does add up every time a new SAWScript command is added.

Things are finally at a tipping point, however, as the amount of RAM required to compile SAWScript.Intepreter is causing the CI to crash. More specifically, in pull request #1958, which adds two commands to primitives, the Windows GHC 8.10.7 CI job has been observed to fail with this error consistently:

[94 of 94] Compiling SAWScript.Interpreter ( src\SAWScript\Interpreter.hs, D:\a\saw-script\saw-script\dist-newstyle\build\x86_64-windows\ghc-8.10.7\saw-script-1.0.0.99\build\SAWScript\Interpreter.o )
ghc.exe: getMBlocks: VirtualAlloc MEM_COMMIT failed: The paging file is too small for this operation to complete.

Ouch. While it is curious that this error only happens with the Windows GHC 8.10.7 job (and none of the other Windows jobs), this suggests that the current status quo is fragile. Googling for this error message suggests that there is not much we can do besides using less RAM (and indeed, the GitHub Actions CI machines are somewhat constrained on RAM).

One thing that we could quite easily do to reduce RAM requirements is to split up the primitives table a bit. It currently contains every SAWScript definition, but we could quite easily divide it into smaller subtables. As a starting point, we could have separate tables for the LLVM-related commands, JVM-related commands, MIR-related commands, Heapster-related commands, etc., and then primitives would consist of the union of all these tables. This would likely have a big impact on maximum RAM requirements without too much effort, and it worth a try before considering more drastic measures, such as putting the commands' documentation in external files (as suggested here).

[RyanGlScott]

Upon a closer look, I'm not entirely sure that SAWScript.Intepreter is to blame here. It turns out that the CI on the master branch of SAW is already triggering this error, as seen here:

[86 of 94] Compiling SAWScript.Builtins ( src\SAWScript\Builtins.hs, D:\a\saw-script\saw-script\dist-newstyle\build\x86_64-windows\ghc-8.10.7\saw-script-1.0.0.99\build\SAWScript\Builtins.o )
ghc.exe: getMBlocks: VirtualAlloc MEM_COMMIT failed: The paging file is too small for this operation to complete.

This time, the error arises in a different module. Urk.

Why doesn't the master branch's CI fail as a result of this? It's because the CI retries the cabal build twice in case of spurious errors, and it just so happens that if you retry cabal build after this error on the master branch, then GHC somehow succeeds on the second attempt. For whatever reason, this does not occur in #1958, as retrying cabal build does not get the build unstuck. Very bizarre.

In light of this, it's much less clear to me what is going on.

[RyanGlScott]

A plausible explanation for why the Windows GHC 9.2 and 9.4 jobs do not suffer from this issue is that those versions of GHC use Windows' large address space allocator (see GHC#12576), whereas GHC 8.10 does not. While it is still a mystery to me why this class of build error has only recently started to appear in Windows GHC 8.10 jobs, it is unclear if it is worth the effort to debug this further, given that GHC 9.2+ has a more principled way of allocating large addresses.

In light of this, I propose that we:

• Change the CI so that the SAW binary artifacts are produced using GHC 9.2 rather than 8.10.
• Mark the Windows GHC 8.10 as allowed to fail for the time being, as it will no longer be critical for that job to succeed.
• When we update SAW to support GHC 9.6 (and therefore move our GHC support window), we can finally drop all vestiges of GHC 8.10 from the CI.