Implement custom memory manager for LLVM

[yuyichao]

Use various ways to reuse the page that has the page protection set in order to avoid wasting a page of memory for each JIT event.

Fixes #14626

Tested on Linux with LLVM 3.7.1 and 3.8 with OrcJIT.
This reduces the memory used in the subarray test by ~100MB (There are around 25k JIT events each page protects at least one page of memory even though the total allocation is only around 10-20MB.)

Should in principle work on OSX, using the mkstemp tmpfile fallback code path.

Does not work on windows yet due to how we handle windows unwinding (we patch the code section after the code is generated, causing a segfault since the load address is never writable). It might be enough to modify those code to write to local address instead of load address. @Keno also mentioned some way to fix this properly. (This is what the request for help is mainly for....) Fixed

[yuyichao]

Another note is that LLVM 3.7.1 seems to generate corrupted eh_frame when using Keno's remote memory manager. Not sure why that happens but my current suspicion is the reuse of local address. Fixed

[vtjnash]

also mentioned some way to fix this properly. (This is what the request for help is mainly for....)

at present, this has been blocked on the need to write our own custom memory manager (specifically, one that supports the small memory model required by ntdll on win64 for all dynamic libraries), and then patch the LLVM runtime relocation code to add support for the ADDR32NB relocation type.

[yuyichao]

specifically, one that supports the small memory model required by ntdll on win64 for all dynamic libraries

What's the requirement? All sections to be smaller and within 4G to each other? The current custom memory manager doesn't do that yet but it's not that hard to add (especially if we can use the reserve function).

[vtjnash]

yes, and consecutively allocated (images must be non-overlapping and can't be interleaved)

[yuyichao]

(images must be non-overlapping and can't be interleaved)

Doesn't this require the current allocation scheme? The page' where we have allocations in basically can't be reused anymore. (edit: or no W^X)

[yuyichao]

Or maybe we can waste virtual memory but not physical memory?

[vtjnash]

I'm not sure. It appears that the actual requirements are undocumented and considered to be an implementation detail. We may be able to get away with some lying, since we are JIT and not statically linked. But unfortunately, there's no way to verify the source code of ntdll to learn it's expectations.

[Keno]

Why would there be an expectation that the sections are non-interleaved?

[vtjnash]

Actually, maybe only dbghelp cares (https://msdn.microsoft.com/en-us/library/windows/desktop/ms681353(v=vs.85).aspx), so perhaps it would only cause problems for WinDbg, but not ntdll.

[Keno]

Move this to codegen_internal.h as well.

[yuyichao]

I think I addressed all the comments apart from the one about cache flushing. I was hoping that mprotect will flush the write to the memory and calling the LLVM helper will flush the instruction cache for the runtime address. Not sure what's still missing.

[Keno]

@yuyichao I think this should be merged. Will you do the honor?

[yuyichao]

Sure, I was just waiting for one of you to merge it ;-p

[Keno]

This may be triggering a compiler bug in gcc 4.8, complaining that the copy constructor of Block was deleted (even though the use in question should have been a move constructor). We may have to comment out that deletion for the GCC 4.8 series.

[yuyichao]

Where does it need a copy constructor for Block (whether move or not)? temp_buff.emplace_back()?

[tkelman]

See buildbot log if it's more useful - https://build.julialang.org/builders/build_ubuntu14.04-x64/builds/940/steps/shell_1/logs/stdio
also quick to reproduce / test things since it only takes a couple minutes to hit.

[Keno]

SmallVector.

[yuyichao]

That one is supposed to use a placement new with the default constructor... I guess the LLVM 3.7 SmallVector implementation might be constructing a Block and use the placement new to move..... GCC 6.1.1 didn't complain though......

[Keno]

What default constructor would it be using? How would it get the Block into its storage without using the move constructor?

[yuyichao]

new (ptr) Block()? The default constructor with no argument is provided.

[Keno]

So you were relying on that particular function not being instantiated because you're not using it?

[yuyichao]

And I want to make sure I don't copy/move it accidentally since it can cause memory leak or memory free'd at the wrong time.

[Keno]

Make the move constructor assert(false)?

[yuyichao]

Actually moving construct is always fine, moving assignment is not and is deleted. I've updated #16950 to implement the moving constructor properly.

[tkelman]

need to check before using O_CLOEXEC https://build.julialang.org/builders/package_tarball64/builds/475/steps/make%20binary-dist/logs/stdio

[yuyichao]

That kernel is actually too old for this method to work although I guess we should still compile this method for this to be included in the generic binary.... Should be fixed in 3f4e1e7

[JeffBezanson]

BTW, the memory savings from this is absolutely great. Awesome work @yuyichao !