Add objwriter from the runtime repo #153
Conversation
* Taken from https://github.com/dotnet/runtime/tree/7ab969c84ef05ba948c0075392716ce335b47744/src/coreclr/tools/aot/ObjWriter. * Updated README.md * Updated CMakeLists.txt to remove reference to CORECLR_INCLUDE_DIR. * Added cordebuginfo.h, cvconst.h, cfi.h from coreclr/inc at the above commit.
MichalStrehovsky
force-pushed
the
ob
branch
2 times, most recently
from
266ebf7 to
72e99c0
Compare
|
@marek-safar Could you please merge this when you get a chance (please don't squash)? The only interesting commit is 94cc457, the rest of the commits are just copying files around from either runtime repo or release/11.x branch. And even that remaining commit is not very interesting. After thinking about it a bit more, I think it will make sense to standardize on a single branch once/if Mono moves to LLVM 12/13 or later version. I didn't want to spend time downgrading ObjWriter to LLVM 11 especially because I just did an upgrade to 12 in the summer and I'm not sure how many local patches would I need to restore again to go back to 11. |
|
/cc @akoeplinger |
|
👍 |
| return Value; | ||
| } | ||
| } | ||
|
|
||
| Optional<MCFixupKind> AArch64AsmBackend::getFixupKind(StringRef Name) const { | ||
| if (!TheTriple.isOSBinFormatELF()) | ||
| return None; | ||
| if (!TheTriple.isOSBinFormatELF()) { |
There was a problem hiding this comment.
@MichalStrehovsky, is it applicable to TheTriple.isOSBinFormatMachO() as well or only isOSBinFormatCOFF? I am not sure if it would help what I am trying to achieve, but if it only applies to windows, we can use if (TheTriple.isOSBinFormatCOFF) and let it return None on other platforms. 👀
ctx: I'm trying to enable nativeaot on osx-arm64, while this minimal change gets it to compilable (and artifacts/packages/Release/Shipping/Microsoft.DotNet.ILCompiler.7.0.0-dev.nupkg buildable) state, I get tons of diagnostics when consuming that nupkg with dotnet7 publish ...:
21:06:05.007 1:7>Target "IlcCompile: (TargetId:181)" in file "/Users/am11/.nuget/packages/microsoft.dotnet.ilcompiler/7.0.0-dev/build/Microsoft.NETCore.Native.targets" from project "/Users/am11/projects/naot1/naot1.csproj" (target "LinkNative" depends on it):
Building target "IlcCompile" completely.
Output file "obj/release/net7.0/osx-arm64/native/naot1.o" does not exist.
Task "Message" skipped, due to false condition; ($(_BuildingInCompatibleMode) != 'true') was evaluated as (true != 'true').
Task "Message" (TaskId:126)
Task Parameter:Text=Generating compatible native code. To optimize for size or speed, visit https://aka.ms/OptimizeCoreRT (TaskId:126)
Task Parameter:Importance=high (TaskId:126)
Generating compatible native code. To optimize for size or speed, visit https://aka.ms/OptimizeCoreRT (TaskId:126)
Done executing task "Message". (TaskId:126)
Task "Exec" (TaskId:127)
Task Parameter:Command="/Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/tools/ilc" @"obj/release/net7.0/osx-arm64/native/naot1.ilc.rsp" (TaskId:127)
"/Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/tools/ilc" @"obj/release/net7.0/osx-arm64/native/naot1.ilc.rsp" (TaskId:127)
<unknown>:0: error: ADR/ADRP relocations must be GOT relative (TaskId:127)
<unknown>:0: error: unknown AArch64 fixup kind! (TaskId:127)
<unknown>:0: error: unknown AArch64 fixup kind! (TaskId:127)
<unknown>:0: error: fixup value out of range (TaskId:127)
<unknown>:0: error: ADR/ADRP relocations must be GOT relative (TaskId:127)
<unknown>:0: error: unknown AArch64 fixup kind! (TaskId:127)
<unknown>:0: error: unknown AArch64 fixup kind! (TaskId:127)
<unknown>:0: error: fixup value out of range (TaskId:127)
... continues for 1000s of lines ...
and the next (clang) task fails:
Set Property: _IgnoreLinkerWarnings=false
Set Property: _IgnoreLinkerWarnings=true
Task "Exec" (TaskId:129)
Task Parameter:IgnoreStandardErrorWarningFormat=True (TaskId:129)
Task Parameter:Command=clang "obj/release/net7.0/osx-arm64/native/naot1.o" -o "bin/release/net7.0/osx-arm64/native/naot1" /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/sdk/libbootstrapper.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/sdk/libRuntime.WorkstationGC.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Native.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Globalization.Native.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.IO.Compression.Native.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Net.Security.Native.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Security.Cryptography.Native.Apple.a -g -Wl,-rpath,'@executable_path' -lstdc++ -ldl -lm -lz -licucore -framework CoreFoundation -framework Foundation -framework Security -framework GSS (TaskId:129)
clang "obj/release/net7.0/osx-arm64/native/naot1.o" -o "bin/release/net7.0/osx-arm64/native/naot1" /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/sdk/libbootstrapper.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/sdk/libRuntime.WorkstationGC.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Native.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Globalization.Native.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.IO.Compression.Native.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Net.Security.Native.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Security.Cryptography.Native.Apple.a -g -Wl,-rpath,'@executable_path' -lstdc++ -ldl -lm -lz -licucore -framework CoreFoundation -framework Foundation -framework Security -framework GSS (TaskId:129)
ld: malformed __LD,__compact_unwind section, bad length file 'obj/release/net7.0/osx-arm64/native/naot1.o' (TaskId:129)
clang: error: linker command failed with exit code 1 (use -v to see invocation) (TaskId:129)
21:06:12.873 1:7>/Users/am11/.nuget/packages/microsoft.dotnet.ilcompiler/7.0.0-dev/build/Microsoft.NETCore.Native.targets(337,5): error MSB3073: The command "clang "obj/release/net7.0/osx-arm64/native/naot1.o" -o "bin/release/net7.0/osx-arm64/native/naot1" /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/sdk/libbootstrapper.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/sdk/libRuntime.WorkstationGC.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Native.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Globalization.Native.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.IO.Compression.Native.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Net.Security.Native.a /Users/am11/.nuget/packages/runtime.osx-arm64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Security.Cryptography.Native.Apple.a -g -Wl,-rpath,'@executable_path' -lstdc++ -ldl -lm -lz -licucore -framework CoreFoundation -framework Foundation -framework Security -framework GSS" exited with code 1. [/Users/am11/projects/naot1/naot1.csproj]
Done executing task "Exec" -- FAILED. (TaskId:129)
21:06:12.873 1:7>Done building target "LinkNative" in project "naot1.csproj" -- FAILED.: (TargetId:182)
There was a problem hiding this comment.
I think it's applicable to TheTriple.isOSBinFormatMachO as well.
On a high level, the objwriter project uses LLVM's APIs for assemblers to generate object files. When I say assemblers, I mean the compilers that translate textual assembly to object files. It's a bit of a misuse, but it works. We have a couple patches to LLVM like this one to work around the fact that the API is not complete enough for our purposes.
This specific spot is working around the fact that LLVM-based assemblers only support the .reloc assemler directive for ELF object files. If you look at lines 348-355 in this file, you'll see that if this if check is not taken, we're going to do some very ELF-specific things that translate the textual string to an ELF-specific reloc (the FirstLiteralRelocationKind + Type part). The ELF-specific reloc is not usable outside of ELF. One really wonders why this code is in AArch64AsmBackend and not in ELFAArch64AsmBackend, but I digress.
This if check is generating LLVM's target-independent relocs instead of relocs that start at FirstLiteralRelocationKind. LLVM target-independent relocs get translated to a physical relocation type by the object writing backend.
There was a problem hiding this comment.
There are a couple warnings from ld that don't seem fatal on x64 macOS as well - if you see them on arm64 they're noise and you can ignore them for the purposes of arm64: dotnet/runtimelab#956
There was a problem hiding this comment.
Thanks you for the pointers @MichalStrehovsky. In this case, ilc was failing after objwriter finish emitting the object (and before clang). I have opened dotnet/runtime#67232 to discuss the reloc issues. Hoping someone might have encountered the same on this platform and clue us in. :)
(Individual commits have descriptions - please don't squash.)