SDL API not loading on M-series Mac

[joskuijpers]

Summary

I am unable to load the SDL API on my Mac, even though the same code works on Windows.

Steps to reproduce

• Platform: Desktop, MacBook Pro M-series
• Framework Version: .NET Core 8
• API: SDL
  Added Silk.NET.SDL to my project
  Then with this code, this works fine on a Windows 64bit (Win10) machine:

    SdlProvider.InitFlags = Sdl.InitVideo | Sdl.InitEvents;
    var sdl = SdlProvider.SDL.Value;

    WindowFlags flags = WindowFlags.AllowHighdpi;    
    var windowHandle = sdl.CreateWindow("Hello World", Sdl.WindowposUndefined, Sdl.WindowposUndefined, 640, 480,      (uint)flags);

while(true) {}

On my MBP, it fails in creating the API (also when using GetApi). It exits before it reaches the line after acquiring the API.
I've looked at the resource folder: libSDL2-2.0.dylib is available in runtimes/osx/native, the rest of the Silk libraries are in runtimes/osx-arm64.

Comments

Maybe the library is not being found in the right place? I tried to find how the build system works for ARM but not successfully yet.

[joskuijpers]

The way https://github.com/dotnet/Silk.NET/blob/main/src/Native/Silk.NET.SDL.Native/build/net461/Ultz.Native.SDL.targets is set up, it should set 'osx-x64' even on ARM (due to the 64bit check). This means it will look in runtimes/osx-x64/native. It is not there.

[Perksey]

Are you using .NET Framework/Mono? That targets file does not apply otherwise.

[alexrp]

If you are not using Framework/Mono, I would be curious if you could try moving the dylib into runtimes/osx-arm64 and see if it works then. If so, then this is indeed a bug in our packaging.

[joskuijpers]

I am not no. :D I barely know what I am doing on the native part.

https://github.com/dotnet/Silk.NET/pull/1645/files#diff-39bb3db1dc156c384f572851e6b01ecb0c6202605e127a734446753414bde82a

I see here that the libs got added here, but no config for OSX

[Perksey]

Does it work if you use dotnet publish or dotnet run with a runtime identifier specified?

[joskuijpers]

Moved the file, but it gets put back on Run. Any tip on how I can try this properly?

[joskuijpers]

Does it work if you use dotnet publish or dotnet run with a runtime identifier specified?

dotnet run -r osx-arm64 -> same issue
dotnet run -r osx-x64 -> wrong CPU error (to make sure...)
dotnet run -r osx -> unknown runtime

[alexrp]

Do something like this:

$ dotnet build
$ cp bin/Debug/net8.0/runtimes/osx/native/libSDL2-2.0.dylib bin/Debug/net8.0/runtimes/osx-arm64/native # replace net8.0 as appropriate
$ dotnet run --no-build

Does this work? (Basically trying to determine if the runtime is even attempting to pick up anything from runtimes/osx.)

[joskuijpers]

% dotnet run --no-build
A
B

Exits with Error code 137 (SIGKILL)

No :( With this code:

unsafe
{
    Console.WriteLine("A");
    SdlProvider.InitFlags = Sdl.InitVideo | Sdl.InitEvents;
    Console.WriteLine("B");
    var sdl = SdlProvider.SDL.Value;
    Console.WriteLine("C");
 }

Sidenote: my osx/ also contains libSkiaSharp, which works fine.

[joskuijpers]

Oh... I remembered now to look at the system Console:

Exception Type: EXC_CRASH (SIGKILL (Code Signature Invalid))
Exception Codes: 0x0000000000000001, 0x0000000000000000

Termination Reason: Namespace CODESIGNING, Code 2 Invalid Page

[alexrp]

What's the output of file runtimes/osx/native/libSDL2-2.0.dylib?

[joskuijpers]

That one is both x64 and arm, but:

codesign --verify bin/Debug/net8.0/runtimes/osx/native/libSDL2-2.0.dylib
bin/Debug/net8.0/runtimes/osx/native/libSDL2-2.0.dylib: invalid signature (code or signature have been modified)
In architecture: arm64

(fine for Skia and libwgpu (from your WebGPU))

[alexrp]

I don't really know how code signing works on macOS. Is there a way to turn off the enforcement just to see if that's the cause of the issue?

[joskuijpers]

I've signed it myself (overwriting the existing signature), and then it runs. So it is definitely the wrong signature.

[alexrp]

Did you do ad-hoc signing or use an actual certificate?

[joskuijpers]

Actual certificate in this case. Let me find how to sign ad-hoc...
I do seem to understand clang auto-signs when it builds on macOS, so that might explain why the wgpu binary is signed correctly.

EDIT: Ad-hoc (using '-' as certificate) also works fine

[joskuijpers]

'broken' signature:

Identifier=libSDL2-55554944bb706839cb39394db041064fbfb14ef3
Format=Mach-O universal (x86_64 arm64)
CodeDirectory v=20400 size=11881 flags=0x2(adhoc) hashes=362+5 location=embedded
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements count=0 size=12

After I signed it adhoc:

Identifier=libSDL2-2-55554944bb706839cb39394db041064fbfb14ef3
Format=Mach-O universal (x86_64 arm64)
CodeDirectory v=20400 size=11787 flags=0x2(adhoc) hashes=362+2 location=embedded
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements count=0 size=12

[alexrp]

Well, we also build SDL on macOS (using xcodebuild even), so the resulting binaries should be ad-hoc signed. My current suspicion is that the signature for the Arm64 portion somehow gets corrupted when we lipo the x64 and Arm64 binaries together. 🤔

I don't have a macOS system, though, so hard for me to investigate this.

[joskuijpers]

I'm glad to help you figure it out :)

To be sure, I've renamed the file to something else, and an existing signature survives that. (I am not sure myself what the signature signs). So the identifier being libSDL2 instead of 2-2 should not be an issue.
A difference I see is the 'size'.

I'd think you need to sign after lipo?

Can I try building this whole thing locally?

[alexrp]

I'd think you need to sign after lipo?

That may be it. My understanding is that each arch portion of a fat binary has its own signature, but maybe you also need to sign the combined fat binary at the end... 🤔

Can I try building this whole thing locally?

Clone the repo and do git submodule update --init --recursive build/submodules/SDL, followed by ./build.sh sdl2 to perform the build.

• This just builds the SDL native binaries, so you shouldn't need anything except .NET SDK 7 (for NUKE) and Xcode.
• Do not clone the entire repository recursively since we have a lot of huge submodules (and some of them don't even clone cleanly).

You can find the build logic here: https://github.com/dotnet/Silk.NET/blob/main/build/nuke/Native/SDL2.cs

[joskuijpers]

Any chance I can forcably use dotnet sdk 8?

EDIT: Bunch of env changes got me access to 7 now. Building...

[joskuijpers]

Last line of the build: :)

[ERR] SDL2: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip: warning: changes being made to the file will invalidate the code signature in: /Users/joskuijpers/Developer/Silk.NET/src/Native/Silk.NET.SDL.Native/runtimes/osx/native/libSDL2-2.0.dylib (for architecture x86_64)
[ERR] SDL2: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip: warning: changes being made to the file will invalidate the code signature in: /Users/joskuijpers/Developer/Silk.NET/src/Native/Silk.NET.SDL.Native/runtimes/osx/native/libSDL2-2.0.dylib (for architecture arm64)

[alexrp]

You can have multiple .NET SDKs installed side-by-side. One way is to use the dotnet-install scripts, e.g. dotnet-install -v <sdk version>.

[Perksey]

That'd do it 🤦

[joskuijpers]

You can have multiple .NET SDKs installed side-by-side. One way is to use the dotnet-install scripts, e.g. dotnet-install -v <sdk version>.

Yeah, but one install was through brew, so it kept taking the brew version. Should switch to botnet-install...

[joskuijpers]

Interestingly:

codesign --verify src/Native/Silk.NET.SDL.Native/runtimes/osx/native/libSDL2-2.0.dylib

succeeds

[alexrp]

Hm. I mean, it does make sense that strip would invalidate the signature if it doesn't re-ad-hoc-sign it.

But codesign --verify then succeeding? That's strange. Does macOS have some special-casing going on for binaries built on the current system?

[joskuijpers]

I have no idea. I can make a PR with extra code-signing at the end. Then I think Github can try running it, and I can verify

[alexrp]

That's a bit involved due to the way our native builds are triggered (it's not as simple as a PR). But, I think @Perksey daily-drives macOS, so maybe you can upload the binary you just built for him to codesign --verify? If we can confirm that it verifies on your system only because you built it, then I think we have sufficient information to know that we indeed just need to ad-hoc sign after stripping.

[joskuijpers]

Alright, I can do that. I made a PR anyways: #2174

[joskuijpers]

@Perksey I have uploaded the files to https://github.com/joskuijpers/Silk.NET/tree/testsign/src/Native/Silk.NET.SDL.Native/runtimes/osx/native

[joskuijpers]

For now I have a workaround:

<Target Name="Codesign" AfterTargets="Build">
    <Exec Command="codesign --force --sign - $(TargetDir)/runtimes/osx/native/libSDL2-2.0.dylib" />
</Target>

[joskuijpers]

Thank you for the super quick responses!

[alexrp]

Thanks for helping diagnose and fix it!