[NativeAOT] cross-compiling to osx-arm64 from osx-x64 does not work

[jkotas]

Reported by @lemonmojo in #73678 (comment):

I'm also running into the same error using .NET 8 Preview 1.

While cross-compiling to x64 from an M1 machine works fine for me, I can't get the reverse (cross-compiling to ARM64 from an Intel machine) to work.

I'm not using any packages in this project. In fact, the issue is very easy to reproduce:

dotnet new classlib -n "AOTTest"
cd AOTTest
dotnet publish -p "PublishAot=true" -r osx-x64
dotnet publish -p "PublishAot=true" -r osx-arm64

Publishing using dotnet publish -r osx-x64 and dotnet publish -r osx-arm64 works fine from an M1 machine.
While on an Intel machine only the x64 build works and the ARM64 one fails with the error message:

/Users/redacted/.nuget/packages/microsoft.dotnet.ilcompiler/8.0.0-preview.1.23110.8/build/Microsoft.NETCore.Native.Publish.targets(74,5): error : The PrivateSdkAssemblies ItemGroup is required for _ComputeAssembliesToCompileToNative [/Path/to/Project.csproj].

So here's what works and what doesn't work in my testing:
ARM64 -> x64: OK
ARM64 -> ARM64: OK
x64 -> x64: OK
x64 -> ARM64: Error

[jkotas]

@filipnavara Do you happen to know what the problem is?

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Reported by @lemonmojo in #73678 (comment):

I'm also running into the same error using .NET 8 Preview 1.

While cross-compiling to x64 from an M1 machine works fine for me, I can't get the reverse (cross-compiling to ARM64 from an Intel machine) to work.

I'm not using any packages in this project. In fact, the issue is very easy to reproduce:

dotnet new classlib -n "AOTTest"
cd AOTTest
dotnet publish -p "PublishAot=true" -r osx-x64
dotnet publish -p "PublishAot=true" -r osx-arm64

Publishing using dotnet publish -r osx-x64 and dotnet publish -r osx-arm64 works fine from an M1 machine.
While on an Intel machine only the x64 build works and the ARM64 one fails with the error message:

/Users/redacted/.nuget/packages/microsoft.dotnet.ilcompiler/8.0.0-preview.1.23110.8/build/Microsoft.NETCore.Native.Publish.targets(74,5): error : The PrivateSdkAssemblies ItemGroup is required for _ComputeAssembliesToCompileToNative [/Path/to/Project.csproj].

So here's what works and what doesn't work in my testing:
ARM64 -> x64: OK
ARM64 -> ARM64: OK
x64 -> x64: OK
x64 -> ARM64: Error

Author:	jkotas
Assignees:	-
Labels:	untriaged, area-NativeAOT-coreclr
Milestone:	-

[am11]

While cross-compiling to x64 from an M1 machine works fine for me, I can't get the reverse (cross-compiling to ARM64 from an Intel machine) to work.

FWIW, the CI is cross-compiling for arm64 on osx-x64 machine.

[filipnavara]

It used to work, and this error is new to me. I'll try to reproduce it on my machine, but just in case that doesn't work, it would be useful to get a MSBuild binlog. It can be produced with a /bl switch on the dotnet publish command line, which will produce a msbuild.binlog file.

[kant2002]

I do have binlog from @lemonmojo and we troubleshoot that issue in #coreclr channel on dotnet evolution.
Issue was that runtime package was not downloaded as part of restore(?) process.
We solve that by manually adding package <PackageReference Include="runtime.osx-arm64.Microsoft.DotNet.ILCompiler" Version="8.0.0-preview.1.23110.8" />.
Discrepancies in differences quite possible was because ARM64 -> x64: OK was first attemp, and ARM64 machine has runtime package in the cache already.

[jkotas]

If you do not use explicit Microsoft.DotNet.ILCompiler package reference, it should just work.

The explicit runtime.osx-arm64.Microsoft.DotNet.ILCompiler package reference should be only needed if you use explicit Microsoft.DotNet.ILCompiler package reference. This is documented in https://github.com/dotnet/runtime/blob/main/src/coreclr/nativeaot/docs/compiling.md#cross-architecture-compilation.

[filipnavara]

It reproduces. The RuntimePackagePath property in ImportRuntimeIlcPackageTarget gets set to an empty value.

[kant2002]

I ask OP manually add this to download package in the cache. I think after this package in the Nuget cache issue will disappear.

[kant2002]

It obviosly still a bug, but I do not troubleshoot that far and I do not have Mac

[filipnavara]

The problem happens on this line:

runtime/src/coreclr/nativeaot/BuildIntegration/Microsoft.DotNet.ILCompiler.SingleEntry.targets

Line 51 in f295ea1

<RuntimePackagePath Condition="'@(ResolvedTargetILCompilerPack)' == '$(RuntimeIlcPackageName)'">@(ResolvedTargetILCompilerPack->'%(PackageDirectory)')</RuntimePackagePath>

ResolvedTargetILCompilerPack is set earlier:

It matches RuntimeIlcPackageName = runtime.osx-arm64.Microsoft.DotNet.ILCompiler.

However, there's no PackageDirectory property on the ResolvedTargetILCompilerPack item.

osx-arm64 binlog (fail):
msbuild.binlog.zip
osx-x64 binlog (success):
msbuild1.binlog.zip

[kant2002]

It happens because RuntimeIlcPackageName package not in the cache. Without that GetPackageDirectory cannot find path to package(it does not exists locally)

[filipnavara]

It happens because RuntimeIlcPackageName package not in the cache. Without that GetPackageDirectory cannot find path to package(it does not exists locally)

Yep, I agree with the assessment. It doesn't restore the NuGet of the target runtime package. If it already exists though then the error doesn't happen.

Here's a proof:

filipnavara@Filips-MacBook-Air AOTTest % ~/.dotnet/dotnet publish -p "PublishAot=true" -r osx-arm64 
MSBuild version 17.6.0-preview-23108-10+51df47643 for .NET
  Determining projects to restore...
  Restored /Users/filipnavara/Projects/AOTTest/AOTTest.csproj (in 537 ms).
/Users/filipnavara/.dotnet/sdk/8.0.100-preview.1.23115.2/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(287,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [/Users/filipnavara/Projects/AOTTest/AOTTest.csproj]
  AOTTest -> /Users/filipnavara/Projects/AOTTest/bin/Release/net8.0/osx-arm64/AOTTest.dll
  Generating native code
  AOTTest -> /Users/filipnavara/Projects/AOTTest/bin/Release/net8.0/osx-arm64/publish/
filipnavara@Filips-MacBook-Air AOTTest % rm -fr ~/.nuget
filipnavara@Filips-MacBook-Air AOTTest % ~/.dotnet/dotnet publish -p "PublishAot=true" -r osx-arm64 
MSBuild version 17.6.0-preview-23108-10+51df47643 for .NET
  Determining projects to restore...
  Restored /Users/filipnavara/Projects/AOTTest/AOTTest.csproj (in 1.88 sec).
/Users/filipnavara/.dotnet/sdk/8.0.100-preview.1.23115.2/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(287,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [/Users/filipnavara/Projects/AOTTest/AOTTest.csproj]
  AOTTest -> /Users/filipnavara/Projects/AOTTest/bin/Release/net8.0/osx-arm64/AOTTest.dll
/Users/filipnavara/.nuget/packages/microsoft.dotnet.ilcompiler/8.0.0-preview.1.23110.8/build/Microsoft.NETCore.Native.Publish.targets(74,5): error : The PrivateSdkAssemblies ItemGroup is required for _ComputeAssembliesToCompileToNative [/Users/filipnavara/Projects/AOTTest/AOTTest.csproj]

[jkotas]

This code should be responsible for downloading both x64 and arm64 packages for cross-compilation:

https://github.com/dotnet/sdk/blob/main/src/Tasks/Microsoft.NET.Build.Tasks/ProcessFrameworkReferences.cs#L668-L684

[filipnavara]

https://github.com/dotnet/sdk/blob/main/src/Tasks/Microsoft.NET.Build.Tasks/ProcessFrameworkReferences.cs#L668-L684

The output of the task has only the host targeting pack in the _PackageToDownload output despite the rest of the inputs and outputs being correct.

The branch that should do this actually doesn't have the code to add to the downloads.

[filipnavara]

I suppose the fix would be placing this somewhere around here:

                if (EnableRuntimePackDownload)
                {
                    // We need to download the target runtime pack
                    TaskItem runtimePackToDownload = new TaskItem(targetIlcPackName);
                    runtimePackToDownload.SetMetadata(MetadataKeys.Version, packVersion);
                    packagesToDownload.Add(runtimePackToDownload);
                }

[filipnavara]

I tested the fix with local build and it works, so I opened a PR.

[jkotas]

This specific cross-compilation scenario is fixed by #82558.

(If there are other cross-compilation scenarios to improve, we should open specific issues for them.)

[lemonmojo]

Thx @jkotas and @filipnavara! I'll test again when the next preview lands. Cheers