Recent WASM package have some issues. #2196

kant2002 · 2023-02-09T19:42:39Z

I update version of packages in my test project to

<PackageReference Include="Microsoft.DotNet.ILCompiler.LLVM" Version="7.0.0-preview.5.23106.1" />
<PackageReference Include="runtime.win-x64.Microsoft.DotNet.ILCompiler.LLVM" Version="7.0.0-preview.5.23106.1" />

and now have issues.

c:\cache\nuget\packages\microsoft.dotnet.ilcompiler.llvm\7.0.0-preview.5.23106.1\build\Microsoft.DotNet.ILCompiler.targets(55,5): error : The PackageDefinitions 
ItemGroup is required for target ImportRuntimeIlcPackageTarget [C:\github\NativeAOTWasm\NativeAOTWasm.csproj]

I manage to fix this using following chages

<PropertyGroup>
    <PublishAot>true</PublishAot>
</PropertyGroup>

<ItemGroup>
      <KnownILCompilerPack Update="Microsoft.DotNet.ILCompiler"
                         TargetFramework="net7.0"
                         ILCompilerPackNamePattern="runtime.win-x64.Microsoft.DotNet.ILCompiler.LLVM"
                         ILCompilerPackVersion="7.0.0-preview.5.23106.1"
                         ILCompilerRuntimeIdentifiers="browser-wasm;linux-musl-x64;linux-x64;win-x64;linux-arm;linux-arm64;linux-musl-arm;linux-musl-arm64;osx-arm64;osx-x64;win-arm;win-arm64;win-x86"
                         />
</ItemGroup>

and adding to file microsoft.dotnet.ilcompiler.llvm\7.0.0-preview.5.23106.1\build\Microsoft.NETCore.Native.Browser.props

<PropertyGroup>
   <!-- Should we have Microsoft.DotNet.ILCompiler.SingleEntry.targets ? -->
      <ILCompilerTargetsPath>$(MSBuildThisFileDirectory)Microsoft.DotNet.ILCompiler.targets</ILCompilerTargetsPath>
</PropertyGroup>

Even after that I receive following error

EXEC : error : One or more errors occurred. (Code generation failed for method '[S.P.CoreLib]Internal.Runtime.CompilerHelpers.ThrowHelpers.ThrowInstanceBodyRemoved()'
) (Code generation failed for method '[S.P.CoreLib]System.Runtime.EH.FailedAllocation(EETypePtr,bool)') (Code generation failed for method '[S.P.CoreLib]System.Runtim
e.TypeCast.LdelemaRef(Array,int32,native int)') (Code generation failed for method '[S.P.CoreLib]System.Runtime.RuntimeExports.RhUnboxAny(object,uint8&,EETypePtr)') [ 
C:\d\github\NativeAOTWasm\NativeAOTWasm.csproj]
  System.AggregateException: One or more errors occurred. (Code generation failed for method '[S.P.CoreLib]Internal.Runtime.CompilerHelpers.ThrowHelpers.ThrowInstance
  BodyRemoved()') (Code generation failed for method '[S.P.CoreLib]System.Runtime.EH.FailedAllocation(EETypePtr,bool)') (Code generation failed for method '[S.P.CoreL
  ib]System.Runtime.TypeCast.LdelemaRef(Array,int32,native int)') (Code generation failed for method '[S.P.CoreLib]System.Runtime.RuntimeExports.RhUnboxAny(object,uin 
  t8&,EETypePtr)')
   ---> ILCompiler.CodeGenerationFailedException: Code generation failed for method '[S.P.CoreLib]Internal.Runtime.CompilerHelpers.ThrowHelpers.ThrowInstanceBodyRemov 
  ed()'
   ---> System.InvalidOperationException: Expected method 'DispatchExLLVM' not found on type '[S.P.CoreLib]System.Exception'
     at Internal.IL.HelperExtensions.GetKnownMethod(TypeDesc type, String name, MethodSignature signature)
     at Internal.IL.ILImporter.ImportThrow()
     at Internal.IL.ILImporter.ImportBasicBlock(BasicBlock basicBlock)
     at Internal.IL.ILImporter.ImportBasicBlocks()
     at Internal.IL.ILImporter.Import()
     at ILCompiler.ILScanner.CompileSingleMethod(ScannedMethodNode methodCodeNodeNeedingCode)
     --- End of inner exception stack trace ---
     at ILCompiler.ILScanner.CompileSingleMethod(ScannedMethodNode methodCodeNodeNeedingCode)
     at System.Threading.Tasks.Parallel.<>c__DisplayClass19_0`1.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCo 
  mpletion)
  --- End of stack trace from previous location ---
     at System.Threading.Tasks.Parallel.<>c__DisplayClass19_0`1.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCo 
  mpletion)
     at System.Threading.Tasks.TaskReplicator.Replica.Execute()
     --- End of inner exception stack trace ---
     at System.Threading.Tasks.TaskReplicator.Run[TState](ReplicatableUserAction`1 action, ParallelOptions options, Boolean stopOnFirstFailure)
     at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithSta 
  te, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally)
  --- End of stack trace from previous location ---
     at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithSta 
  te, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally)
     at ILCompiler.ILScanner.CompileMultiThreaded(List`1 methodsToCompile)
     at ILCompiler.ILScanner.ComputeDependencyNodeDependencies(List`1 obj)
     at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.ComputeMarkedNodes()
     at ILCompiler.ILScanner.ILCompiler.IILScanner.Scan()
     at ILCompiler.Program.Run(String[] args)
     at ILCompiler.Program.Main(String[] args)
   ---> (Inner Exception #1) ILCompiler.CodeGenerationFailedException: Code generation failed for method '[S.P.CoreLib]System.Runtime.EH.FailedAllocation(EETypePtr,bo 
  ol)'
   ---> System.InvalidOperationException: Expected method 'DispatchExLLVM' not found on type '[S.P.CoreLib]System.Exception'
     at Internal.IL.HelperExtensions.GetKnownMethod(TypeDesc type, String name, MethodSignature signature)
     at Internal.IL.ILImporter.ImportThrow()
     at Internal.IL.ILImporter.ImportBasicBlock(BasicBlock basicBlock)
     at Internal.IL.ILImporter.ImportBasicBlocks()
     at Internal.IL.ILImporter.Import()
     at ILCompiler.ILScanner.CompileSingleMethod(ScannedMethodNode methodCodeNodeNeedingCode)
     --- End of inner exception stack trace ---
     at ILCompiler.ILScanner.CompileSingleMethod(ScannedMethodNode methodCodeNodeNeedingCode)
     at System.Threading.Tasks.Parallel.<>c__DisplayClass19_0`1.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCo 
  mpletion)
  --- End of stack trace from previous location ---
     at System.Threading.Tasks.Parallel.<>c__DisplayClass19_0`1.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCo 
  mpletion)
     at System.Threading.Tasks.TaskReplicator.Replica.Execute()<---

   ---> (Inner Exception #2) ILCompiler.CodeGenerationFailedException: Code generation failed for method '[S.P.CoreLib]System.Runtime.TypeCast.LdelemaRef(Array,int32, 
  native int)'
   ---> System.InvalidOperationException: Expected method 'DispatchExLLVM' not found on type '[S.P.CoreLib]System.Exception'
     at Internal.IL.HelperExtensions.GetKnownMethod(TypeDesc type, String name, MethodSignature signature)
     at Internal.IL.ILImporter.ImportThrow()
     at Internal.IL.ILImporter.ImportBasicBlock(BasicBlock basicBlock)
     at Internal.IL.ILImporter.ImportBasicBlocks()
     at Internal.IL.ILImporter.Import()
     at ILCompiler.ILScanner.CompileSingleMethod(ScannedMethodNode methodCodeNodeNeedingCode)
     --- End of inner exception stack trace ---
     at ILCompiler.ILScanner.CompileSingleMethod(ScannedMethodNode methodCodeNodeNeedingCode)
     at System.Threading.Tasks.Parallel.<>c__DisplayClass19_0`1.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCo 
  mpletion)
  --- End of stack trace from previous location ---
     at System.Threading.Tasks.Parallel.<>c__DisplayClass19_0`1.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCo 
  mpletion)
     at System.Threading.Tasks.TaskReplicator.Replica.Execute()<---

   ---> (Inner Exception #3) ILCompiler.CodeGenerationFailedException: Code generation failed for method '[S.P.CoreLib]System.Runtime.RuntimeExports.RhUnboxAny(object 
  ,uint8&,EETypePtr)'
   ---> System.InvalidOperationException: Expected method 'DispatchExLLVM' not found on type '[S.P.CoreLib]System.Exception'
     at Internal.IL.HelperExtensions.GetKnownMethod(TypeDesc type, String name, MethodSignature signature)
     at Internal.IL.ILImporter.ImportThrow()
     at Internal.IL.ILImporter.ImportBasicBlock(BasicBlock basicBlock)
     at Internal.IL.ILImporter.ImportBasicBlocks()
     at Internal.IL.ILImporter.Import()
     at ILCompiler.ILScanner.CompileSingleMethod(ScannedMethodNode methodCodeNodeNeedingCode)
     --- End of inner exception stack trace ---
     at ILCompiler.ILScanner.CompileSingleMethod(ScannedMethodNode methodCodeNodeNeedingCode)
     at System.Threading.Tasks.Parallel.<>c__DisplayClass19_0`1.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCo
  mpletion)
  --- End of stack trace from previous location ---
     at System.Threading.Tasks.Parallel.<>c__DisplayClass19_0`1.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCo
  mpletion)
     at System.Threading.Tasks.TaskReplicator.Replica.Execute()<---

c:\cache\nuget\packages\microsoft.dotnet.ilcompiler.llvm\7.0.0-review.5.23106.1\build\Microsoft.NETCore.Native.targets(293,5): error MSB3073: The command ""c:\cache\nuget\packages\runtime.win-x64.microsoft.dotnet.ilcompiler.llvm\7.0.0-preview.5.23106.1\tools\\ilc" @"obj\Debug\net7.0\browser-wasm\native\NativeAOTWasm.ilc.rsp"" exited with code 1. [C:\github\NativeAOTWasm\NativeAOTWasm.csproj]

/cc @yowl and @SingleAccretion

The text was updated successfully, but these errors were encountered:

kant2002 · 2023-02-09T19:59:26Z

Maybe solution to this is publish of the runtime.browser-wasm.Microsoft.DotNet.ILCompiler.LLVM package. Because I see that
https://github.dev/dotnet/runtimelab/blob/e467ad07e897213123e5e529c979691f2ab89926/src/coreclr/nativeaot/System.Private.CoreLib/src/System.Private.CoreLib.csproj#L218
and
https://github.dev/dotnet/runtimelab/blob/e467ad07e897213123e5e529c979691f2ab89926/src/coreclr/nativeaot/Directory.Build.props#L83-L91

but file C:\cache\nuget\packages\runtime.win-x64.microsoft.dotnet.ilcompiler.llvm\7.0.0-preview.5.23106.1\sdk\System.Private.CoreLib.dll does not have Exception.DispatchExLLVM method.

SingleAccretion · 2023-02-09T20:06:22Z

I agree it looks like a package composition problem, presumably related to us starting to pick up upstream's integration into the SDK.

ruby0b · 2023-02-17T17:05:31Z

experiencing the same problem (weirdly enough even if I use much older versions like 7.0.0-alpha.1.22071.1)

kant2002 · 2023-02-17T17:06:45Z

@ruby0b what version of Net SDK do you have ?

ruby0b · 2023-02-17T17:17:09Z

7.0.200, freshly installed (had to get a windows setup working for this)

SingleAccretion · 2023-02-17T17:49:44Z

It is definitely odd to see this not working with the older package. I hope to get to investigating this issue soon.

kant2002 · 2023-02-22T13:50:16Z

@ruby0b can you try 7.0.0-preview.4.23079.1 version? This is version which working for me, as you can see here https://github.com/kant2002/NativeAOTWasm.

SingleAccretion · 2023-02-22T22:28:35Z

Making some progress on this...

The immediate problem is that the 7.0.200 SDK contains a change in it which is not compatible with the way targets get to runtime packages. Downgrading to some 7.0.1** SDK via global.json is a possible workaround.

SingleAccretion · 2023-02-22T22:35:47Z

Looks like setting EmitLegacyAssetsFileItems to true is enough to make it work.

@kant2002, @ruby0b could you please confirm if using the following publish line:

dotnet publish --self-contained -r browser-wasm /p:MSBuildEnableWorkloadResolver=false /bl /p:EmitLegacyAssetsFileItems=true

Works for you?

I will set this as the default in the package in that case, until the dust of SDK support settles at least.

kant2002 · 2023-02-23T01:55:04Z

Only specific version which I mention works for me (probably other preview.4 bits would work too). Preview 5 require a bit of improvements. I add following file Microsoft.DotNet.ILCompiler.LLVM.props to Microsoft.DotNet.ILCompiler.LLVM package

<Project>
  <PropertyGroup>
    <!-- N.B. The ILCompilerTargetsPath is used as a sentinel to indicate a version of this file has already been imported. It will also be the path
         used to import the targets later in the SDK. -->
    <ILCompilerTargetsPath>$(MSBuildThisFileDirectory)Microsoft.DotNet.ILCompiler.targets</ILCompilerTargetsPath>
  </PropertyGroup>
</Project>

Literally borrowing https://github.com/dotnet/runtime/blob/1c6752813014463c93ae753ee45994284878f007/src/coreclr/nativeaot/BuildIntegration/Microsoft.DotNet.ILCompiler.props and use runtime.browser-wasm.Microsoft.DotNet.ILCompiler.LLVM but that gives another error

'"C:\nativeaotwasm\.packages\runtime.browser-wasm.microsoft.dotnet.ilcompiler.llvm\7.0.0-preview.5.23113.1\tools\\ilc"' is not recognized as an internal or external command, operable program or batch file.

If I use runtime.win-x64.microsoft.dotnet.ilcompiler.llvm then I have issue with missing members.

Missing bit fo dotnet#2196 You would still required to have ``` <KnownILCompilerPack Update="Microsoft.DotNet.ILCompiler" TargetFramework="net7.0" ILCompilerPackNamePattern="runtime.**RID**.Microsoft.DotNet.ILCompiler.LLVM" ILCompilerPackVersion="7.0.0-preview.5.23113.1" ILCompilerRuntimeIdentifiers="browser-wasm;linux-musl-x64;linux-x64;win-x64;linux-arm;linux-arm64;linux-musl-arm;linux-musl-arm64;osx-arm64;osx-x64;win-arm;win-arm64;win-x86" /> ```

ruby0b · 2023-02-23T15:07:25Z

Using /p:EmitLegacyAssetsFileItems=true fixed it for me (for all versions).

RReverser · 2023-03-01T17:45:24Z

Using /p:EmitLegacyAssetsFileItems=true fixed it for me (for all versions).

With this suggestion I can compile @kant2002's helloworld with both 7.0.0-preview.4.23079.1 and 7.0.0-preview.5.23113.1, but it appears there's a pretty significant Wasm size regression between the two versions.

In particular, I'm using his helloworld.csproj with

    <InvariantGlobalization>true</InvariantGlobalization>
    <IlcGenerateStackTraceData>false</IlcGenerateStackTraceData>
    <IlcDisableReflection>true</IlcDisableReflection>

for size reduction, and with 7.0.0-preview.4.23079.1 I'm getting 4.03MB helloworld.wasm in Release mode, while with 7.0.0-preview.5.23113.1 it results in 6.27MB - more than 1.5x increase.

I wonder where is that coming from.

SingleAccretion · 2023-03-01T18:07:02Z

This is definitely unexpected. I am not sure when we will get to investigating. Some notes if someone is interested in doing that:

-c Release is not actually "Release" currently, since it doesn't enable optimizations in Emscripten (/p:NativeDebugSymbols=false does).
One can use llvm-objdump -h to quickly asses the size of sections.
This may be an upstream regression in reflection-free support. Unlikely but possible.

RReverser · 2023-03-01T18:08:49Z

-c Release is not actually "Release" currently, since it doesn't enable optimizations in Emscripten (/p:NativeDebugSymbols=false does).

Oh thanks, this is valuable. Perhaps worth documenting?

SingleAccretion · 2023-03-01T18:11:06Z

It should be fixed s.t. that -c Release does "the right thing". The problem is with the current setting you get no debug info whatsoever (i. e. not even function names). It should instead have debug info, possibly split out into a separate file by default.

Edit: It should also be tested in CI.

RReverser · 2023-03-01T18:19:59Z

Ok with /p:NativeDebugSymbols=false it's definitely much better than both before and than the traditional Wasm backend size-wise.

helloworld.wasm regression is down to 862.75KB -> 870.42KB, which is pretty negligible, so I'm assuming the main difference earlier came from changes in debug symbols. I could use something like twiggy to dig deeper, but doesn't seem worth it at this point.

Missing bit for #2196 You would still required to have ``` <KnownILCompilerPack Update="Microsoft.DotNet.ILCompiler" TargetFramework="net7.0" ILCompilerPackNamePattern="runtime.**RID**.Microsoft.DotNet.ILCompiler.LLVM" ILCompilerPackVersion="7.0.0-preview.5.23113.1" ILCompilerRuntimeIdentifiers="browser-wasm;linux-musl-x64;linux-x64;win-x64;linux-arm;linux-arm64;linux-musl-arm;linux-musl-arm64;osx-arm64;osx-x64;win-arm;win-arm64;win-x86" /> ```

nifanfa · 2023-04-09T13:51:59Z

Using /p:EmitLegacyAssetsFileItems=true fixed it for me (for all versions).

@ruby0b thank you so much

kant2002 mentioned this issue Feb 23, 2023

Create Microsoft.DotNet.ILCompiler.LLVM.props #2203

Merged

jkotas added the area-NativeAOT-LLVM LLVM generation for Native AOT compilation (including Web Assembly) label Feb 23, 2023

RReverser mentioned this issue Mar 6, 2023

[NativeAOT-LLVM, WASM] Export methods #2204

Open

RReverser mentioned this issue Mar 21, 2023

[NativeAOT-LLVM, Wasm] Cannot marshal multiple arrays by reference #2233

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Recent WASM package have some issues. #2196

Recent WASM package have some issues. #2196

kant2002 commented Feb 9, 2023

kant2002 commented Feb 9, 2023

SingleAccretion commented Feb 9, 2023 •

edited

Loading

ruby0b commented Feb 17, 2023

kant2002 commented Feb 17, 2023

ruby0b commented Feb 17, 2023

SingleAccretion commented Feb 17, 2023

kant2002 commented Feb 22, 2023

SingleAccretion commented Feb 22, 2023

SingleAccretion commented Feb 22, 2023 •

edited

Loading

kant2002 commented Feb 23, 2023

ruby0b commented Feb 23, 2023

RReverser commented Mar 1, 2023

SingleAccretion commented Mar 1, 2023

RReverser commented Mar 1, 2023

SingleAccretion commented Mar 1, 2023 •

edited

Loading

RReverser commented Mar 1, 2023

nifanfa commented Apr 9, 2023

Recent WASM package have some issues. #2196

Recent WASM package have some issues. #2196

Comments

kant2002 commented Feb 9, 2023

kant2002 commented Feb 9, 2023

SingleAccretion commented Feb 9, 2023 • edited Loading

ruby0b commented Feb 17, 2023

kant2002 commented Feb 17, 2023

ruby0b commented Feb 17, 2023

SingleAccretion commented Feb 17, 2023

kant2002 commented Feb 22, 2023

SingleAccretion commented Feb 22, 2023

SingleAccretion commented Feb 22, 2023 • edited Loading

kant2002 commented Feb 23, 2023

ruby0b commented Feb 23, 2023

RReverser commented Mar 1, 2023

SingleAccretion commented Mar 1, 2023

RReverser commented Mar 1, 2023

SingleAccretion commented Mar 1, 2023 • edited Loading

RReverser commented Mar 1, 2023

nifanfa commented Apr 9, 2023

SingleAccretion commented Feb 9, 2023 •

edited

Loading

SingleAccretion commented Feb 22, 2023 •

edited

Loading

SingleAccretion commented Mar 1, 2023 •

edited

Loading