Add `-aot` parameter to the console template #31739

MichalStrehovsky · 2023-04-12T09:09:38Z

This is similar to the api template in ASP.NET that also allows -aot (or --publish-native-aot) option (see dotnet/aspnetcore#46064).

We set similar properties:

<PublishAot>true</PublishAot> (this is the obvious one)
<InvariantGlobalization>true</InvariantGlobalization> to get rid of ICU dependency on Linux and make the hello world executable ~20% smaller in general, even outside Linux.
a commented out <StripSymbols>true</StripSymbols>. This is for discoverability. We default this to false due to platform conventions, but many users would be fine setting this to true. true is the .NET convention.

I took the string from ASP.NET. I could have also copied the localized strings but I guess it's better to not step on the toes of the localization team.

Cc @dotnet/ilc-contrib @DamianEdwards

This is similar to the `api` template in ASP.NET that also allows `-aot` (or `--publish-native-aot`) option. We set similar properties: * `<PublishAot>true</PublishAot>` (this is the obvious one) * `<InvariantGlobalization>true</InvariantGlobalization>` to get rid of ICU dependency on Linux and make the hello world executable ~20% smaller in general, even outside Linux. * a commented out `<StripSymbols>true</StripSymbols>`. This is for discoverability. We default this to `false` due to platform conventions, but many users would be fine setting this to `true`. I took the string from ASP.NET. I could have also copied the localized strings but I guess it's better to not step on the toes of the localization team.

MichalStrehovsky · 2023-04-12T09:14:44Z

I've not done anything for F# since even a hello world produces both trimming and AOT warnings and it's not the experience we're shooting for with PublishAot.

MichalStrehovsky · 2023-04-12T09:15:14Z

...on.ProjectTemplates.8.0/content/ConsoleApplication-CSharp/Company.ConsoleApplication1.csproj

+    <InvariantGlobalization>true</InvariantGlobalization>
+
+    <!-- Uncomment below line to make native publish outputs a lot smaller on Linux and macOS -->
+    <!-- <StripSymbols>true</StripSymbols> -->


I've struggled a bit with the level of detail here. We don't want paragraphs of text that explain native debuggers/profilers/etc may or may not support GNU debuglink and if they don't extra steps are needed to make sense of the resulting binary, etc. Most people will be fine because most tools do support it.

What about making an aka.ms link, and then you can point it to docs that explains it fully?

eerhardt · 2023-04-12T13:15:41Z

...on.ProjectTemplates.8.0/content/ConsoleApplication-CSharp/Company.ConsoleApplication1.csproj

+    <InvariantGlobalization>true</InvariantGlobalization>
+
+    <!-- Uncomment below line to make native publish outputs a lot smaller on Linux and macOS -->
+    <!-- <StripSymbols>true</StripSymbols> -->


What about making an aka.ms link, and then you can point it to docs that explains it fully?

eerhardt · 2023-04-12T13:19:51Z

...on.ProjectTemplates.8.0/content/ConsoleApplication-CSharp/Company.ConsoleApplication1.csproj

+    <InvariantGlobalization>true</InvariantGlobalization>
+
+    <!-- Uncomment below line to make native publish outputs a lot smaller on Linux and macOS -->
+    <!-- <StripSymbols>true</StripSymbols> -->


I think we should follow the same approach for ASP.NET and Hosting templates. We shouldn't have different strategies for the different templates if there are no technical reasons for it. We should decide here if this is the strategy we want to take, and apply it in all templates.

I also think that this just clutters the template. IMHO - I think StripSymbols=true should be the default without specifying it in the .csproj. It is what most normal .NET devs expect.

I think StripSymbols=true should be the default without specifying it in the .csproj. It is what most normal .NET devs expect.

I think this is the crux of it. Lot of feedback I heard was that .NET Devs expect debuggability by default, which would mean we have to leave the symbols there. Another view is what is the typical behavior for the target platform (Linux, containers).

I forgot to add that I think there's a difference between symbols for managed assemblies and symbols for native AOT. Managed assemblies are debugable even without symbols, the symbols only improve the experience somewhat. Native AOT is completely undebugable without symbols. To me that is the key difference between the two and why I think we should treat them separately.

Could we base it on Release or not?

Personally I don't think so - but I have no hard data.
In short - I think the number of people publishing AOT with Debug will be tiny. Non-AOT .NET makes little difference between Debug and Release - the debugging experience is very similar. So I think that lot of people don't think about the configuration setting when considering if something is debuggable or not. But these are just my personal guesses...

If we decide to default StripSymbols to true on the SDK side, I have some additional questions:

.NET developers are used to symbols having the .pdb extension, no matter their underlying representation (managed/portable are very different file formats). Should we also name the native symbol file with the Windows (and by extension .NET) convention? If we don't want people to learn the platform conventions, this follows the principle of least surprise.

Is there any point in the AOT console template? My main motivation for it was really just about the discoverability of StripSymbols. <PublishAot>true</PublishAot> doesn't seem to be interesting enough as a differentiator. I've also put <InvariantGlobalization>true</InvariantGlobalization> because it gets rid of ICU on Linux and helps discoverability, but that one is basically just getting a ride. Are the things that are left still important enough that we need a template?

I think it's still worth adding the option to the template if for nothing else that visibility and discovery that native AOT is now a first-class option.

Should we also name the native symbol file with the Windows (and by extension .NET) convention?

This would not work well even if we wanted to. Many tools infer the file format from the extension. If we started giving .pdb extension to DWARF files, it would be endless source of breaks.

We could maybe go with .pdb.dbg/.pdb.dwarf. It's just whether we want people to know these are the debugging symbols, not a vital part of a non-single-file app. We assume .NET developers don't know about the underlying platform. But I guess this would be orthogonal to this PR and at this point it would also be a "breaking" change from 7.0 (so will be the new default though).

@dotnet/templating-engine-maintainers could someone have a look? I think we settled on the non-technical questions and we might tweak this based on the defaults in the coming weeks.

We could maybe go with .pdb.dbg/.pdb.dwarf. It's just whether we want people to know these are the debugging symbols, not a vital part of a non-single-file app.

I do not think we need to go this far. People will need to learn some amount of platform specific details in any case. The problem is that we do not have a good trigger for learning about symbols today. People do not read the manual; they just try to do stuff. We have seen many variants of a story that goes like this: Somebody builds a native AOT binary on Linux for the first time, sees a single giant binary in the publish folder, and concludes that this technology is unusable. I think that having two files - one smaller executable and one large non-executable - would be good enough to trigger the learning even without the .pdb hint. Seeing two files will make people ask why there are two files, whether they need both, etc.

jkotas

LGTM.

If we choose to change the StripSymbols defaults, it can be done as a follow up across the board.

vlada-shubina · 2023-04-18T12:58:51Z

@baronfel could you please review as this is a modification to console template? Thanks

vlada-shubina

Please consider adding tests to https://github.com/dotnet/sdk/blob/main/src/Tests/dotnet-new.Tests/CommonTemplatesTests.cs

MichalStrehovsky · 2023-04-20T20:15:44Z

@baronfel do you have any opinions? I'd like this to make it into Preview 4 and it's getting close.

baronfel · 2023-04-20T20:21:16Z

LGTM - wanted to make sure the parameters aligned with ASP.NET's template (of course they do).

Any reason why the F# console template can't have this as well?

MichalStrehovsky · 2023-04-20T20:22:32Z

LGTM - wanted to make sure the parameters aligned with ASP.NET's template (of course they do).

Any reason why the F# console template can't have this as well?

#31739 (comment) :)

baronfel · 2023-04-20T20:24:54Z

Ah, thanks :) IIRC @KevinRansom is doing some more trimming/aot work right now that might make that disappear. @vzarytovskii want to make a note to check the experience after that lands and update the F# console template with this option as well?

baronfel

LGTM!

See discussion in dotnet/sdk#31739 (comment). Once this merges, we should: - [ ] Update the `dotnet new console -aot` template - [ ] Update the `dotnet new api -aot` template - [ ] Update docs and declare different defaults.

am11 · 2023-04-21T06:25:23Z

Thanks @MichalStrehovsky! It would be nice(r) if we use single or double hyphens consistently with dotnet new. -aot is the odd one out: dotnet new console --name foo --framework net8.0 -aot as Unix-y tools arguments convention suggest: -aot == -a -o -t. We used to have one such anomaly: #4277, now we have two. 🥲

MichalStrehovsky · 2023-04-21T06:27:57Z

Thanks @MichalStrehovsky! It would be nice(r) if we use single or double hyphens consistently with dotnet new. -aot is the odd one out: dotnet new console --name foo --framework net8.0 -aot as Unix-y tools arguments convention suggest: -aot == -a -o -t. We used to have one such anomaly: #4277, now we have two. 🥲

@DamianEdwards what do you think? (For this template I just aligned with dotnet/aspnetcore#46064 and there wasn't much to think about because the precedent was set.)

DamianEdwards · 2023-04-21T16:10:47Z

I'm happy to say from now on we should strive to match the Unix-y tool convention and change the -aot alias to be --aot on all the templates.

baronfel · 2023-04-21T16:18:13Z

Definitely would echo this - I was biasing towards compat with aspnetcore here, but I would prefer POSIX-y.

DamianEdwards · 2023-04-21T16:39:10Z

Should we consider ret-conning this across all templates, by adding -- aliases for any non-single char - prefixed parameters in 8.0 and even 7.0.xxx, and consider a deprecation plan (warnings, etc.) for the now non-conforming - prefixed parameters?

See discussion in dotnet/sdk#31739 (comment).

eerhardt · 2023-04-24T15:42:50Z

I've not done anything for F# since even a hello world produces both trimming and AOT warnings and it's not the experience we're shooting for with PublishAot.

@MichalStrehovsky - I am able to successfully publish and run an F# "worker" app with AOT with no warnings:

What warnings were you getting with a hello world F# app?

eerhardt · 2023-04-24T15:46:02Z

Oh, I just tried it manually, and it appears to be coming from printfn "Hello from F#". It actually completely breaks ILC:

EXEC : error : Failed to load type '!0' from assembly 'FSharp.Core, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' [C:\DotNetTest\Net8FSharp\Net8FS
harp.fsproj]
  Internal.TypeSystem.TypeSystemException+TypeLoadException: Failed to load type '!0' from assembly 'FSharp.Core, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f
  5f7f11d50a3a'
     at Internal.TypeSystem.ThrowHelper.ThrowTypeLoadException(ExceptionStringID, String, String) + 0x37
     at System.Reflection.TypeNameParser.GetType(String, ReadOnlySpan`1, String) + 0x15c
     at System.Reflection.TypeNameParser.NamespaceTypeName.ResolveType(TypeNameParser&, String) + 0x3e
     at System.Reflection.TypeNameParser.GenericTypeName.ResolveType(TypeNameParser&, String) + 0x73
     at Internal.TypeSystem.Ecma.CustomAttributeTypeProvider.GetTypeFromSerializedName(String) + 0x71
     at System.Reflection.Metadata.Ecma335.CustomAttributeDecoder`1.DecodeArgument(BlobReader&, CustomAttributeDecoder`1.ArgumentTypeInfo) + 0xb4
     at System.Reflection.Metadata.Ecma335.CustomAttributeDecoder`1.DecodeFixedArguments(BlobReader&, BlobReader&, Int32, BlobReader) + 0x11e
     at System.Reflection.Metadata.Ecma335.CustomAttributeDecoder`1.DecodeValue(EntityHandle, BlobHandle) + 0x39c
     at Internal.TypeSystem.Ecma.MetadataExtensions.<GetDecodedCustomAttributes>d__3.MoveNext() + 0x21a
     at ILCompiler.DependencyAnalysis.DynamicDependencyAttributesOnEntityNode.GetStaticDependencies(NodeFactory) + 0xbf
     at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.GetStaticDependenciesImpl(DependencyNodeCore`1) + 0x44
     at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.GetStaticDependencies(DependencyNodeCore`1) + 0x32
     at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.ProcessMarkStack() + 0xb5
     at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.ComputeMarkedNodes() + 0x4e
     at ILCompiler.ILScanner.ILCompiler.IILScanner.Scan() + 0x14
     at ILCompiler.Program.<Run>g__RunScanner|3_0(Program.<>c__DisplayClass3_0&) + 0x108
     at ILCompiler.Program.Run() + 0x153f
     at ILCompiler.ILCompilerRootCommand.<>c__DisplayClass206_0.<.ctor>b__0(InvocationContext context) + 0x20c

I'll remove the F# worker template's -aot parameter to be consistent everywhere until F# fully supports AOT.

MichalStrehovsky · 2023-04-25T01:24:25Z

It actually completely breaks ILC:

Heh, F# bug: dotnet/fsharp#15140. Nevertheless, we should not crash. This should be in a try/catch and we should ignore the invalid blobs here.

vzarytovskii · 2023-04-25T10:06:22Z

It actually completely breaks ILC:

Heh, F# bug: dotnet/fsharp#15140. Nevertheless, we should not crash. This should be in a try/catch and we should ignore the invalid blobs here.

Huh, interesting, it seems it was caused by our recent (last week) change in DU codegen, when we try to account for refleciton we do.

MichalStrehovsky requested a review from a team as a code owner April 12, 2023 09:09

dotnet-issue-labeler bot added Area-Infrastructure untriaged Request triage from a team member labels Apr 12, 2023

MichalStrehovsky commented Apr 12, 2023

View reviewed changes

eerhardt reviewed Apr 12, 2023

View reviewed changes

Test failure

fe2e45f

jkotas approved these changes Apr 15, 2023

View reviewed changes

vlada-shubina requested a review from baronfel April 18, 2023 12:58

vlada-shubina reviewed Apr 18, 2023

View reviewed changes

Add test

be0bbec

baronfel mentioned this pull request Apr 19, 2023

[Epic] Common template features for .NET templates (sln, editorconfig) dotnet/templating#3208

Open

Merge branch 'main' into aottemplate

c3a13f1

baronfel approved these changes Apr 20, 2023

View reviewed changes

MichalStrehovsky merged commit d10b354 into dotnet:main Apr 20, 2023

MichalStrehovsky deleted the aottemplate branch April 20, 2023 20:44

MichalStrehovsky mentioned this pull request Apr 21, 2023

Enable StripSymbols by default dotnet/runtime#85139

Merged

3 tasks

MichalStrehovsky added a commit to dotnet/runtime that referenced this pull request Apr 23, 2023

Enable StripSymbols by default (#85139)

8c2c15b

See discussion in dotnet/sdk#31739 (comment).

eerhardt mentioned this pull request Apr 24, 2023

Remove F# Worker AOT parameter. dotnet/aspnetcore#47858

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `-aot` parameter to the console template #31739

Add `-aot` parameter to the console template #31739

MichalStrehovsky commented Apr 12, 2023

MichalStrehovsky commented Apr 12, 2023

MichalStrehovsky Apr 12, 2023

eerhardt Apr 12, 2023

eerhardt Apr 12, 2023

eerhardt Apr 12, 2023

vitek-karas Apr 12, 2023

vitek-karas Apr 12, 2023

eerhardt Apr 12, 2023

vitek-karas Apr 12, 2023

MichalStrehovsky Apr 16, 2023

DamianEdwards Apr 16, 2023

jkotas Apr 17, 2023

MichalStrehovsky Apr 17, 2023

jkotas Apr 17, 2023

jkotas left a comment

vlada-shubina commented Apr 18, 2023

vlada-shubina left a comment

MichalStrehovsky commented Apr 20, 2023

baronfel commented Apr 20, 2023

MichalStrehovsky commented Apr 20, 2023

baronfel commented Apr 20, 2023

baronfel left a comment

am11 commented Apr 21, 2023

MichalStrehovsky commented Apr 21, 2023

DamianEdwards commented Apr 21, 2023

baronfel commented Apr 21, 2023

DamianEdwards commented Apr 21, 2023

eerhardt commented Apr 24, 2023

eerhardt commented Apr 24, 2023 •

edited

Loading

MichalStrehovsky commented Apr 25, 2023

vzarytovskii commented Apr 25, 2023

Add -aot parameter to the console template #31739

Add -aot parameter to the console template #31739

Conversation

MichalStrehovsky commented Apr 12, 2023

MichalStrehovsky commented Apr 12, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkotas left a comment

Choose a reason for hiding this comment

vlada-shubina commented Apr 18, 2023

vlada-shubina left a comment

Choose a reason for hiding this comment

MichalStrehovsky commented Apr 20, 2023

baronfel commented Apr 20, 2023

MichalStrehovsky commented Apr 20, 2023

baronfel commented Apr 20, 2023

baronfel left a comment

Choose a reason for hiding this comment

am11 commented Apr 21, 2023

MichalStrehovsky commented Apr 21, 2023

DamianEdwards commented Apr 21, 2023

baronfel commented Apr 21, 2023

DamianEdwards commented Apr 21, 2023

eerhardt commented Apr 24, 2023

eerhardt commented Apr 24, 2023 • edited Loading

MichalStrehovsky commented Apr 25, 2023

vzarytovskii commented Apr 25, 2023

Add `-aot` parameter to the console template #31739

Add `-aot` parameter to the console template #31739

eerhardt commented Apr 24, 2023 •

edited

Loading