Support Single-File Apps in .NET 5

[swaroop-sridhar]

The goal of this effort is enable .Net-Core apps to be published and distributed as a single executable.

Goals

The .Net 5.0 single file solution should be:

• Widely compatible: Apps containing IL assemblies, ready-to-run assemblies, composite assemblies, native binaries, configuration files, etc. can be packaged into one executable.
• Can run managed components of the app directly from bundle, without need for extraction to disk.
• Usable with debuggers and tools.

User Experience

Here's the overall experience for publishing a HelloWorld single-file app in .net 5:

• Framework-dependent

  • Publish command: dotnet publish -r win-x64 --self-contained=false /p:PublishSingleFile=true
  • Published files: HelloWorld.exe, HelloWorld.pdb

• Self-contained (Linux)

  • Publish command: dotnet publish -r linux-x64 /p:PublishSingleFile=true
  • Published files: HelloWorld, HelloWorld.pdb

• Self-contained (Windows):

  • Publish command: dotnet publish -r win-x64 /p:PublishSingleFile=true
  • Published files: HelloWorld.exe, HelloWorld.pdb, coreclr.dll, clrjit.dll, clrcompression.dll, mscordaccore.dll

• Self-contained (Windows) with bundled native components:

  • Publish command: dotnet publish -r win-x64 /p:PublishSingleFile=true /p:IncludeNativeLibrariesInSingleFile=true
  • Published files: HelloWorld.exe, HelloWorld.pdb

Design Document

A detailed discussion of the goals, non-goals, related-work, options, design decisions, and implementation details of supporting single-file apps is available in this design document

Tracking Progress

• Initial proposal and poll for customer response:
  • Poll on GitHub: Support single-file distribution #11201
  • Enterprise customer poll
• Propose staged implementation of the feature.
• Support single-file apps through extraction (.net core 3)
  • Self-Extractor Design.
  • SDK support for publishing apps as a single-file. Publish to Single-File sdk#3132
  • AppHost support for extracting contents of a single-file bundle at startup. AppHost: Support bundles (stage 1) core-setup#5742
• Support single-file apps running from bundle (.net 5)
  • HostModel library
    • Implement bundle format version 2. AppHost: Support bundles (stage 1) core-setup#5742
  • SDK
    • Implement support for publishing a few files when necessary for PublishSingleFile. Single-File: Update Bundler Invocation. sdk#11586
    • Implement optional settings for single-file publishing. Use SingleFileHost sdk#11797
    • Use SingleFileHost when publishing self-contained single-file apps. Use SingleFileHost for self-contained single-file apps sdk#11567 Use SingleFileHost sdk#11797
    • Trim the set of native dependencies used for single-file publish. Produce DropFromSingleFile annotations in RuntimeList.xml #36578 Use SingleFileHost sdk#11797
  • Host
    • Move bundle-processing from the host to the framework . Single-File: Process bundles in the framework #34274
    • Process deps.json and runtimeconfig.json files directly from bundle. Single-File: Process bundles in the framework #34274
    • Implement a callback for the runtime to probe the contents of the bundle. Single-File: Pass BUNDLE_PROBE property to the runtime #34845
  • Host Builds
    • Windows: Implement SingleFileHost with AppHost, HostFxr, and HostPolicy statically linked. Single-File: Implement statically linked apphost #32823 Build an apphost with hostfxr and hostpolicy linked in #36230
    • Linux: Implement SingleFileHost with host and runtime components statically linked.
      • Statically link CoreCLR and ClrJIT with SingleFileHost Single-File: Implement Linux SingleFileHost with statically linked runtime #37119 Host with coreclr linked in #36847
      • Statically link core native libraries with SingleFileHost SingleFile: Statically link core native libraries to Linux SingleFileHost #38304
  • Runtime
    • Load assemblies directly from bundle. Single-File: Implement loading assemblies from bundle #32822 Single-File: Run from Bundle #36052
    • Dependency resolution: Resolve assemblies from bundle. Single-File: Run from Bundle #36052
  • APIs
    • AppContext.BaseDirectory should return the location of the single-file executable Single-File: Process bundles in the framework #34274

[ghost]

Tagging subscribers to this area: @swaroop-sridhar
Notify danmosemsft if you want to be subscribed.

[YZahringer]

Single-File deployment support for IIS and Azure Web Apps is planned? Related to dotnet/aspnetcore#14473

[psmulovics]

Any chance of implementing this to be single file app for windows?

[swaroop-sridhar]

@YZahringer Loading single-file apps out-of-proc by IIS should be supported. However, loading single-file apps in-proc is not planned for .net 5.

In a single-file app, since host-components (hostfxr. hostpolicy) don't exist as real files on disk, in-process loading of such apps require a new build configuration. That is:

• ASP.NET would need to produce a unmanaged .dll that is a combination of the IIS module and the single file host.
• SDK would need to allow customization of the single file host to use.

[swaroop-sridhar]

Here's the Table of supported host configuration, thanks to @vitek-karas

FD = Framework Dependent
SD = Self-Contained

App	Normal FD	Normal SC	Single-File 3.1 FD	Single-File 3.1 SC	Single-File 5.0 FD	Single-File 5.0 SC
.exe app	Yes	Yes	Yes	Yes	Yes	Yes
Native hosting app (IIS scenario)	Yes	Yes	Not Planned	Not Planned (complex)	Not Planned	Not Planned (needs new build config)
Native hosting component (assembly) (COM server, plugins, …)	Yes	No (desire exists)	Not Planned	Not Planned	Not Planned	Not Planned
Load a plugin	Yes	Yes	Yes	Yes	Yes	Yes

[swaroop-sridhar]

@psmulovics: Are you asking about Asp.net/IIS issue, or Windows single-file apps in general?
Yes this feature is planned for Windows. Please see the notes in the "User Experience" section, and this design doc. Thanks.

[YZahringer]

@swaroop-sridhar thank you for the answer and the details. This is not blocking, but the structure of the deployed files is not very "clean" at the moment.

When deployed in Self-contained mode, I have 313 .NET Framework files at root folder of my AzureWebApp/IIS website. It's a bit messy and it can cause human errors during updates or complicate the editing of configuration files like appsettings.json and web.config. Previously with ASP.NET 4, these files are in a bin folder, no binary at the root and more clean.

It might be cleaner/magic with a Single-File deployed 😉

[davidfowl]

@YZahringer unfortunately IIS in process doesn't support single file. Out of process does.

[Lonli-Lokli]

Is it supposed that all runtimes (Alpine as well) will support PublishReadyToRun flag?

[swaroop-sridhar]

@Lonli-Lokli did you mean to ask about PublishSingleFile or PublishReadyToRun property?
PublishSingleFile is expected to work on all platforms, including Alpine.

[Lonli-Lokli]

@swaroop-sridhar yep, you are right, it was a typo.
Actually, given the description

Widely compatible: Apps containing IL assemblies, ready-to-run assemblies, composite assemblies, native binaries, configuration files, etc. can be packaged into one executable.

PublishReadyToRun should be supported as well for all runtimes, otherwise SFA cannot be verified for all runtimes.

[Serentty]

I would personally like to see a single-file distribution option that uses a file format that can be associated with the dotnet loader instead of each individual platform's native executable loader, similar to JAR files in the JVM world, so that there can be an option for easy-to-open portable binaries that don't depend on providing a loader for each platform.

[Serentty]

Obviously not suggesting this for .NET 5 or anything. Just for the future in general, it would be nice to have a file format that any user with .NET installed can open, without worrying about having a matching native loader.

[Serentty]

Judging by the reactions I guess that isn't a popular idea. To be honest I would be happy even if it weren't a single file, but just a small text file or something that points to an entry point in a DLL, and maybe holds some dependency information and stuff like that. Just anything nicer than a shell script as a portable solution for launching a program. If it's not single-file it starts to be a bit off-topic for this issue though, so let me know if I should open something somewhere else.

EDIT: Reading my previous comment I also realize that it made it sound like I was saying I wanted that instead of single native executables. That's not what I meant and I see why ready-to-run self-contained binaries are useful.

[swaroop-sridhar]

@Serentty Platform independent single-file apps is tracked by #13677. As you've noted in your comments, this is beyod the scope for .net5. We'll keep this in mind for .net6 planning. Thanks.

[Serentty]

Oh wow, looks like I've found and commented in that issue before, and then forgotten about it. Alright, thanks, I'll be on my way over there.

[jjxtra]

@vitek-karas Can you use Assembly.Load on a read-to-run byte[]?

[jjxtra]

Is there a #define that is turned on when single file publishing is active?

[vitek-karas]

Re Assembly.Load on R2R images: (caveat: I didn't try this) I think it will work, but the image will be read as pure IL - the R2R part will be ignored. So you won't get the performance benefits of R2R.

Re define: There is no define, because single-file is a publish option - so post-build. I can publish the same app multiple times with different settings without rebuilding the app.

[jjxtra]

Thanks for the response. Is there any call at runtime I could make to reliably determine the exe is single file, short of a define of my own in the build process?

[agocke]

If you need to know about a specific assembly, I'd check if Assembly.Location returns empty string. That's a good check because it should work for regular scenarios where you're loading from memory and for single-file. You could probably check corelib (e.g.typeof(System.Object).Assembly) for single-file as a whole, but it's probably best to ask the narrowest question you need the answer to, because it may be more flexible for more scenarios.

[vitek-karas]

Mostly agree with @agocke - try to make your decisions based on local knowledge as much as possible.
I'm curious - what is the scenario where you think you will need to know this?

[Sewer56]

Not the original person to ask the question but I can think of a scenario.

Various 3rd party libraries and internal solutions for self-updating applications don't clean up unused files from previous updates; only overwrite existing ones and add new files. If the application were to update from for example self contained to single file (with native dependencies); you probably wouldn't want an entire unused second copy of the runtime (assuming no trimming was used).

Edit: I implemented this the other day in one of my applications; though FDD Single File WPF applications broke due to dotnet/wpf#3516 in .NET 5 RC1 (this is a regression) so I'm holding out for a bit. I did it by checking if Assembly.Location is an empty string (same as suggestion above) after reading through the design doc.

[jjxtra]

This is one of my use cases - automatic software updates and cleaning out non self contained stuff. This code will all be in a shared library. If the new version is self contained, I want to clean out the non-self contained files.

[harold-b]

It seems like /p:IncludeNativeLibrariesInSingleFile=true no longer works with dotnet 5.0 rc1.

dotnet 5.0.100-rc.1.20452.10 produces clrcompression.dll, clrjit.dll, coreclr.dll and mscordaccore.dll, along with the executable and pdb. Whilst version 5.0.100-preview.8.20417.9 correctly produces just the executable and the pdb file.

[Sewer56]

It seems like /p:IncludeNativeLibrariesInSingleFile=true no longer works with dotnet 5.0 rc1.

dotnet 5.0.100-rc.1.20452.10 produces clrcompression.dll, clrjit.dll, coreclr.dll and mscordaccore.dll, along with the executable and pdb. Whilst version 5.0.100-preview.8.20417.9 correctly produces just the executable and the pdb file.

Per the design: https://github.com/dotnet/designs/blob/main/accepted/2020/single-file/design.md#user-experience

The property was renamed to IncludeNativeLibrariesForSelfExtract as opposed to IncludeNativeLibrariesInSingleFile.

[harold-b]

Thank you, @Sewer56, I missed that. I was about to look to see if there was a rename when I see your response.

[alexb5dh]

While this is marked as closed I'm getting "Including symbols in a single file bundle is not supported when publishing for .NET5 or higher" while using the following publish file:

<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <Configuration>Release</Configuration>
    <Platform>Any CPU</Platform>
    <PublishDir>bin\Release\net5.0\publish\</PublishDir>
    <PublishProtocol>FileSystem</PublishProtocol>
    <TargetFramework>net5.0</TargetFramework>
    <RuntimeIdentifier>linux-x64</RuntimeIdentifier>
    <SelfContained>true</SelfContained>
    <PublishSingleFile>True</PublishSingleFile>
    <PublishTrimmed>False</PublishTrimmed>
  </PropertyGroup>
</Project>

Issue has no mention that publish files are not supported and I tried to exlude them via setting Debugging Information to None in advanced build settings to no avail. Could someone please clarify if this issue is really closed and if single-file publish is supported in .NET 5?

[vitek-karas]

@alexb5dh it's likely that somewhere in your project file you have IncludeSymbolsInSingleFile set to true. It doesn't have to be in the publish file, it can be in the .csproj for example.
You can also try to grab a binlog (just add /bl to the command line when publishing the app) and then open it in https://msbuildlog.com/. And search for IncludeSymbolsInSingleFile it should show you where it's set to true.

[alexb5dh]

Thanks @vitek-karas, that was the case.

For those looking for a way to embed pdb - you probably need to set DebugType to embedded, as IncludeSymbolsInSingleFile won't be supported: #42278 (comment).

[vitek-karas]

Bundling PDBs would be "weird" in .NET 5 since the assemblies (.dll) are never extracted out of the bundle onto disk. So having the pdbs on disk would make little sense. Runtime should be able to access the embedded pdbs in all cases it needs to (generating stack traces, attached debugger).

[michalhosala]

I just tried to migrate my windows service to .NET 5.0 (entity framework dependency too) and publish a single file exactly as I was doing it before with .NET core 3.1. However, my publish directory contains more than single file as it used to, most importantly Microsoft.Data.SqlClient.SNI.dll, why is that so?

[agocke]

We've changed single-file to do proper load-from-memory instead of extracting to a temporary directory. Unfortunately we didn't get the runtime all the way down to one file in .NET 5, and native libraries (Microsoft.Data.SqlClient.SNI) cannot be loaded from memory. We're aiming to get the runtime down to one file for 6.0.

If you need exactly one file in 5.0 you can extract only the native libraries using <IncludeNativeLibrariesForSelfExtract>true, although that will necessarily extract the native binaries to a temporary directory.

[michalhosala]

@agocke thank you for the useful info. The main reason for us to migrate to 5.0 was that binaries will no longer have to be extracted to temp location as we were experiencing a known issue where extracted files were getting deleted randomly from there (I know there is a workaround). So using <IncludeNativeLibrariesForSelfExtract>true would effectively be the same as 3.1 for us, will have to wait for 6.0.

[jkotas]

We do not plan to solve the single-file problem with native assets of 3rd party libraries in .NET 6. I expect that Microsoft.Data.SqlClient.SNI.dll is still going to be there either on disk or be extracted into temp directory.

[cincuranet]

We do not plan to solve the single-file problem with native assets of 3rd party libraries in .NET 6. I expect that Microsoft.Data.SqlClient.SNI.dll is still going to be there either on disk or be extracted into temp directory.

Out of curiosity. Is this a technical challenge (and if so what) or simple planning/resources reason?

[jkotas]

It is technical and ecosystem challenge. We would need to:

• Introduce a new way to distribute nuget package unmanaged assets that is based on either unmanaged C/C++ sources or unmanaged .obj files.
• Add runtime support and SDK tooling for this modality. E.g. Among other things, this woudl mean that .NET SDK would have dependency on the C/C++ toolset for this modality.
• Convince the NuGet package authors with unmanaged assert to adopt this distribution model, in addition to what they have. This may be particularly challenging for NuGet package like Microsoft.Data.SqlClient where the unmanaged assets are not open source.

[jkotas]

The alternative is for NuGet package authors to eliminate unmanaged dependencies by rewriting them in C#. In fact, it is where Microsoft.Data.SqlClient is heading with UseManagedNetworkingOnWindows configuration option: https://docs.microsoft.com/en-us/sql/connect/ado-net/appcontext-switches?view=sql-server-ver15#enabling-managed-networking-on-windows . This option is only supported for testing and debugging purposes only currently.

[agocke]

So using true would effectively be the same as 3.1 for us, will have to wait for 6.0.

Not quite. Only extracting the native files is significantly less than before, so it may still be a net improvement

[michalhosala]

@agocke I meant it is going to be the same because there are going to be more files in the temp folder than just one exe and these files can again get randomly purged, as it is happening. That's why it is "the same" for us, while I of course agree the process was overally improved quite a bit.

[agocke]

We've also implemented some fixes to validate that the proper files are present whenever the app is started, but you're right: there's still an inherent race condition risk with the file system, and these changes only reduce risk, not eliminate it.