How to pass APPDOMAIN_IGNORE_UNHANDLED_EXCEPTIONS using hostfxr

[cheverdyukv]

I was using https://docs.microsoft.com/en-us/dotnet/core/tutorials/netcore-hosting#create-a-host-using-mscoreeh to host .NET runtime. But after discussion in #39167 I was told to use hostfxr to host .NET runtime.

But as far as I can see there is no way to pass APPDOMAIN_IGNORE_UNHANDLED_EXCEPTIONS and our application needs for 2 reasons:

1. We use some 3rd party components that create threads and raises unhandled exception.
2. Our application handles all unhandled exceptions anyway using appropriate callbacks, send report to us etc. We just don't need runtime to terminate process at that moment.

This is last issue that prevents us to use hostfxr

[ghost]

Tagging subscribers to this area: @vitek-karas, @swaroop-sridhar, @agocke
Notify danmosemsft if you want to be subscribed.

[jkotas]

This option is left-over from Silverlight. It worked ok for Silverlight sandboxed applications that were severely limited in what they can do.

For .NET Core, there is number of ways managed exceptions can go unhandled even when this is set. I am not sure whether it makes sense to try to make it more first class in its current form. It is a poorly defined feature.

Related discussion: #11546

[cheverdyukv]

1. Well it is already first class as it is exposed via public API (Create a host using Mscoree.h)
2. Perhaps that feature is not cover every possible case, but it covers 95% of our need
3. I believe host should not be terminated because some plugin throws exception. Yes, we can put try catch around every call even it requires months of work. But as I said nothing prevents plugin to create new thread, task etc and throw exception there.
4. If later we will discover that not all cases are covered, it is easier to deal with some rare case
5. As I mentioned before, right now we know for sure, that 3rd party component will throw in thread it creates and then will lead to host termination. I want to migrate hosting to use hostfxr but this issue prevents us to use hostfxr. And without hostfxr AssemblyDependencyResolver does not work.

[rseanhall]

This may not be supported, but if you call hostfxr_initialize_for_runtime_config or hostfxr_initialize_for_dotnet_command_line then that is enough to get AssemblyDependencyResolver to work. You can then load the runtime through Mscoree.h as you did before. This is assuming that you're loading the runtime from the same directory that hostfxr was going to load it from.

[cheverdyukv]

@rseanhall I don't really want to initialize it twice. It may create some unwanted effects.
And I think that it not that hard to add support for what I want. Let's say I will set gv_hostfxr_set_runtime_property_value value "UNSUPPORTED.APPDOMAIN_IGNORE_UNHANDLED_EXCEPTIONS" to 1 and then couple of lines that will read it and set it.
I think it is way simpler than initialize runtime twice.

[rseanhall]

I'm just trying to point out a potential workaround in versions of the runtime where this isn't implemented. It's not possible to initialize the runtime multiple times. The hostfxr_initialize_* methods don't initialize the runtime, they initialize hostfxr and get everything ready to initialize the runtime. The potentially unsupported thing is that the "get everything ready to initialize the runtime" part currently does everything that AssemblyDependencyResolver needs to actually work.

[cheverdyukv]

I appreciate your help and got your idea. I just feel that workaround will bite me later. But looks like there is no other choice.
Thank you.

[nxrighthere]

Unhandled exceptions in embedded CoreCLR is a serious problem. In our case, the runtime is embedded into a game engine. To achieve a similar behavior like in Unity with Mono where unhandled exceptions are implicitly redirected to the console window and log files of the engine, we wrap the entire functionality with a single function like this, which invokes managed function pointers from unmanaged code here. As you can see, that means a very limited and predefined set of function signatures that can be used for execution, and it also involves performance cost.

This way we prevent termination of the process, and handle redirection of unhandled exceptions to log files, the console window, and on-screen messages of the engine.

It would be great if we get a hostfxr functionality that allows us to redirect exceptions to a custom function without handling them explicitly with try-catch blocks to prevent termination of the process.

[vitek-karas]

@nxrighthere : The important distinction between Mono's native APIs and the hostfxr based native hosting of CoreCLR is that Mono implements a full set of embedding APIs - so it can handle exceptions, GC interactions, reflection, managed object creation and so on, all from native code. That was not the aspiration of the hostfxr based native hosting APIs for CoreCLR. The design for hostfxr native hosting was to allow execution of a simple managed entry point from various scenarios. It is fully expected, that if a more complex communication is to be created between the native and managed side, there would be custom code to handle that. That includes exception handling.

We also discussed what it would mean to implement full embedding APIs for CoreCLR, but other than the obvious (lot of work) it would also require non-trivial amount of design - mono embedding API has known issues/limitations which we would like to avoid, CoreCLR has slightly different behavior in certain cases and so on. Hopefully we'll get to it at some later date.

The question in this issue is about handling exception on other threads when using native hosting APIs - which doesn't seem to be the issue you're trying to solve.

[nxrighthere]

Our only problem as stated in this issue:

Our application handles all unhandled exceptions anyway using appropriate callbacks, send report to us etc. We just don't need runtime to terminate process at that moment.

Personally, we don't really need embedding API similar to Mono, but we need a way to at least stop the termination of the process because in the development stage where a product is not ready for consumption by the end-user it makes no sense.

If a game is in development under the full control of IDE and a debugger before we ship it, what's the point to terminate the entire environment once an unhandled exception occurs? It makes sense to terminate the process when a game/application shipped to a customer, but before that, it doesn't carry any benefits.

[nxrighthere]

Please, can we reach a consensus regarding hostfxr API that will allow preventing termination of the process? We are considering forking CoreCLR to modify that part for our needs to stop crashing the editor of the engine as long as the application is in development. This issue is a ship stopper for us, and we can't implement new features for our customers because of that.

[cheverdyukv]

Same here, we cannot switch to hoxtfxr and have to use using-mscoreeh method to host runtime. I'm curing are they planning to use .NET 5 in SQL Server? How SQL Server team will solve it?

[jkotas]

they planning to use .NET 5 in SQL Server? How SQL Server team will solve it?

There are no plans to for SQL CLR based on .NET 5.

[jkotas]

@nxrighthere @cheverdyukv Could you please share API proposal(s) that you would like to see added to address your scenarios? The API signature (either managed or unmanaged), how the API would be used and details about the guarantees that the API would provide.

Let's treat this as a new API proposal and pretend that APPDOMAIN_IGNORE_UNHANDLED_EXCEPTIONS does not exist.

[nxrighthere]

A perfect solution in our case would be API similar to hostfxr_set_error_writer_fn, say hostfxr_set_unhandled_exceptions_writer_fn which prevents termination of the process and writes unhandled exceptions using a custom function. Our current implementation is very straightforward which writes unhandled exceptions like this:

[jkotas]

prevents termination of the process

It can be done on threads started by .NET runtime. What should it do on foreign threads?

[cheverdyukv]

I believe that foreign threads belongs to host and host should deal with them. Do runtime terminates process in this case?

[nxrighthere]

My proposal would be API that allows spawning threads from .NET runtime instead of using native platform-specific/language-specific API so the application that embedding hostfxr can use it instead.

[jkotas]

I believe that foreign threads belongs to host

They belong to whoever created them. For example, the thread may be created by the OS or a library.

Do runtime terminates process in this case?

On Windows, the runtime lets the exception to propagate using SEH. It typically results into exception being unhandled. It is a bad practices for libraries to handle exceptions that they do not understand.
On Unix, the runtime terminates the process using abort.

[jkotas]

My proposal would be API that allows spawning threads from .NET runtime

It would not solve the problem with the foreign threads.

[cheverdyukv]

@jkotas Could you please elaborate on "On Windows, the runtime lets the exception to propagate using SEH"? If some host thread will call runtime and there is unhanded exception I think thread function should deal with that. If it is say OS thread pool thread and there is exception, allow thread pool deal with that. Basically exception would be propagated to caller.
I'm not sure if I provided correct examples as you guys knows way more about different situations and it is why I would I need more details.

[cheverdyukv]

Host process should be in control on any runtime decision about whether terminate process or not. It is really like how SQL Server hosts .NET runtime. Runtime in this case is just guest in somebody’s house and it should not enforce its own rules. I can accept that certain exceptions like memory corruption, stack overflow or out of memory, will lead to process termination. Handling these cases is really hard. But all other type of unhandled exceptions should go via some global handler. Handler should accept exception object and returns if it is ok to terminate application.

I believe that load_assembly_and_get_function_pointer_type could be used to retrieve handler delegate and use something like hostfxr_set_runtime_property_value_type but this time it will be delegate. I believe signature should be something like that:
public static void HandleUnhandledException(Exception exception, ref int actionToTake)

I think it should be in managed code, because there is no API to process exception. Depending on result, exception can be suppressed, suppressed and hostfxr_set_error_writer_fn can called, or default processing which is terminate process.
Another approach would be to do it in unmanaged way and pass some information about exception that later could be recovered on managed site. So, if some code does not care about type of exception it is and only need message and maybe stack, then exception can be dealt with completely on unmanaged side.

[nxrighthere]

@jkotas If I understand you correctly, you would like to handle edge cases where the code in the host application is not aware of .NET runtime but still interoperate with managed code?

[cheverdyukv]

It is a bad practices for libraries to handle exceptions that they do not understand.

We are talking about host, not library.

[nxrighthere]

I would stick with the current proposal. Introducing a setter for IsTerminating is an anti-pattern. To me, the Ignore property sounds reasonable, this API at first glance is asking to pay attention carefully before using it, and that will be covered in the documentation.

[jkotas]

@janvorli @vitek-karas Could you please chime in your thoughts about the managed API shape? Once we get a local agreement, I am going to create API proposal issue to submit for API review approval.

[john-h-k]

Personally I think CanIgnore or IsFatal would be clearer than IsTerminating

[vitek-karas]

The proposal sounds good.

Do we know how mono/Xamarin behave in this area today? That is - is the IsTerminating also always true?

[janvorli]

The proposal makes sense to me too.

[jkotas]

Do we know how mono/Xamarin behave in this area today? That is - is the IsTerminating also always true?

I think so. @lambdageek Do you have any concerns about 1. from #39587 (comment) from the Mono side?

[lambdageek]

Do we know how mono/Xamarin behave in this area today? That is - is the IsTerminating also always true?

I think so. @lambdageek Do you have any concerns about 1. from #39587 (comment) from the Mono side?

Right now Mono always passes IsTerminating = true.

If I understand the proposal (1), we would pass IsTerminating = false for threads created by the runtime, and respect the Ignore property when the handlers return to the runtime - at which point the thread with the unhandled exception dies, but the runtime continues running.

I don't think this will be a problem for Mono.

---

From an embedding point of view, I think it would be interesting to think about foreign threads that are allowed to set Ignore = true. For example native platform threadpool threads on Apple or Android platforms. It's not really clear how the host can express a policy for the system threadpool threads. The host may pass some some managed callback to a native API or to a third-party library which queues a platform threadpool task behind the scenes. Seems like none of the components would have the complete picture.

[jkotas]

API proposal issue: #42275

[julianxhokaxhiu]

Is there any progress being made? I'm currently working on a Mod Manager for FF7 on PC ( https://github.com/tsunamods-codes/7th-Heaven ) and I've been impacted exactly by this. By hooking two APIs ( CloseHandle and DuplicateHandle ) using the Hostxfr, which do throw unhandled exceptions if the handle is invalid, makes the game crash. Having the possibility to ignore unhandled exceptions or handle them instead of force closing the process would be HIGHLY appreciated.

Thank you in advance.

[vladimir-cheverdyuk-altium]

You can use GetCLRRuntimeHost to initialize runtime second time and disable unhanded exceptions. I didn't full battle test it, but our huge app starts fine with it. I also test it with unhanded exception and it suppresses them.

It just must be at appropriate time.

[julianxhokaxhiu]

@vladimir-cheverdyuk-altium I'd be very interested to test this. Do you have a sample code I can use as a basis? Thanks!

[vladimir-cheverdyuk-altium]

I have it but it in Delphi (Pascal like language). If you want I can paste a few functions here.

[julianxhokaxhiu]

Yep, fine by me. I can relate, thanks :)

[vladimir-cheverdyuk-altium]

// hostfxr does not have ability to suppress unhandled exception. See https://github.com/dotnet/runtime/issues/39587
// As result I had to use obsolete way to do it.
// This function suppose to do nothing except setting up APPDOMAIN_SECURITY_FLAGS.APPDOMAIN_IGNORE_UNHANDLED_EXCEPTIONS

// Unfortunatelly we cannot use this way to initialize .NET runtime because this API do not initialize AssemblyLoadContext
// For more info check https://github.com/dotnet/runtime/issues/39167
Procedure DisableUnhandledExceptions;
Const
    cDefaultAppDomainId = 1; // There is only one domain in .NET Core
Type
    TGetCLRRuntimeHostFunc = Function(Const riid : TGUID) : IUnknown; Safecall;
Var
    ClrDirectory          : String;
    hCoreClr              : THandle;
    GetCLRRuntimeHostFunc : TGetCLRRuntimeHostFunc;
    Host                  : ICLRRuntimeHost2;
    DomainId              : DWORD;
    PropCount             : SIZE_T;
    PropKeys              : Array Of pchar_t;
    PropValues            : Array Of pchar_t;
    ErrorCode             : Int32;
Begin
    ClrDirectory := ExtractFilePath(GetRuntimeProperty('FX_DEPS_FILE'));

    hCoreClr := LoadLibraryEx(PChar(ClrDirectory + 'coreclr.dll'), 0, 0);
    If hCoreClr = 0 Then
        RaiseLastOSError(GetLastError, 'Failed to load CoreCLR');

    GetCLRRuntimeHostFunc := GetProcAddress(hCoreClr, 'GetCLRRuntimeHost');
    If Not Assigned(GetCLRRuntimeHostFunc) Then
        Raise Exception.Create('Cannot find "GetCLRRuntimeHost" function');

    Host := ICLRRuntimeHost2(GetCLRRuntimeHostFunc(ICLRRuntimeHost2));
    If Host = Nil Then
        Raise Exception.Create('Failed to create ICLRRuntimeHost2');

    // Host already initialized, so there is no need to do it again
    // Host.SetStartupFlags(STARTUP_FLAGS.STARTUP_CONCURRENT_GC Or STARTUP_FLAGS.STARTUP_SINGLE_APPDOMAIN Or STARTUP_FLAGS.STARTUP_LOADER_OPTIMIZATION_SINGLE_DOMAIN);
    Host.Start();

    PropCount := 0;
    // Get number of properties
    ErrorCode := gv_hostfxr_get_runtime_properties_fn(gv_HostfxrHandle, PropCount, Nil, Nil);
    If (ErrorCode <> StatusCode.Success) And (ErrorCode <> Int32(StatusCode.HostApiBufferTooSmall)) Then
        Raise Exception.Create('Failed to call hostfxr_get_runtime_properties');

    // Allocate arrays for names and values
    SetLength(PropKeys, PropCount);
    SetLength(PropValues, PropCount);

    // Retrieve names and values
    If gv_hostfxr_get_runtime_properties_fn(gv_HostfxrHandle, PropCount, @PropKeys[0], @PropValues[0]) <> StatusCode.Success  Then
        Raise Exception.Create('Failed to call hostfxr_get_runtime_properties');

    // There is only one domain possible to .NET Core, so we can create it here
    // We pass all properties from hostfxr because otherwise they will be empty
    DomainId := Host.CreateAppDomainWithManager(
        'Default Domain',
        APPDOMAIN_SECURITY_FLAGS.APPDOMAIN_ENABLE_PLATFORM_SPECIFIC_APPS Or
        APPDOMAIN_SECURITY_FLAGS.APPDOMAIN_ENABLE_PINVOKE_AND_CLASSIC_COMINTEROP Or
        APPDOMAIN_SECURITY_FLAGS.APPDOMAIN_DISABLE_TRANSPARENCY_ENFORCEMENT Or
        APPDOMAIN_SECURITY_FLAGS.APPDOMAIN_IGNORE_UNHANDLED_EXCEPTIONS,
        Nil,
        Nil,
        PropCount,
        @PropKeys[0],
        @PropValues[0]);

    // Just o check that it didn't fail
    If DomainId <> cDefaultAppDomainId Then
        // Only one domain can be created in .NET. Retriving default domain is not thread safe and as a result we will use constant
        Raise Exception.Create('Invalid domain id');
End;


Procedure InitializeHostFxr(HostPath, DotNetRoot, NativeImagesPaths, TrustedPlatformAssemblies : String);
Var
    HostDirectory                            : String;
    ErrorCode                                : Integer;
    HostfxrHandle                            : THandle;
    hostfxr_initialize_for_runtime_config_fn : hostfxr_initialize_for_runtime_config_type;
    hostfxr_get_runtime_delegate_fn          : hostfxr_get_runtime_delegate_type;
    InitializePameters                       : hostfxr_initialize_parameters;
Begin
    HostDirectory := ExtractFileDir(HostPath);
    HostfxrHandle := LoadLibraryEx(PChar(HostDirectory + '\hostfxr.dll'), 0, 0);
    If HostfxrHandle = 0 Then
        RaiseLastOSError(GetLastError, 'Failed to load CoreCLR');

    hostfxr_initialize_for_runtime_config_fn := FindProcAddress(HostfxrHandle, 'hostfxr_initialize_for_runtime_config');
    hostfxr_get_runtime_delegate_fn          := FindProcAddress(HostfxrHandle, 'hostfxr_get_runtime_delegate');
    gv_hostfxr_get_runtime_property_value_fn := FindProcAddress(HostfxrHandle, 'hostfxr_get_runtime_property_value');
    gv_hostfxr_set_runtime_property_value_fn := FindProcAddress(HostfxrHandle, 'hostfxr_set_runtime_property_value');
    gv_hostfxr_get_runtime_properties_fn     := FindProcAddress(HostfxrHandle, 'hostfxr_get_runtime_properties');

    InitializePameters.size        := SizeOf(InitializePameters);
    InitializePameters.host_path   := pchar_t(HostPath);
    InitializePameters.dotnet_root := pchar_t(DotNetRoot);

    ErrorCode := hostfxr_initialize_for_runtime_config_fn(pchar_t(ChangeFileExt(HostPath, '.runtimeconfig.json')), InitializePameters, gv_HostfxrHandle);
    If ErrorCode <> StatusCode.Success Then
        Raise Exception.CreateFmt('hostfxr_initialize_for_runtime_config returned error code %x', [ErrorCode]);

    // PROBING_DIRECTORIES will be populated from additionalProbingPaths but it will resolve relative paths based on current directory instead of directory that contains
    // application's .runtimeconfig.json file. As result I created new section called additionalAssemblyPaths in .runtimeconfig.json.
    // Function ReadAdditionalAssemblyPaths will read .runtimeconfig.json file, parse it and resolve relative papths manualy.
    SetRuntimeProperty('APP_PATHS', ReadAdditionalAssemblyPaths(ChangeFileExt(HostPath, '.runtimeconfig.json')));

    SetRuntimeProperty('APP_NI_PATHS', NativeImagesPaths);

    If TrustedPlatformAssemblies <> '' Then
    Begin
        // CreateDotNetDelegate without Assembly name parameter works only with types and classes from trusted platform assemblies. This is way to add assembly to that list
        // TRUSTED_PLATFORM_ASSEMBLIES will populate automatically during call to hostfxr_initialize_for_runtime_config_fn
        // More info: https://docs.microsoft.com/en-us/dotnet/core/dependency-loading/default-probing
        SetRuntimeProperty('TRUSTED_PLATFORM_ASSEMBLIES', GetRuntimeProperty('TRUSTED_PLATFORM_ASSEMBLIES') + ';' + TrustedPlatformAssemblies);
    End;

    // Required for AppContext.BaseDirectory property to work
    SetRuntimeProperty('APP_CONTEXT_BASE_DIRECTORY', HostDirectory);

    ErrorCode := hostfxr_get_runtime_delegate_fn(gv_HostfxrHandle, hostfxr_delegate_type.hdt_get_function_pointer, @gv_hostfxr_get_function_pointer_fn);
    If ErrorCode <> StatusCode.Success Then
        Raise Exception.CreateFmt('hostfxr_get_runtime_delegate returned error code %x', [ErrorCode]);

    ErrorCode := hostfxr_get_runtime_delegate_fn(gv_HostfxrHandle, hostfxr_delegate_type.hdt_load_assembly_and_get_function_pointer, @gv_hostfxr_load_assembly_and_get_function_pointer_fn);
    If ErrorCode <> StatusCode.Success Then
        Raise Exception.CreateFmt('hostfxr_get_runtime_delegate returned error code %x', [ErrorCode]);

    DisableUnhandledExceptions;

    // Just in case, we not suppose to finilize it because our application keeps .NET forever
End;

[julianxhokaxhiu]

Amazing! Thanks a lot @vladimir-cheverdyuk-altium very interesting approach, I'll try to translate this into C++ native code and I'll post here the results ASAP I'll be able to test this. Cheers!

[vladimir-cheverdyuk-altium]

Great!

[vladimir-cheverdyuk-altium]

Just in case this code is assume that framework will be packaged with application and there will be .runtimeconfig.json file that application used for own needs in different than intended way. I just pasted that code to demonstrate where you need to call DisableUnhandledExceptions.

[julianxhokaxhiu]

Yup, crystal clear how's the approach, I never thought about "abusing it" this way, pretty smart move. It might solve also my own issues. Thanks again, appreciated!

[julianxhokaxhiu]

Allright for anyone interested I translated this into C++, unfortunately doesn't fix my issue but the code works as intended:

#include <stdio.h>
#include "mscoree.h" // https://github.com/dotnet/coreclr/blob/master/src/pal/prebuilt/inc/mscoree.h

hostfxr_handle context = nullptr; // here lives the hostfxr context from hostfxr_initialize_for_runtime_config()
static const wchar_t* coreCLRInstallDirectory = L"%programfiles%\\dotnet\\shared\\Microsoft.NETCore.App\\7.0.0";
static const wchar_t* coreCLRDll = L"coreclr.dll";

// -------------------------------

// Check for CoreCLR.dll in a given path and load it, if possible
HMODULE LoadCoreCLR(const wchar_t* directoryPath)
{
    wchar_t coreDllPath[MAX_PATH];
    wcscpy_s(coreDllPath, MAX_PATH, directoryPath);
    wcscat_s(coreDllPath, MAX_PATH, L"\\");
    wcscat_s(coreDllPath, MAX_PATH, coreCLRDll);

    return LoadLibraryExW(coreDllPath, NULL, 0);
}

void DisableUnhandledExceptions()
{
    wchar_t coreRoot[MAX_PATH];
    ::ExpandEnvironmentStringsW(coreCLRInstallDirectory, coreRoot, MAX_PATH);
    HMODULE coreCLRModule = LoadCoreCLR(coreRoot);

    ICLRRuntimeHost2* runtimeHost;

    FnGetCLRRuntimeHost pfnGetCLRRuntimeHost =
        (FnGetCLRRuntimeHost)::GetProcAddress(coreCLRModule, "GetCLRRuntimeHost");

    // Get the hosting interface
    HRESULT hr = pfnGetCLRRuntimeHost(IID_ICLRRuntimeHost2, (IUnknown**)&runtimeHost);

    runtimeHost->Start();

    size_t num_props = 0;
    DWORD domainId;

    hostfxr_get_runtime_properties(context, &num_props, nullptr, nullptr);

    const char_t** fxr_keys = new const char_t * [num_props];
    const char_t** fxr_values = new const char_t * [num_props];

    hostfxr_get_runtime_properties(context, &num_props, fxr_keys, fxr_values);

    // Create the AppDomain
    hr = runtimeHost->CreateAppDomainWithManager(
        L"Default Domain",
        APPDOMAIN_ENABLE_PLATFORM_SPECIFIC_APPS ||
        APPDOMAIN_ENABLE_PINVOKE_AND_CLASSIC_COMINTEROP ||
        APPDOMAIN_DISABLE_TRANSPARENCY_ENFORCEMENT ||
        APPDOMAIN_IGNORE_UNHANDLED_EXCEPTIONS,
        NULL,
        NULL,
        num_props,
        fxr_keys,
        fxr_values,
        &domainId
    );
}

// Just call DisableUnhandledExceptions() after hostfxr_close();

Main differences from the above version:

• It loads the coreclr.dll from the latest .NET 7 framework
• It does not provide safe checks

Feel free to tweak at your own heart and desire. Thanks again @vladimir-cheverdyuk-altium for the snippet on your side!

[vladimir-cheverdyuk-altium]

Does it suppress unhanded exceptions?

[jkotas]

Please check #101560 and let us know if it addresses your scenario.

[vladimir-cheverdyuk-altium]

Hi @jkotas. It looks like what we want to. Thank you!

[elinor-fung]

I'm going to close this against #101560, as that seems to address the requested scenarios here.