App pool crashed due to System.ExecutionEngineException was thrown

[hieudole]

Hi @kleisauke,

I got System.ExecutionEngineException when my app was processing a specific image using NetVips 2.0.1 and NetVips.Native 8.11.4. Most of images were processed successfully, but it failed when processing several specific images. Although I had try catch all exceptions, my app pool crashed, and it was unable to recover.

This error was disappear after I upgraded the libraries to the latest version (NetVips 2.1.0, NetVips.Native 8.12.2). I think it was fixed in this version, but today I get this error again.

Managed Exception = System.ExecutionEngineException:
CallStack - Managed Exception

CallStack - Crashing Thread

 InlinedCallFrame
 InlinedCallFrame
 DomainBoundILStubClass.IL_STUB_PInvoke(NetVips.Operation)
 NetVips.Operation.Call(System.String, NetVips.VOption, NetVips.Image, System.Object[])
 NetVips.Image.Gravity(CompassDirection, Int32, Int32, System.Nullable`1<Extend>, Double[])

Is there any way to handle this exception to prevent app pool from crashing for the quick fix while waiting for a fix in native library?
Thank you!

[kleisauke]

Hi @hieudole,

I'm not sure if the upcoming libvips 8.13 would fix this. Are you able to provide a sample image and a standalone code sample that demonstrates this problem? Also, do you think this issue is related to #164?

If it's a bug in the native code, it's a bit weird that the (now obsolete) ExecutionEngineException is thrown. Typically, these native exceptions are mapped to .NET exceptions. For example, this code:

using var image = Image.Black(100, 100);
using var image2 = image.Bandjoin(Array.Empty<int>());

Would throw a AccessViolationException with libvips 8.12.2. This was fixed with commit libvips/libvips@a77a107.

Is there any way to handle this exception to prevent app pool from crashing for the quick fix while waiting for a fix in native library?

libvips 8.13 allows users to block any operation with the vips_operation_block_set() function. Prior to that version, you would need to disable foreign loaders/savers at compile time.

You could also prevent certain loaders from running using the Image.FindLoad* functions. For example:

net-vips/samples/NetVips.Samples/Samples/ThumbnailPipeline.cs

Lines 61 to 69 in fc320a7

	// Find the name of the load operation vips will use to load a buffer
	// so that we can work out what options to pass to NewFromBuffer().
	var loader = Image.FindLoadBuffer(buffer);
	if (loader == null)
	{
	// No known loader is found, stop further processing.
	throw new Exception("Invalid or unsupported image format. Is it a valid image?");
	}

if (loader.StartsWith("VipsForeignLoadHeif"))
{
    // Disable processing HEIF images.
    throw new Exception("HEIF support is disabled.");
}

Or, using the vips-loader property:

// The failOn option makes NetVips throw an exception on a file format error
using var image = Image.NewFromBuffer(buffer, failOn: Enums.FailOn.Error, access: Enums.Access.Sequential);

// vips-loader is the operation nickname, rather than the
// canonical name returned by vips_foreign_find_load().
var vipsLoader = (string)image.Get("vips-loader");
if (vipsLoader.StartsWith("heifload"))
{
    throw new Exception("HEIF support is disabled.");
}

[hieudole]

Hi @kleisauke,

I think this issue is not related to #164. I'm trying to reproduce it.

I also got System.AccessViolationException that might be related to the fix in libvips that you mentioned.

Managed Exception = System.AccessViolationException:Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
CallStack - Managed Exception
========================================================
     System.Web.Hosting.UnsafeIISMethods.MgdIsLastNotification(IntPtr, System.Web.RequestNotificationStatus)
     System.Web.HttpRuntime.FinishRequestNotification(System.Web.Hosting.IIS7WorkerRequest, System.Web.HttpContext, System.Web.RequestNotificationStatus ByRef)
     System.Web.HttpRuntime.OnRequestNotificationCompletionHelper(System.IAsyncResult)
     System.Web.HttpRuntime.OnRequestNotificationCompletion(System.IAsyncResult)
     System.Web.HttpAsyncResult.Complete(Boolean, System.Object, System.Exception, System.Web.RequestNotificationStatus)
     System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)
     System.Web.HttpApplication.ResumeStepsFromThreadPoolThread(System.Exception)
     System.Web.HttpApplication+CallHandlerExecutionStep.OnAsyncHandlerCompletion(System.IAsyncResult)

[hieudole]

Hi @kleisauke,

I could reproduce the System.ExecutionEngineException error in a console app using Image.NewFromStream() from NetVips 2.0.1, NetVips.Native 8.11.4. This happened on both .Net Framework and .Net Core.

using var stream = fileInfo.OpenRead(); 
var image = Image.NewFromStream(stream, access: Enums.Access.Sequential);

It took more than 10 seconds until throwing the unhandled exception. When I used Image.NewFromBuffer() to load the same image, it was working well. Anyway, I think it was fixed in the latest version of NetVips/NetVips.Native.

The System.ExecutionEngineException caused by the operation NetVips.Image.Gravity is the error that I cannot reproduce so far. It might be a bug in the native library.

[kleisauke]

The ExecutionEngineException in Image.NewFromStream might have been fixed with commit 8682c66 (released in NetVips 2.1.0). Disabling libvips' operation cache also ought to fix that (Cache.Max = 0).

The ExecutionEngineException caused by Image.Gravity sounds curious, I'm not sure what's causing that.

Just wondering, do you process untrusted images on your web server? If so, you might be interested in these security/performance tips for image processing with libvips: #96 (comment).

[hieudole]

No, I did not process untrusted images.

Thanks @kleisauke for the tips of processing images with libvips.

[kleisauke]

I'm removing the triage label, without a reproduction it'll be hard to debug further.

[kleisauke]

@hieudole Were you able to make any progress with this?

[hieudole]

@kleisauke I'm closing this issue since I'm not able to reproduce it.