HTTP model enhancements

[fabiocav]

This epic tracks the issues/work items that are part of the out-of-proc (and more specifically, .NET Isolated) HTTP model enhancements that will bring support for more advanced HTTP scenarios, HTTP performance improvements and deeper integration with ASP.NET.

The current state: what we're solving

The HTTP model exposed by the host to out-of-proc workers relies on a representation of the HTTP request being passed over the worker gRPC channel as part of the function invocation. The HTTP response representation is returned to the host using the same channel in the invocation result.

This is the model all out-of-proc workers (Node, Java, Python, PowerShell and .NET) use, and some of its key limitations are:

• The worker has no access to the request/response streams.
• Integration with HTTP frameworks in out-of-proc scenarios is difficult.
  • Although possible for simple cases, integrations based on the existing model are very fragile and have several limitations, leading to reliability problems, surprising/inconsistent behavior and poor development experiences.
• Significant performance impact
• Higher resource utilization
• Inability to support common use cases supported by the in-proc model (e.g., file/large blob transfers, streaming scenarios, etc.)

The HTTP model supported by the .NET Isolated worker operates within those constraints, utilizing HTTP request and response representation types (HttpRequestData and HttpResposeData) that expose APIs reflecting what the current HTTP model supports.

We decided to take this approach, instead of providing implementations of HttpRequest and HttpResponse on top of the existing ASP.NET model for two main reasons:

1. Because of the limitations described above, several of the commonly used APIs and features supported by the ASP.NET model would either not work or, worse yet, not work as expected.
2. From the beginning, the plan was to proceed with the plans outlined in this epic, which would leave us with two different implementations, with different underlying models, based on the same set of abstractions. Because of previous experiences with in-proc model, we wanted to avoid that as it leads to confusion and difficult documentation, support and development experiences.

Where we are headed: The new HTTP model

With the new HTTP model, the host will become an intermediary and and act as a reverse proxy, proxying the actual HTTP request to the worker for handling.

Workers supporting this new model will be enhanced to listen on an HTTP endpoint, in addition to the gRPC channel used for invocations, allowing them to maintain the existing support for input and output bindings, with the existing execution model, while exposing a parallel pipeline used for the HTTP request payload.

The following diagram illustrates, at a high level, the flow proposed with the new HTTP model:

flowchart LR
    A[Host]--> |Request| B[HTTP]
    A[Host]--> |Invocation| C[gRPC]
    subgraph Worker
    B-->D
    C-->D
    D[Dispatcher] --> Function
    end

Loading

With the flow above, the host will split the HTTP request and function invocation message, creating a correlation between the two. The HTTP request goes through the HTTP pipeline while the function invocation, with all associated input binding data and context, goes through the existing dispatching pipeline. Eventually, the pipelines merge, where the worker augments the FunctionContext with the HttpRequest information coming from the HTTP pipeline and exposing that to the Function (both, in the context and/or in automatic parameter binding).

With the new model, a simple HTTP triggered function would change from this:

[Function("HttpFunction")]
public HttpResponseData Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequestData req,
    FunctionContext executionContext)
{
    var response = req.CreateResponse(HttpStatusCode.OK);
 
    response.WriteString("Welcome to .NET isolated worker !!");

    return response;
}

To the following:

[Function("HttpFunction")]
public IActionResult Run(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequest req, FunctionContext executionContext)
{
    return new OkObjectResult("Welcome to .NET isolated worker !!");
}

This model will also enable deeper integrations with web frameworks (e.g., ASP.NET), providing better control over the HTTP pipeline and enabling scenarios not currently supported in the in-proc experience, like the ability to register and manage ASP.NET middleware and better control over authentication/authorization.

Work items

This epic will be updated with references to the work items associated with the HTTP model enhancements, as those are defined.

Host Items

Host items can be found under this label.

Worker Items

Worker items will use the same label as this issue (feature:HTTP types).

[bradygaster]

Is there a corresponding issue for the template change resulting from this nice improvement? I have some thoughts on the template I'd like to posit if there's an existing issue.

[paulyuk]

Like this direction @fabiocav .

Ideas:

• Can we assume async Task<T> Run (..).. as the main case especially for POST?
• can we see more simple examples with a GET query param and a POST body data? Would love those to be easy for common cases
• Could we have overloads that take POCO classes/records as inputs and outputs? In an API I'd want that.

[fabiocav]

Is there a corresponding issue for the template change resulting from this nice improvement? I have some thoughts on the template I'd like to posit if there's an existing issue.

Created #1465 to track. Let's discuss additional templates we want to consider there.

[rodolfograve]

In @fabiocav's response in #366 (1 month ago) there is an explicit acceptance that whilst the improvements proposed in this issue are great and we are all looking forward to them, there is a considerable number of people (including myself) who need a more immediate way to handle file uploads:

With that said, features like the one tracked by this issue are important quality-of-life improvements and we should be addressing with the current model before we get to where we want to be, so I'm sorry this was deprioritized and will once again mark this issue for implementation.

Can someone, please, provide an update on when can we expect those "quality-of-life improvements" addressed with the current model?

[rodolfograve]

Anyone from the team, please? ^^^

[rodolfograve]

It has been almost 4 months since an apology about lack of updates was posted by this team (#366 (comment))

Folks, sincere apologies for the lack of updates on this issue.

I agree that communication needs to improve to make sure we're providing more frequent updates on those long running efforts, otherwise, the perception is that this area is not getting the attention it needs. I also realize that it is difficult to track those that work with issues and activity across the different repos. We'll be discussing actions on this in the coming week.

On that same post, there seemed to be an understanding that this issue could not wait until the "new model" was ready and that "quality-of-life improvements" would be done.

It has been almost 2 months since my original question in #366 and over 3 weeks here. There has been a complete silence from this team.

All I'm asking is for some update on what was offered. We need to make a decision on how to implement file uploads and are completely blocked by the lack of support in the code (which is understandable, as writing code takes time and effort), but also by the complete absence of updates from the team, which is less understandable and feels unprofessional and uncaring about your customers.

[fabiocav]

@rodolfograve I'll continue providing updates on that thread.

Updating here that the initial preview of the work tracked by this epic is now in preview: https://learn.microsoft.com/en-us/azure/azure-functions/dotnet-isolated-process-guide#aspnet-core-integration-preview

[davidpeden3]

Not all features of ASP.NET Core are exposed by this model. Specifically, the ASP.NET Core middleware pipeline and routing capabilities are not available. In the initial preview versions of the integration package, route info is missing from the HttpRequest and HttpContext objects, and accessing route parameters should be done through the FunctionContext object or via parameter injection.

@fabiocav Will the middleware pipeline be added? If not, why not? If so, will the builder models IFunctionWorkerApplicationBuilder and WebAppkicationBuilder be consolidated or at least normalized by a shared interface?

It would be very helpful to be able to use standard middlewares in an azure function pipeline.

[SeanFeldman]

Would be nice if the samples in the repo would contain a sample for the enhanced HTTP model.

[fabiocav]

Would be nice if the samples in the repo would contain a sample for the enhanced HTTP model.

Actively working on that! :)

[SeanFeldman]

Actively working on that! :)

I'll stop reading your mind 😃

[fabiocav]

@fabiocav Will the middleware pipeline be added? If not, why not? If so, will the builder models IFunctionWorkerApplicationBuilder and WebAppkicationBuilder be consolidated or at least normalized by a shared interface?

It would be very helpful to be able to use standard middlewares in an azure function pipeline.

@davidpeden3 eventually, yes. We plan on exposing the ASP.NET Core pipeline for full customization, but the initial release is targeting parity with the in-proc experience, and once that is in place, we'll GA the feature so folks can confidently use it in production. The additional enhancements will come after, but not too far behind.

[thushanperera95]

@rodolfograve I'll continue providing updates on that thread.

Updating here that the initial preview of the work tracked by this epic is now in preview: https://learn.microsoft.com/en-us/azure/azure-functions/dotnet-isolated-process-guide#aspnet-core-integration-preview

This works great locally, but I'm having some trouble getting it to work when deployed to Azure.

It never manages to get past this step

It almost feels like the AzureWebJobsFeatureFlags is being ignored but this has definitely been set in the Application Settings.

Any ideas?

[thushanperera95]

Looks like there's some sort of blocking issue

[thushanperera95]

Seems to be working now 🤷

[kshyju]

Seems to be working now 🤷

Glad to know. Did you make any changes from your side to make it work? Would you mind sharing? Thanks!

[thushanperera95]

Seems to be working now 🤷

Glad to know. Did you make any changes from your side to make it work? Would you mind sharing? Thanks!

I had to remove the FunctionContext. Which I think is related to this locking thing here.

Looks like there's some sort of blocking issue

Anyway FunctionContext was only used to access the logger instance, which I am now injecting into the constructor.

[mattchenderson]

Edit 2023-06-21: moving 1543, and we're actually treating 164 as separate since it's really a general HTTP enhancement not scoped to ASP.NET integration.
Edit 2023-06-16: These items are being moved to https://github.com/Azure/azure-functions-dotnet-worker/milestone/58

Outlining a possible preview2 update scope:

• Consolidate to one extension method to enable ASP.NET Core integration #1597 (moved to milestone)
• ASP.NET Core integration (preview) breaks Timer Triggers #1584 (moved to milestone)
• Blocking issue with ASP.NET Core integration #1583 (moved to milestone)
• [Enhanced Http] Using GetHttpRequestDataAsync in enhanced http model #1543 (moved to milestone)
• Register HTTP trigger routes #1527 (moved to milestone)
• HTTP trigger - obtaining route information in middleware #1493 (moved to milestone)
• HttpTrigger with complex type does not work #164

Another short-term item that is not a part of preview2 since it is in the host, but which we wish to track here: Azure/azure-functions-host#9320

[fabiocav]

Closing this as resolved now that we have the ASP.NET Core integration out and in GA state.

If you run into any problems, please open a separate issue.

Thank you!