Creating custom input and output bindings

This wiki describes how to define a custom binding extension for the WebJobs SDK. These same extensions can be used, without modification, in Azure Functions. For convenience, this article mentions "WebJobs extensions," but these are also "Azure Functions extensions."

Bindings must be authored in .NET, but can be consumed from any supported language. For example, as long as there is a JObject or JArray conversion, a custom binding can be used from a JavaScript Azure Function.

This wiki focuses on how to define custom input and output bindings. Custom triggers are not available for Azure Functions.

For more information on how the binding process works, see the wiki in the WebJobs SDK extensions repo.

Binding extensions are meant to be authored in a declarative fashion, with the framework doing most of the heavy lifting. This is accomplished through binding rules and converters.

To author a custom binding for Azure Functions, you must use the 2.0 runtime. See Azure Functions 2.0 support.

Sample extensions

For sample binding extensions, see:

• Service Bus Extension
• Event Grid Extension
• Durable Functions
• Microsoft Graph
• Slack output binding sample

Overview

To author an extension, you must perform the following tasks:

1. Declare an attribute, such as [Blob]. Attributes are how customers consume the binding.
2. Choose one or more binding rules to support.
3. Add some converters to make the rules more expressive.

Terms

Bindings generally wrap an SDK provided by a service (e.g., Azure Storage, Event Hub, Dropbox, etc.). Here, we used the term native SDK to describe the SDK being wrapped. The SDK can then expose native types. For example, you can interact with Azure queue storage using the WindowsAzure.Storage nuget package, using native types such as CloudQueueMessage.

An extension should provide the following:

• Bindings that expose the SDK native types.
• Bindings to BCL types (like System, byte[], stream) or POCOs. That way, customers can use the binding without having to directly use the native SDK.
• Bindings to JObject and JArray. This enables the binding to be consumed from non-.NET languages, such as JavaScript and Powershell.

Consuming a custom binding

To use the binding in C#, simply add a reference to the project or assembly and use the binding via attributes. When you run locally or in Azure, the extension will be loaded.

For JavaScript, the process is currently more manual. Do the following:

1. Copy the extension to an output folder such as "extensions". This can be done in a post-build step in the .csproj
2. Add the app setting AzureWebJobs_ExtensionsPath to local.settings.json (or in Azure, in App Settings). Set the value to the parent of your "extension" folder from the previous step.

The binding pipeline

The WebJobs SDK does the following when it encounters a binding [MyBinding] on a type T:

1. Look up an extension MyExtension that implements IExtensionConfigProvider and supports MyBindingAttribute.
2. Apply the binding rule(s) that have been defined for MyExtension.
3. Apply converters for types T.

1. Define binding attribute(s)

A binding attribute is simply a .NET attribute with the [Binding] meta-attribute applied. For example, this is the definition of DocumentDBAttribute:

[AttributeUsage(AttributeTargets.Parameter | AttributeTargets.ReturnValue)]
[Binding]
public sealed class DocumentDBAttribute : Attribute

For more information, see the wiki page Binding Attributes.

App Settings

Bindings can support the use of app settings. This enables customers to manage secrets and connection strings using app settings rather than configuration files. For more information on how this feature is used, see Resolving app settings.

Supporting app settings in your extension is quite easy: just apply the [AppSetting] attribute.

For example, the Event Hub binding allows the Connection attribute to be an app setting:

public sealed class EventHubAttribute : Attribute, IConnectionProvider
{       
    // Other properties ... 

    [AppSetting]
    public string Connection { get; set; }
}   

Binding expressions

Most bindings also support binding expressions. These are patterns that can be used in other bindings or as method parameters. See Binding expressions and patterns.

Binding expressions are also easy to enable, just add the attribute [AutoResolve]. This provides the following features:

• AppSetting support (with % signs)
• {key} values. Since these are resolved per trigger, the values can be runtime data.

For example, the path parameter of the [Blob] attribute supports auto-resolve. Customers can write code such as:

class Payload { public string name {get;set; } }

void Foo([QueueTrigger] Payload msg, [Blob("%container%/{name}")] TextReader reader)  { ... }

Here, %container% is resolved at startup based on the app setting value. Since Foo is triggered on a queue message, the trigger provides a runtime value for name based on the queue payload. So, if the container appsetting value is storagetest and the queue receives a message with name = 'bob', then the blob path would be invoked with 'storagetest/bob'.

For example, here's the attribute definition for a Slack binding (see SlackOutputBinding sample).

[AttributeUsage(AttributeTargets.Parameter | AttributeTargets.ReturnValue)]
[Binding]
public sealed class SlackAttribute : Attribute
{
    [AppSetting(Default = "SlackWebHookKeyName")]
    public string WebHookUrl { get; set; }

    [AutoResolve]
    public string Text { get; set; }

    [AutoResolve]
    public string Username { get; set; }

    [AutoResolve]
    public string IconEmoji { get; set; }

    [AutoResolve]
    public string Channel { get; set; }
}

Validation

You can use the standard System.ComponentModel.DataAnnotations attributes on the properties of your attribute to apply validation rules. For example, you might use [RegularExpressionAttribute].

The SDK runs this validation as early as possible. If there are no "{ }" tokens, then validation is run at index time. If there are "{ }" tokens, then validation is done at runtime, after the [AutoResolve] substitution.

2. Define binding rules

The extension itself is defined by implementing IExtensionConfigProvider. For more information on registering extensions, see Extension Registration.

The key method is void Initialize(ExtensionConfigContext context).

Binding rules provide strong semantics and support several common patterns. An extension describes which rules it supports, and then the SDK picks the appropriate rule based on the target functions' signature.

The rules are:

1. BindToInput. Just what it says, bind to an input object.
2. BindToCollector. Bind to output via IAsyncCollector. This is used in output bindings for sending discrete messages like Queues and EventHub.
3. BindToStream. Bind to stream based systems. This rule is useful for blob, file, DropBox, ftp, etc.

BindToInput rule

Use this rule to bind to a single input type like CloudTable, CloudQueue, etc. For example:

IConverter<TAttribute, TObject> builder = ...; // builder object to create a TObject instance
bf.BindToInput<TAttribute, TObject>(builder);

Here, TAttribute is the type of the attribute to which this rule applies. TType is the type of the target parameter in the user's function signature.

The IConverter in the BindToInput rule is run per invocation. Your code should transform an attribute, with resolved values for [AppSetting] and [AutoResolve], into an input type.

Here is an example of registering a basic input rule. See WebJobsExtensionSamples.

public class SampleExtensions : IExtensionConfigProvider
{
    /// <summary>
    /// This callback is invoked by the WebJobs framework before the host starts execution. 
    /// It should add the binding rules and converters for our new <see cref="SampleAttribute"/> 
    public void Initialize(ExtensionConfigContext context)
    {
        context.AddConverter<SampleItem, string>(ConvertToString);

        // Create an input rules for the Sample attribute.
        var rule = context.AddBindingRule<SampleAttribute>();

        rule.BindToInput<SampleItem>(BuildItemFromAttr);
    }

    private SampleItem BuildItemFromAttr(SampleAttribute attribute)
    {
        ... 
        return new SampleItem
        {
            Name = attribute.FileName,
            Contents = contents
        };
    }    
}

A user function can use the binding as follows:

public void ReadSampleItem(
    [Queue] string name, 
    [Sample(FileName = "{name}")] SampleItem item,
    TextWriter log)
{
    log.WriteLine($"{item.Name}:{item.Contents}");
}

BindToCollector rule

Use this rule to support the output of discrete messages, such as a queue message.

For example, the [Table] attribute supports binding to IAsyncCollector<ITableEntity>, which is accomplished as follows:

var rule = context.AddBindingRule<TableAttribute>();
rule.BindToCollector<ITableEntity>(builder);

A single BindToCollector rule enables multiple patterns:

User Parameter Type	Becomes
IAsyncCollector<T>	Identity
ICollector<T>	Sync wrapper around IAsyncCollector<T>
out T item	ICollector<T> collector; collector.Add(item);
out T[] array	ICollector<T> collector; foreach(var item in array) collector.Add(item);

This also automatically applies any applicable converters that have been registered with IConverterManager.

BindToStream

Use this rule to support binding to a System.IO.Stream. For example this is used by the Azure Storage extension to allow user code to interact with a blob as a stream. The storage extension uses this rule here

3. Add Converters

Converters can be used for both BindToInput and BindToCollector.

Suppose you use BindToCollector to support IAsyncCollector<AlphaType>. If you define a converter from AlphaType to BetaType, the SDK also binds to IAsyncCollector<BetaType>.

Here's an example of defining a conversion from a string to SampleItem for an input binding:

public void Initialize(ExtensionConfigContext context)
{
    ...
    context.AddConverter<SampleItem, string>(ConvertToString);
    ...
}

private SampleItem ConvertToItem(string arg)
{
    var parts = arg.Split(':');
    return new SampleItem
    {
            Name = parts[0],
            Contents = parts[1]
    };
}

Now a user function can use a string instead of SampleItem:

public void Reader(
    [QueueTrigger] string name,  // from trigger
    [Sample(FileName = "{name}")] string contents)
{
    ...
}

Binding to generic types with OpenTypes

To enable binding to a POCO, the SDK has a sentinel type, called OpenType. This is a placeholder for a generic type T. This type is required because the extension's Initialize method is not generic and cannot directly refer to a type T in its implementation.

For example, you could register a generic SampleItem to T converter via:

cm.AddConverter<SampleItem, CustomType<OpenType>>(typeof(CustomConverter<>));

Here, the builder is generic. The SDK will determine the correct value for T:

private class CustomConverter<T> : IConverter<SampleItem, CustomType<T>>
{
    public CustomType<T> Convert(SampleItem input)
    {
        // Do some custom logic to create a CustomType<>
        var contents = input.Contents;
        T obj = JsonConvert.DeserializeObject<T>(contents);
        return new CustomType<T>
        {
            Name = input.Name,
            Value = obj
        };
    }
}

This enables the following user function:

public class CustomType<T>
{
    public string Name { get; set; }
    public T Value;
}

public void AnotherReader(
    [Queue] string name, 
    [Sample(Name = "{name}")] CustomType<int> item,
    TextWriter log)
{
    log.WriteLine($"Via custom type {item.Name}:{item.Value}");
}

Constraints on types.

OpenType is a base class with a method IsMatch(Type t). In general, the SDK does pattern matching for to enable types like OpenType[] and ISomeInterface<OpenType>.

If you have additional constraints on the POCO definition (such as to require an "Id" property), create a subclass of OpenType and override IsMatch to implement the constraint.

More details

See the ConverterManagerTests for more examples of conversions: https://github.com/Azure/azure-webjobs-sdk/blob/dev/test/Microsoft.Azure.WebJobs.Host.UnitTests/ConverterManagerTests.cs.

Extending extensions

The converter manager is centralized and shared across extensions. That means extensions can extend other extensions. For example, you could extend the existing [Table] binding by adding a CloudTable to MyCustomObject<T> converter.

The blob binding is not yet extensible; see Support Blob POCO bindings #995.

Implicit conversions

The converter manager allows some implicit conversions. For example, if TDest is assignable from TSrc, it provides an implicit conversion.

OpenTypes and Rules

You can also use OpenType with binding rules. For example, you may want to use an intermediate converter to support a direct binding to generic parameters.

Example built-in bindings

• Table
• Queue
• EventHub