API Proposal: C# 10 interpolated strings support, part 2

[stephentoub]

Background and Motivation

Part 1 covers the APIs underlying interpolated strings when building strings directly.

But the C# 10 support, as outlined in https://github.com/dotnet/csharplang/blob/main/proposals/improved-interpolated-strings.md#improved-interpolated-strings, enables the compiler to target a builder pattern, and we can use that builder pattern to enable additional scenarios beyond just string s = $"...".

There are multiple parts to this proposal, each of which stands independently and enables a different but related scenario (with the exception of the first section, an attribute that will be used by the rest).

Proposed API: Attribute for Passing State Into Builder Create Methods

TBD: There will be some attribute that can be placed on one or more method arguments to signal to the compiler it should forward those into the builder’s Create method. See usage in later sections, but its design is still underway.

namespace System.Runtime.CompilerServices
{
    [AttributeUsage(AttributeTargets.Parameter, AllowMultiple = false, Inherited = false)]
    public sealed class InterpolatedBuilderArgumentAttribute : Attribute
    {
        public InterpolatedBuilderArgumentAttribute(string argument);
        public InterpolatedBuilderArgumentAttribute(params string[] arguments);

        public string[] Arguments { get; }
    }
}

Proposed API: Writing into Destination Spans

It is becoming increasingly common to format into spans, which enables formatting into stack-allocated memory, into existing buffers code may have, into pooled arrays, and so on. Such formatting today for composite operations involves manually tracking how many characters have been written, slicing the target buffer, and doing length checks for each component. With interpolated strings, we can push all of that handling to the compiler. For example, consider a type like:

struct Point
{
    public int X, Y;
}

It can implement ISpanFormattable like:

public bool TryFormat(Span<char> destination, out int charsWritten, ReadOnlySpan<char> format, IFormatProvider? provider)
{
    charsWritten = 0;
    int tmpCharsWritten = 0;

    if (!X.TryFormat(destination, out int tmpCharsWritten, default, provider)) return false;
    destination = destination.Slice(tmpCharsWritten);

    if (destination.Length < 2) return false;
    ", ".AsSpan().CopyTo(destination);
    tmpCharsWritten += 2;
    destination = destination.Slice(2);

    if (!Y.TryFormat(destination, out int tmp, default, provider)) return false;
    charsWritten = tmp + tmpCharsWritten;
    return true;
}

We can instead enable a developer to write:

public bool TryFormat(Span<char> destination, out int charsWritten, ReadOnlySpan<char> format, IFormatProvider? provider) =>
    destination.TryWrite(provider, $"{X}, {Y}", out charsWritten) ;

by exposing the following API:

namespace System
{
    public static class MemoryExtensions
    {
        public static bool TryWrite(this Span<char> span, [InterpolatedBuilderArgument("span")] ref InterpolatedSpanBuilder builder, out int charsWritten);
        public static bool TryWrite(this Span<char> span, IFormatProvider? provider, [InterpolatedBuilderArgument("span", "provider")] ref InterpolatedSpanBuilder builder, out int charsWritten);
        …
    }
}

namespace System.Runtime.CompilerServices
{
    public ref struct InterpolatedSpanBuilder
    {
        public static InterpolatedSpanBuilder Create(int literalLength, int formattedCount, Span<char> destination, out bool success);
        public static InterpolatedSpanBuilder Create(int literalLength, int formattedCount, Span<char> destination, IFormatProvider? provider, out bool success);

        // Same members as on InterpolatedStringBuilder
        public bool AppendLiteral(string s);
        public bool AppendFormatted<T>(T value);
        public bool AppendFormatted<T>(T value, string? format);
        public bool AppendFormatted<T>(T value, int alignment);
        public bool AppendFormatted<T>(T value, int alignment, string? format);
        public bool AppendFormatted(ReadOnlySpan<char> value);
        public bool AppendFormatted(ReadOnlySpan<char> value, int alignment = 0, string? format = null);
        public bool AppendFormatted(string? value);
        public bool AppendFormatted(string? value, int alignment = 0, string? format = null);
        public bool AppendFormatted(object? value, int alignment = 0, string? format = null);
    }
}

For the previously cited example, the compiler will generate code akin to:

public bool TryFormat(Span<char> destination, out int charsWritten, ReadOnlySpan<char> format, IFormatProvider? provider)
{
    var builder = InterpolatedSpanBuilder.Create(2, 2, destination, provider, out bool success);
    _ = success &&
        builder.AppendFormatted(X) &&
        builder.AppendLiteral(", ") &&
        builder.AppendFormatted(Y);
    return MemoryExtensions.TryWrite(destination, ref builder, out int charsWritten, provider);
}

Issues for discussion:

1. CompilerServices or nested type. The above proposal defines InterpolatedSpanBuilder to live in System.Runtime.CompilerServices. Should it instead be a nested type inside of MemoryExtensions, and if so, should it be named something more like InterpolatedTryWriteBuilder?
2. TryWrite naming. I initially had TryFormat, but most TryFormat methods are about formatting the receiver into the destination span, whereas the structure of this API has the span as the receiver, and it’s formatting one of the arguments, so swapping the normal positions. TryWrite seemed more apt. Another option would be TryAppend.

Proposed API: String Formatting With a Provider

#50601 covered basic support for string interpolation using the new builder support. But it didn’t cover customizing that to support custom IFormatProviders. We do that by exposing a new string API and augmenting the builder proposed in the previous issue:

namespace System
{
    public sealed class String
    {
        public static string Create(IFormatProvider? provider, [InterpolatedBuilderArgument("provider")] ref InterpolatedStringBuilder builder);
        …
    }
}

namespace System.Runtime.CompilerServices
{
    public ref struct InterpolatedStringBuilder // from https://github.com/dotnet/runtime/issues/50601
    {
        public static InterpolatedStringBuilder Create(int literalLength, int formattedCount, IFormatProvider? provider); // additional factory
        …
    }
}

This enables a developer to write code like:

string s = string.Format(CultureInfo.InvariantCulture, $"{X}, {Y}");

which the compiler will translate into approximately:

var tmp = CultureInfo.InvariantCulture;
var builder = InterpolatedStringBuilder.Create(2, 2, tmp);
builder.AppendFormatted(X);
builder.AppendLiteral(", ");
builder.AppendFormatted(Y);
string s = string.Format(tmp, ref builder);

Proposed API: String Formatting With Stack Allocation

The above made it possible for a developer to pass in an IFormatProvider. However, for best efficiency, we want to enable a developer to pass in additional state: a scratch buffer that the formatting can use as part of its operation. Often this scratch space will be stackalloc’d space, which is often enough to handle a significant number of formatting operations.

namespace System
{
    public sealed class String
    {
        public static string Create(IFormatProvider? provider, Span<char> scratchBuffer, [InterpolatedBuilderArgument("provider", "scatchBuffer")] ref InterpolatedStringBuilder builder);
        …
    }
}

namespace System.Runtime.CompilerServices
{
    public ref struct InterpolatedStringBuilder // from https://github.com/dotnet/runtime/issues/50601
    {
        public static InterpolatedStringBuilder Create(int literalLength, int formattedCount, IFormatProvider? provider, Span<char> scratchBuffer); // additional factory
        …
    }
}

This enables a developer to write code like:

string s = string.Create(null, stackalloc char[32], $"{X}, {Y}");

which the compiler will translate into approximately:

Span<char> scratchBuffer = stackalloc char[32];
var builder = InterpolatedStringBuilder.Create(2, 2, null, scratchBuffer);
builder.AppendFormatted(X);
builder.AppendLiteral(", ");
builder.AppendFormatted(Y);
string s = string.Format(null, scratchBuffer, ref builder);

The method makes no guarantees about using the scratch buffer, but it’s allowed to use it however it wants for the duration of the operation. If the buffer is not large enough for the formatting, the builder will grow to using ArrayPool buffers as it otherwise would if no scratch buffer was provided.

Proposed API: Appending to a StringBuilder

StringBuilder today has multiple AppendFormat overloads today that take an optional provider, the format string, and arguments. Just as with string formatting, we can provide an AppendFormat overload that accepts a builder, to make this a bit more efficient while also allowing the more convenient syntax. The builder’s TryFormat methods will delegate to corresponding functionality on the StringBuilder.

namespace System.Text
{
    public sealed class StringBuilder
    {
        public StringBuilder Append([InterpolatedBuilderArgument("this")] ref InterpolatedAppendFormatBuilder builder);
        public StringBuilder Append(IFormatProvider? provider, [InterpolatedBuilderArgument("this", "provider")] ref InterpolatedAppendFormatBuilder builder);

        public StringBuilder AppendLine([InterpolatedBuilderArgument("this")] ref InterpolatedAppendFormatBuilder builder);
        public StringBuilder AppendLine(IFormatProvider? provider, [InterpolatedBuilderArgument("this", "provider")] ref InterpolatedAppendFormatBuilder builder);

        public struct InterpolatedAppendFormatBuilder
        {
            public static InterpolatedAppendFormatBuilder Create(int literalLength, int formattedCount, StringBuilder stringBuilder);
            public static InterpolatedAppendFormatBuilder Create(int literalLength, int formattedCount, StringBuilder stringBuilder, IFormatProvider? provider);

            // Same members as on InterpolatedStringBuilder
            public void AppendLiteral(string s);
            public void AppendFormatted<T>(T value);
            public void AppendFormatted<T>(T value, string? format);
            public void AppendFormatted<T>(T value, int alignment);
            public void AppendFormatted<T>(T value, int alignment, string? format);
            public void AppendFormatted(ReadOnlySpan<char> value);
            public void AppendFormatted(ReadOnlySpan<char> value, int alignment = 0, string? format = null);
            public void AppendFormatted(string? value);
            public void AppendFormatted(string? value, int alignment = 0, string? format = null);
            public void AppendFormatted(object? value, int alignment = 0, string? format = null);
        }

        …
    }
}

This enables a developer to write code like:

StringBuilder sb = …;
sb.Append($"{X}, {Y}");

which the compiler will translate into approximately:

StringBuilder sb = …;
var builder = InterpolatedAppendFormatBuilder.Create(2, 2, sb);
builder.AppendFormatted(X);
builder.AppendLiteral(", ");
builder.AppendFormatted(Y);
sb.Append(ref builder);

Issues for discussion:

1. CompilerServices or nested type. The above proposal defines InterpolatedAppendFormatBuilder to be a type nested within StringBuilder. Should it instead live in System.Runtime.CompilerServices?
2. Append. There are a fair number of examples today of code doing sb.Append($"...");. If we were to add the AppendFormat functionality instead (or also) under the Append name, those existing uses would get a lot better automatically upon recompilation.

[bartonjs]

We didn't finish this in API Review, here were the notes so far

• InterpolatedBuilderArgumentAttribute: AllowMultiple should be false
• String.Format(IFormatProvider, InterpolatedStringBuilder) should take the builder as ref (because it's a mutable ref struct)

[bartonjs]

Video

• Builders that are nested types should be [EditorBrowsable(Never)]
• Change System.Runtime.CompilerServices.InterpolatedSpanBuilder to System.MemoryExtensions+InterpolatedTryWriteBuilder
• Interpolated string target methods on StringBuilder should be Append, not AppendFormat, since no separate format string is supplied.
• Also do it for StringBuilder.AppendLine
• The scratchBuffer String.Format should an overload of String.Create
  • Therefore the IFormatProvider and builder one should also be String.Create
• We're not fully happy with the name scratchBuffer, but we're not sure what a good answer is. The adjective temporary received a minor happy response. temporaryBuffer?

namespace System.Runtime.CompilerServices
{
    [AttributeUsage(AttributeTargets.Parameter, AllowMultiple = false, Inherited = false)]
    public sealed class InterpolatedBuilderArgumentAttribute : Attribute
    {
        public InterpolatedBuilderArgumentAttribute(string argument);
        public InterpolatedBuilderArgumentAttribute(params string[] arguments);

        public string[] Arguments { get; }
    }

    public ref struct InterpolatedStringBuilder // from https://github.com/dotnet/runtime/issues/50601
    {
        public static InterpolatedStringBuilder Create(int literalLength, int formattedCount, IFormatProvider? provider); // additional factory
        public static InterpolatedStringBuilder Create(int literalLength, int formattedCount, IFormatProvider? provider, Span<char> scratchBuffer); // additional factory
        …
    }
}

namespace System
{
    public static class MemoryExtensions
    {
        public static bool TryWrite(this Span<char> span, [InterpolatedBuilderArgument("span")] ref InterpolatedSpanBuilder builder, out int charsWritten);
        public static bool TryWrite(this Span<char> span, IFormatProvider? provider, [InterpolatedBuilderArgument("span", "provider")] ref InterpolatedSpanBuilder builder, out int charsWritten);
        …

        [EditorBrowsable(Never)]
        public ref struct InterpolatedTryWriteBuilder
        {
            public static InterpolatedTryWriteBuilder Create(int literalLength, int formattedCount, Span<char> destination, out bool success);
            public static InterpolatedTryWriteBuilder Create(int literalLength, int formattedCount, Span<char> destination, IFormatProvider? provider, out bool success);
    
            // Same members as on InterpolatedStringBuilder
            public bool AppendLiteral(string s);
            public bool AppendFormatted<T>(T value);
            public bool AppendFormatted<T>(T value, string? format);
            public bool AppendFormatted<T>(T value, int alignment);
            public bool AppendFormatted<T>(T value, int alignment, string? format);
            public bool AppendFormatted(ReadOnlySpan<char> value);
            public bool AppendFormatted(ReadOnlySpan<char> value, int alignment = 0, string? format = null);
            public bool AppendFormatted(string? value);
            public bool AppendFormatted(string? value, int alignment = 0, string? format = null);
            public bool AppendFormatted(object? value, int alignment = 0, string? format = null);
        }
    }

    public partial sealed class String
    {
        public static string Create(IFormatProvider? provider, Span<char> scratchBuffer, [InterpolatedBuilderArgument("provider", "scatchBuffer")] ref InterpolatedStringBuilder builder);
        public static string Create(IFormatProvider? provider, [InterpolatedBuilderArgument("provider")] ref InterpolatedStringBuilder builder);
    }
}

namespace System.Text
{
    public sealed class StringBuilder
    {
        public StringBuilder Append([InterpolatedBuilderArgument("this")] ref InterpolatedAppendFormatBuilder builder);
        public StringBuilder Append(IFormatProvider? provider, [InterpolatedBuilderArgument("this", "provider")] ref InterpolatedAppendFormatBuilder builder);
        public StringBuilder AppendLine([InterpolatedBuilderArgument("this")] ref InterpolatedAppendFormatBuilder builder);
        public StringBuilder AppendLine(IFormatProvider? provider, [InterpolatedBuilderArgument("this", "provider")] ref InterpolatedAppendFormatBuilder builder);

        [EditorBrowsable(Never)]
        public struct InterpolatedAppendFormatBuilder
        {
            public static InterpolatedAppendFormatBuilder Create(int literalLength, int formattedCount, StringBuilder stringBuilder);
            public static InterpolatedAppendFormatBuilder Create(int literalLength, int formattedCount, StringBuilder stringBuilder, IFormatProvider? provider);

            // Same members as on InterpolatedStringBuilder
            public bool AppendLiteral(string s);
            public bool AppendFormatted<T>(T value);
            public bool AppendFormatted<T>(T value, string? format);
            public bool AppendFormatted<T>(T value, int alignment);
            public bool AppendFormatted<T>(T value, int alignment, string? format);
            public bool AppendFormatted(ReadOnlySpan<char> value);
            public bool AppendFormatted(ReadOnlySpan<char> value, int alignment = 0, string? format = null);
            public bool AppendFormatted(string? value);
            public bool AppendFormatted(string? value, int alignment = 0, string? format = null);
            public bool AppendFormatted(object? value, int alignment = 0, string? format = null);
        }

        …
    }
}