LDM Notes for June 10th, 2024

[333fred]

https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-06-10.md

Agenda

• ref structs implementing interfaces and in generics

This agenda comes with an explicit request for you; yes, you reading this discussion! Do you have a use case for ref structs implementing interfaces? If so, we want to hear about it, so we can ensure that we're building the right feature. Please let us hear your use cases!

[colejohnson66]

Off the bat: ref structs implementing interfaces removes the need for duck typing, like foreach and await use. Future language features could use interfaces to define the structure.

[cap5lut]

for me came the IDisposable duck typing to mind, tho that alone would not be worth it.

[neon-sunset]

For that to work effectively, C# would probably need an associated type support, in order to allow the inference of the enumerator type, which itself would have to satisfy the allows ref struct, err, anti-constraint. This could enable Span and other types subject to ref lifetimes to be supported by (yet another) struct LINQ implementation, though this time around being much closer to what is offered by Rust.

[colejohnson66]

Not particularly. Again, Span<T> cannot implement IEnumerable, and we cannot base the codegen of foreach purely on IEnumerable, as IEnumerator is too costly. Without that, the entire idea of throwing away structural typing here doesn't hold water.

I’m not suggesting requiring it for existing features, as that would break backwards compatibility. I’m only talking about future ones.

[333fred]

I’m only talking about future ones.

And we couldn't do that for foreach if we were redesigning it from the ground up today with IEnumerable<T>. While I'm also for IEnumerable2 that uses associated types to get better enumerator generics to flow through the system, we don't have that, and we won't have that for C# 13. What we're looking for here is concrete use cases that we can validate the design of ref structs implementing interfaces against. Not hypotheticals that wouldn't work without still other language features.

[Richiban]

I think the C# compiler is (arguably) embracing duck-typing more and more, rather than less. Patterns essentially compile down into method / property / indexer calls, and there might be multiple of them in a single pattern.

[agocke]

FYI if you're willing to operate on Enumerators directly, you can still get a lot of the benefit in .NET 9:

using System;
using System.Collections;
using System.Collections.Generic;

List<int> list = [ 1, 2, 3];
Span<int> span = [ 1, 2, 3];

C.M(list.GetEnumerator());
C.M(span.GetEnumerator2());

public class C {
    public static void M<TEnum>(TEnum enumerator)
        where TEnum : IEnumerator<int>, allows ref struct
    {
        while (enumerator.MoveNext())
        {
            Console.WriteLine(enumerator.Current);
        }
    }
}

// Stand-in code until we implement IEnumerator for span enumerator
static class SpanExt
{
    public static SpanEnum<T> GetEnumerator2<T>(this Span<T> span)
        => new SpanEnum<T>(span.GetEnumerator());
}

ref struct SpanEnum<T> : IEnumerator<T>
{    
    private readonly Span<T>.Enumerator _enum;
    public SpanEnum(Span<T>.Enumerator e)
    {
        _enum = e;
    }
    
    
    public T Current => _enum.Current;

    object IEnumerator.Current => _enum.Current!;

    public void Dispose()
    {
    }

    public bool MoveNext()
    {
        return _enum.MoveNext();
    }

    public void Reset()
    {
        throw new NotImplementedException();
    }
}

[yaakov-h]

Looking over it I was expecting to maybe be able to write M<TEnumerable, TEnumerator> but that won't work either without a new interface IEnumerable<TElement, TEnumerator> where TEnumerator : TEnumerator<TElement> or similar.

[yaakov-h]

One use case I've come across but never really needed that much is a way to pass a Span around that I can also be finished with early. I expect with C# 13 I could do something roughly like:

interface ITemporarySpannable<T> : IDisposable
{
    public Span<T> Span { get; }
}

explicit extension SpannableMemory<T> for Memory<T> : ITemporarySpannable<T>
{
    // Memory<T> already has a Span property, not sure if we would need to redeclare it here?
    public void Dispose() { } // no-op as this has no cleanup
}

explicit extension SpannableSelf<T> for Span<T> : ITemporarySpannable<T>
{
    public Span<T> Span => return this;
    public void Dispose() { } // no-op as this has no cleanup
}

ref struct ArrayLease<T> : ITemporarySpannable<T>
{
    public ArrayLease(ArrayPool<T> pool, int size)
    {
        this.pool = pool;
        this.array = pool.Rent(size);
    }

    T[] array;
    ArrayPool<T> pool;
    
    public Span<T> Span => array.AsSpan();

    public void Dispose() => pool.Return(array); // return to pool when we are finished, even if still in scope.

    // missing IsDisposed checks etc.
}

This might be a dumb idea but I do keep finding myself wishing that ArrayPool was a little bit easier to work with than var/try/finally every time.

[333fred]

Well, you were never going to be able to implement interfaces with extensions in C# 13; that was, at best, going to be a preview, and almost certainly won't make the cut at all. But the last example is somewhat interesting.

[jjonescz]

With ref struct interfaces, the lock statement could be extended to allow other types than just the new Lock - e.g., SpinLock. See the first bullet point in https://github.com/dotnet/csharplang/blob/3ab828dcfe2ae5eafcc8537e71ac543eb2927340/proposals/lock-object.md#alternatives.

[Ollhax]

Here's my a use case for ref structs with interfaces: I'm working on an entity component system where I have my data in one struct ComponentData and a facade struct Component:

public struct ComponentData
{
	public float UpdateTime;
}

public readonly struct Component(FarmSystem system, ComponentIndex index) : IComponentView<Component>
{
	public static Component Create(FarmSystem system, ComponentIndex index) => new(system, index);

	public readonly ref ComponentData Data => ref System.GetData(Index);
	public readonly FarmSystem System = system;
	public readonly ComponentIndex Index = index;

	public readonly float UpdateTime { get => Data.UpdateTime; set => System.SetUpdateTime(value); }
}

What I'd like to do is to make Component a ref struct and store the ComponentData reference in the struct, so I can avoid the cost of accessing it each time use the Data property. I still need the IComponentView<Component> interface to provide the static Create factory function to get the machinery to work.

[hez2010]

One of the marquee original intentions for ref structs implementing interfaces was for Span and ReadOnlySpan to implement IEnumerable

We need an new interface to carry the enumerator type instead of boxing it into IEnumerator<T>:

interface IEnumerable<T, TEnum> where TEnum : IEnumerator<T>, allows ref struct
{
    TEnum GetEnumerator();
}

ref struct Span<T> : IEnumerable<T, SpanEnumerator<T>>
{
    SpanEnumerator<T> GetEnumerator() => new (this);
}

ref struct SpanEnumerator<T>(Span<T> span) : IEnumerator<T>
{
    public T Current { get; }
    public bool MoveNext();
}

But this can be hard to use without associated types, ideally it should be:

interface IEnumerable<T>
{
    type TEnum : IEnumerator<T>, allows ref struct;
    TEnum GetEnumerator();
}

ref struct Span<T> : IEnumerable<T>
{
    type TEnum = SpanEnumerator<T>;
    TEnum GetEnumerator() => new (this);
}

ref struct SpanEnumerator<T>(Span<T> span) : IEnumerator<T>
{
    public T Current { get; }
    public bool MoveNext();
}

[TahirAhmadov]

Unfortunately, I found an issue.
SharpLab

I'm not sure if it can be changed for ref struct, but I doubt it. Here is the problem: under current rules, C# copies the struct-typed field before invoking a method on it, when that field's type is a generic parameter. This means the MoveNext's state changes are lost, precluding the idea of a no-allocation enumeration.
Unless I'm missing something ...

[agocke]

@TahirAhmadov You're missing something.

You marked the enumerator as readonly:

readonly TEnumerator _source;

That's causing the copy when you call _source.MoveNext(). You need to leave the field as mutable.

[TahirAhmadov]

Yes of course! Made it readonly by habit 🤣

[jnm2]

readonly on fields being an attractive nuisance once again :)

[TahirAhmadov]

Really though, another item in the "when we re-write everything from scratch" bucket:
Disallow emitting "copy" in IL. If a struct method is non-readonly, prohibit calling on return values, readonly fields, etc.

[TonyValenti]

For me, the ref struct implementing interfaces would be super nice for span/enumerables. That is my main use case.

Maybe this was considered, but, perhaps instead of making spans implement interfaces, there should be a method to have the compiler generate a wrapper type that implements the interface.

[KennethHoff]

Compiler generate a wrapper type that implements the interface.

That's basically the plan for "Extensions implementing interfaces", so one could potentially utilize the same compilation strategy for that - whatever that turns out being - similar to how params collections utilized collection expressions.

[TahirAhmadov]

The wrapper type would need to be a ref struct in order to hold a Span, so in order for it to implement interfaces, we still need ref struct to be able to implement interfaces and at that point, why not just add it to Span? Or I misunderstood the point you were trying to make?

[TonyValenti]

I have never actually rolled my own ref structs and have really only used Span/ROSpan.

Here is an interesting idea:
Is this something that should be handled by C# or the runtime?

If I understood one of the challenges, it had to do with interface dispatch.

Would a method like this allow the runtime to specialize that away?

Public static void Foo<TCollection, TEelement>(TCollection source) where TCollection : IEnumerable<TElement>
Where TElement allows ref struct {

}

Pardon typos. Typing on my phone.

[yaakov-h]

@TonyValenti it wouldn't generally help because you would still have the enumerator from IEnumerable<TElement>.GetEnumerator() as IEnumerator<TElement>. You could have your collection but you would never be able to perform useful operations over it.

[ufcpp]

I am wondering if it could be used to implement Lists with ref struct whose content is almost the same, only the Resize policy is different.
https://github.com/ufcpp/UfcppSample/tree/master/Demo/2024/ValueList/Collections

[hez2010]

Worth to note that the runtime has added APIs to allow using ref structs for comparers, so that one can implement IComparer<T> or IEquatable<T> for a ref struct. This is definitely a use case for ref struct implementing interfaces.

[neon-sunset]

I'm an author of https://github.com/U8String/U8String project (the one that uses cursed source generator api). Due to ever increasing amount of APIs, concerns regarding internal structure in combination with future extensibility and lack of time, the project has been progressing slowly.

However, upon evaluating ref structs implementing interfaces and passed as generics, it appears that this exact feature will allow to unify and generalize significant amount of internal logic, bringing the approach closer to how Rust generalizes its own string implementations.

Currently, U8String has a lot of logic for slicing, splitting, enumerating and otherwise manipulating UTF-8 strings. This logic, however, is hard tied to U8String itself. This sometimes clashes with C#'s de-facto UTF-8 string type being ReadOnlySpan<byte>. The implication of this is users are expected to either start with U8String as a data type from the beginning, which the library tries to make as convenient as possible, or to reallocate ROS<byte>s into new U8Strings instead.

With this change, I have a prototype that accepts TU8String (or even just TString) that can serve both U8String and U8Span implementing the common set of interfaces, allowing cleaner integration with ROS<byte> and lighting it up for scenarios like splitting without duplicating certain types and methods. This could be expanded in the future to e.g. NativeString and others.

[MitchRazga]

Bit of a niche use case but ref structs implementing interfaces would allow me to reduce the number source generator marker attributes that I use to workaround the absence of ref record structs.

...
[WithInterfaceProperties]
public readonly ref partial struct Inputs : IAreaThresholds, IDeviationThresholds, IMeanThresholds;

vs

...
[WithInterfaceProperties<IAreaThresholds>]
[WithInterfaceProperties<IDeviationThresholds>]
[WithInterfaceProperties<IMeanThresholds>]
public readonly ref partial struct Inputs;

[ds5678]

Thank you all so much! I refactored my HTML generation library to make heavy use of ref struct interfaces.

In my case, all the HTML element types are ref structs, in order to encourage proper API use and cultivate a pit of success.

The introduction of ref struct interfaces allows me to write code more generically. One example is the CustomAttributes class which provides support for emitting attributes that have not been pre-defined. Previously, these methods were generated into every HTML element type.

[TahirAhmadov]

In case it wasn't already brought up: imagine you have a lazy-instantiated collection, and a "proxy collection" to wrap around it:

class Foo
{
  public IList<string> Strings => this._list ??= [];
  public Wrapper Numbers => new Wrapper(ref this._list);

  List<string>? _list;
}
public readonly ref struct Wrapper : IEnumerable<int> // currently, this is CS8343
{
  public Wrapper(ref List<string> list)
  {
    this._list = list;
  }
  public readonly void Add(int i)
  {
    (this._list ??= []).Add(i.ToString());
  }

  readonly ref List<string>? _list;

  public IEnumerator<int> GetEnumerator() => throw new NotSupportedException();
  IEnumerator IEnumerable.GetEnumerator() => throw new NotSupportedException();
}

var foo = new Foo
{
  Strings = { "abc" },
  Numbers = { 123 } // currently, this is CS1918
};
Console.WriteLine(string.Join(", ", foo.Strings));
// output: abc, 123

To accomplish something similar, I have to do some weird stuff with delegates and such.

[TahirAhmadov]

I just noticed another issue - CS1918 is generated because it's a struct, I think it shouldn't be an error for collection initalization, or probably an even more encompassing rule - if member is readonly, allow.
I ran some memory profiling and working through excessive allocations, it would be cool to have this feature.
@333fred does this sound reasonable? Should I open a discussion for it?

[333fred]

Honestly, this example is very confusing. Why is that a ref struct? Where's the implementation of IEnumerable? What's the error you're getting? What are you suggesting changing?

[TahirAhmadov]

Please my edits above. It's a ref struct so that I can store the underlying List<string> as a ref field. Then the lazy instantiation can happen whether I add to Strings itself or to Numbers in an object initializer. IEnumerable serves solely as a marker to enable collection syntax in object initializer; it's not meant to be used by itself, therefore it just throws. It's generating CS8343, which is what this discussion is about, but also CS1918. I think it should be allowed to invoke stuff on a struct in object initializer within some boundaries, specifically, if the members are readonly.

[333fred]

@TahirAhmadov for implementing IEnumerable, it sounds like you're not using a current compiler in preview mode. As to the other suggestion, you could certainly open a discussion.

[TahirAhmadov]

#8427