Proposal: ref interface

[YairHalberstadt]

Problem

Currently there is no mechanism by which it is possible to abstract over ref structs.

For example, currently it's possible to write a version of linq which work on spans:

public ref struct SelectSpanEnumerable<T, TTarget>
{
     private ReadOnlySpan<T> source;
     ...
     public SelectSpanEnumerator<T, Target> GetEnumerator() => ...;
}

public SelectSpanEnumerable<T, TTarget> Select<T, TTarget>(this ReadOnlySpan<T> span, Func<T, TTarget> func) => ...

You can do something similar for Where, and all the other linq methods.

However what happens when you need to call span.Select(...).Where(...)?

Well you haven't defined any Where method which works on a SelectSpanEnumerable. So you need to define one, and another custom enumerable for that method. But then you need to define a Where method which works on the new custom enumerable, and another one which works on the custom enumerable for that, and another ... ad infinitum.

Code generation wont help here since you can't generate infinite methods and classes.

What about interfaces?

Well ref structs can't implement interfaces since they can't be boxed.

There are existing proposals to allow ref structs to implement interfaces, but not allow them to be cast to the interface, instead to be used as Type Parameters to a method constrained to that interface.

There are two problems with that. Firstly the following is legal, and would box the struct:

public object M<T>(T t) where T : MyInterface => t;

This can be solved by introducing the ref struct constraint (#1148).

Secondly, if the interface defines a DIM which the ref struct doesn't implement, the struct will be boxed to call the method. You could ban a ref struct from implementing an interface with a DIM, but that would mean adding a DIM would be a breaking change, which defeats the entire purpose of DIMs.

I've spent the last few hours trying to think of any hack I could use to implement SpanLinq, but although I had a couple of initially promising ideas, at the end I've come up empty handed.

Proposal

Allow declaring a ref interface.

A ref interface cannot define DIMs (alternative: can only define a DIM which is guaranteed inlineable and does not box).
A ref interface can only be used as Type Constraint. If something akin to Rusts Impl Trait is ever implemented, it could be used there too.
A ref interface Type Constraint implicitly adds a ref struct constraint.
A ref struct can implement a ref interface.
A normal class, struct, or interface can implement/inherit a ref interface.

This would allow us to write SpanLinq as follows:

public ref interface RefEnumerable<T, TEnumerator> where TEnumerator : RefEnumerator<T>
{
     public TEnumerator GetEnumerator();
}

public ref interface RefEnumerator<T>
{
    public bool MoveNext();
    public T Current { get; }
}

public ref struct SelectSpanEnumerable<T, TTarget, TEnumarable, TEnumerator> : RefEnumerable<TTarget, SelectSpanEnumerator<T, TTarget, TEnumarable, TEnumerator>> where TEnumerable : RefEnumerable<T, TEnumerator>, where TEnumerator : RefEnumerator<t>
{
     private TEnumerable source;
     ...
     public SelectSpanEnumerator<T, TTarget, TEnumarable, TEnumerator> GetEnumerator() => ...;
}

public SelectSpanEnumerable<T, TTarget, TEnumarable, TEnumerator> Select<T, TTarget, TEnumarable, TEnumerator>(this TEnumerable span, Func<T, TTarget> func) where TEnumerable : RefEnumerable<T, TEnumerator>, where TEnumerator : RefEnumerator<t> => ...

Completely horrible, but workable. Note that this is very similiar to the code I wrote for the alloc free linq proposal. Although that was coming from a different angle, the ideas there would definitely help this.

Is this worth it?

By itself? Almost definitely not.

However the problem here is real. I think ref structs are really powerful, and C# is only exploring the very tip of the iceberg of the things they make possible. Mostly that's because it lacks the tools to safely abstract over them. Take a look at Rust for an idea of how powerful a combination ownership/lifetime semantics, generics, and traits are.

Now is the solution to add a mini Rust like language into C#?

On the one hand I think it's the only solution, in the sense that there is no other way to make abstraction over ref structs possible.

However, C# is not Rust, and never will be. ref structs allow you to write extremely fast code, but at the cost of abstraction, and that fits perfectly into C#s philosophy. If you really need it you can write fast code. Otherwise you can write nice code.

So I'm leaving this here as a dumping ground for some ideas, and it's possible that other events will transpire to make this more likely to occur. Who knows 🤷.

[orthoxerox]

Why does this remind me of shapes?

[YairHalberstadt]

@orthoxerox

There's a lot of overlap.

Theres a couple of extras that are needed to make this work with ref structs.

Also the latest incarnation of shapes looks like it might be built on top of interfaces.

[john-h-k]

seems easier to have where T : ref struct no? then where T : ref struct, IEnumerable<int>. No need to remake every single interface ever

[YairHalberstadt]

@john-h-k

I was thinking that would lead to issues with DIMs, but you could just not allow calling a DIM on something constrained to a ref struct

[YairHalberstadt]

Actually no. That would make adding a default implementation to an existing method a source breaking change.

[mphelt]

@YairHalberstadt
Is link for 'alloc free linq proposal' a valid one?

[YairHalberstadt]

@mphelt Thanks, fixed

[mikernet]

Yesssss, I was just about to propose this if I didn't find this issue. This has a lot more uses than just LINQ, particularly with Span/ReadOnlySpan. This would make our "binary processor" implementation so much nicer, for example. Think of a binary processor kind of like either a stream reader or writer depending on how it is initialized. Right now I have to mangle together the reader and writer into one implementation, i.e. something like this:

public ref struct BufferProcessor
{
    private ReadOnlySpan<byte> _readBuffer;

    private Span<byte> _writeBuffer;

    public bool IsReader => _readBuffer.Length > 0;

    public bool IsWriter => _writeBuffer.Length > 0;

    public int Position { get; set; }

    public void Process(ref int value)
    {
        if (IsReader)
            value = ReadInt(_readBuffer, Position);
        else if (IsWriter)
            WriteInt(value, _writeBuffer, Position);

        Position += 4;
    }

    public void Process(ref long value) /// ... etc
}

This would instead allow me to separate the reading and writing into different ref struct implementations which means cleaner code and more performant code as the structs become smaller in size and IsReader/IsWriter checks are eliminated for every single field being processed:

public ref interface IBufferProcessor
{
    public int Position { get; set; }

    void Process(ref int value);
}

public ref struct BufferReader : IBufferProcessor
{
    private ReadOnlySpan<byte> _readBuffer;

    public int Position { get; set; }

    public void Process(ref int value)
    {
        value = ReadInt(_readBuffer, Position);
        Position += 4;
    }
}

public ref struct BufferWriter : IBufferProcessor
{
    // Separate implementation for writer
}

public class Person
{
    private string _name;
    private int _age;

    private void Process<T>(ref T processor) where T : ref struct, IBufferProcessor
    {
        processor.Process(ref _name);
        processor.Process(ref _age);
    }
}

It gets even more complicated currently as there is also a big or little-endian mode on the buffer processor as well so that adds another layer of mangling and branching when instead the BufferReader functionality could just be split into a BigEndianBufferReader and LittleEndianBufferReader implementation.

So, in other words, I think this would be fantastic. Please make this happen 🙏

Then just add a way to pass an auto-implemented property by ref (by actually passing its backing field by ref) and everything will be perfect 😃

[mikernet]

Also, I believe ref interfaces could potentially also have DIM support for feature parity with regular interfaces, no? The this parameter of the default implementation would just have to effectively be a generic where T : ref struct, IWhatever parameter. Or am I off-base here?

[YairHalberstadt]

@mikernet

They could, but that would add a huge amount of complexity to the runtime for an extremely niche case.

[mikernet]

Yeah, probably :)

[YairHalberstadt]

The original issue was spurred by trying to write Linq for Span<T>.

I've now done that using SourceGenerators! Check it out at https://github.com/YairHalberstadt/SpanLinq.

Warning - it's still in preview for now.