[Proposal] New ref convention for returning references

[nmsmith]

Hi everyone,

@lattner and I have developed an alternative "auto-deref" proposal to the one that Chris posted a few weeks ago. The new idea is to make auto-dereferencing a result convention, rather than a feature of the Reference type.

The proposal can be found here.

We'd love to hear feedback from the Mojo community. You can share your thoughts below.

[mzaks]

Great Proposal! I have two things which popped up in my mind.

1. I think calling the Reference type just a Pointer doesn’t do it justice, or rather might cause false associations with users coming from other languages. The whole SafePointer vs UnsafePointer is IMHO very judgmental 😄. As it doesn’t translate the why, it implicitly tells you don’t use the second one always use the first one. In my head TrackedPointer makes more sense.

2. Regarding return type, how about infixing the fact that the return is not by value but rather a location e.g. fn foo() -> Int@<origin>:

[nmsmith]

Thanks for the feedback! I'll respond to each of your points separately:

1. I guess I'm not the best person to comment on the naming scheme for pointers in Mojo. Mojo will end up with several "pointer-like" types and obviously, they can't all be called "pointer". I expect that Reference (whatever its name ends up being) will be the "canonical" pointer type. It will be the thing that the average Mojo user reaches for when they want to point to a variable located somewhere else. In that light, it makes sense that the name is short and simple. UnsafePointer is something that only low-level "hackers" need to know about, so it deserves a more sophisticated name. 🙂
2. The issue with Int@<origin> is that it looks like a fancy type, and the new conventions are not types. If we were to use that syntax, I'd be worried that a lot of learners would attempt to declare types like List[Int@<origin>] and wonder why that doesn't work. Ideally then, the syntax we end up with would start with the origin/lifetime and a name for the argument/result (if applicable), and end with the type. And bonus points if the type is delimited by :.

[dmitry-salin]

Rename Reference to Pointer

When ref is explicitly promoted to Pointer with a call to its constructor, to people coming from other languages it may look like conversion from safe reference to unsafe pointer. I think ref could be renamed to ptr for consistency. Thus, Pointer and ptr can be documented as safe type and safe convention respectively.

Syntax for the convention

I think the square bracket syntax is more consistent because ref convention is generic over lifetime and address space. This way, parameterization syntax is the same for types and convention, and no additional keywords are required.

fn foo(...) -> ref[lifetime, addr_space] T: ...

vs

fn foo(...) -> ref from lifetime, addr_space: T: ...

What about lifetime elision rules?

Looks like this is a syntactic sugar for ref[_] self with elided lifetime:

fn __add__(borrow self, rhs: MyInt) -> MyInt: ...
fn __iadd__(mut self, rhs: MyInt): ...

This also looks like example of lifetime elision:

for (ref elt) in mutableList:
    elt = 42

These rules could potentially apply in many situations. In this example, the returned ref has __lifetime_of(self):

fn __getitem__(ref self, index: Int) -> ref Self.ElementType:

The LSP may have a configuration option to always display elided lifetimes as inlay hints. This way they can still be explicit to the user.

[nmsmith]

ptr is definitely an option worth considering.

Regarding lifetime elision, I expect Mojo will do less of that than Rust. IMO it makes sense for arguments (the lifetime of each argument is an implicit parameter by default, as you say), but not so much for return types, because the lifetime associated with a returned reference affects how long it is usable for. Obscuring this information might do more harm than good.

In general, I personally believe the best way to make lifetimes accessible to Mojo users is to make them simpler: a more intuitive syntax, an easier to explain model, and less restrictive borrowing rules. We should be able to achieve all of this.

[dmitry-salin]

Since Mojo is another approach to lifetimes, it doesn't necessarily have to copy the Rust model. But some common things are inevitable and should be learned by users. The lifetime elision concept is just one option that could be used more widely at any time later as it does not interfere with this proposal. __lifetime_of(self) is such a common pattern that it will be everywhere, so time will tell whether it is better for users to write it or just read it as inlay hints.

[yinonburgansky]

I greatly appreciate the enhanced ergonomics provided by the new ref return type for users interacting with the Reference type.

questions

I do have some questions and concerns I'd like to ask:

1. How can we prevent the "infection" of user code when calling functions that return:

• other reference types like Arc[T] / Mutex[T].
• nested reference types like List[Reference[T]], List[Arc[T]].
• SomeStruct with a field of type Reference[T] / Arc[T].

2. The decision whether to auto-deref or not, is shifted from the user to the library author.
   Since the concept of restoring the Reference type on the user side, applies only to the builtin Reference type, but not to other reference types like Arc / Mutex / RWLock due to information lost. Therefore, the auto-deref feature remains unique to the builtin ref type, managed by the compiler and inaccessible from the user side. unlike other builtin types which are accessible from the user side.
   Library authors aiming to enhance user experience for those unfamiliar with References must provide 2 function signatures: one for returning types like T / List[T], and another for returning special Reference types like Arc[T] / Mutex[T] / List[Arc[T]] / List[Mutex[T]].

3. Limiting borrowed / inout to the builtin ref type, rather than allowing it to work with other reference types, such as sending a RWLock to inout argument, will acquire and release the lock upon read / write. This enables end-user functions to work with any reference type transparently without ever mentioning Reference, working seamlessly in single and multi threaded processing. However, this proposed convention may impose a higher learning curve for getting good performance in multi-threaded applications.

4. Using __getitem__ for setting/mutating the reference, deviates from Python convention of __setitem__. While understandable due to other mutations like +=, the name is a bit misleading, and it lacks "hooks" for acquiring and releasing resources for reference types like Mutex / RWLock.

5. IMHO working with references remains verbose as constant dereferencing is required (search for [] in the stdlib). Implementing proper auto-deref support, as proposed earlier would significantly reduce verbosity, since reading from a reference is more common than writing to it. This would make the common case more concise and easier to read, especially with multi-threaded reference variants prevalent in multi-threaded processing.

---

Some thoughts

I believe the previous proposal allowing auto-deref for user-defined types holds more potential.
From what I understand from the discussions, a key missing component is a mechanism to prevent auto-deref when manual handling of the actual reference is necessary.

I am hopeful that a solution to this issue exists.

Disclaimer:
As primarily a Python user, I am still in the process of learning about mojo, lifetimes, and the borrow-checker. Therefore, I may be overlooking or misunderstanding certain aspects of how they function; I apologize for any such oversight. During my exploration of references and examination of the stdlib code, I couldn't help but notice the widespread use of [], which often distracts from the essence of the code. Personally, I advocate for its sparing usage, reserving it for situations where it is absolutely necessary. The following are some ideas I've come up with (disregarding keyword renaming and syntax discussions):

preventing auto-dereferencing

Introducing "no-auto-deref variables" with a special syntax can effectively disable auto-dereferencing when working with these variables:

var x = 1
var &a = Reference(Reference(x))  # No auto-deref since `a` is declared with `&` (consider postfix `&` or other syntax).
var b = a  # Reference[Reference[Int]], since `a` is declared with `&` it doesn't auto-deref.
var c = b  # auto-deref to Int, equivalent to a[][]
var d: Reference[Int] = b  # Control decay by providing an explicit type annotation, equivalent to a[]

var e = List(b, d)  # List[Int] since `b` and `d` auto-deref.
var f = List[Reference[Int]](b, d)  # List[Reference[Int]] control auto-deref with type annotation.
var g = List(a)  # List[Reference[Reference[Int]]] since `a` doesn't auto-deref.

system programmers who wish to work with raw references without auto-deref, can prefix variable declarations with &, similar to C/C++.

var lst: List[Reference[Reference[Int]]]

for x in lst: ...  # `x` is Int
for &x in lst: ...  # `x` is Reference[Reference[Reference[Int]]]

var [a, *b] = lst  # `a` is Int, `b` is Iterable[Int]
var [&a, *&b] = lst  # `a` is Reference[Reference[Int]], `b` is Iterable[Reference[Reference[Int]]]

the rules:

1. When the LHS variable is declared with the no-auto-deref symbol & - the RHS expression won't auto-deref.
2. Any variable declared with no-auto-deref symbol & will not auto-deref, regardless of where it is used and require manual dereferencing with [].
3. Explicit type annotation can be utilized to control auto-deref.

complexity

If simplicity remains a concern, and we aim for even simpler rules, we can eliminate rule 3 entirely, as it's not essential for complete functionality. This effectively reduces the rules to:

There are 2 variable types:

• "Regular" variable declaration which will always auto-deref all the way down.
• "Reference" variables which will never auto-deref and require manual dereferencing with [].

This essentially shifts the decision of auto-dereferencing from the callee-signature to the caller-"signature" defined by the prefix & variable declaration.
The primary distinction lies in not discriminating between builtin-ref and user-reference, maintaining consistency with existing builtin types thus far.

---

integration with inout and borrowed

Enabling inout and borrowed to work with other reference types, such as Mutex and RWLock can be achieved through a mechanism like so:

trait Referable[T, lifetime]:
    fn __enter__(self) -> Referable[T, lifetime]: ... # called when borrowing the reference

    fn __exit__(self):
        pass  # By default, no action when the reference is returned after borrowing.


struct RWLock[T, lifetime](Referable[T, lifetime]):
    fn __enter__(self) -> Referable[T, lifetime]:
        @parameter
        if self.lifetime.is_mutable:
            # handle write... acquire lock
            return self.unsafe_ptr
        else:
            # handle read...

    fn __exit__(self):
        # release lock

Any type that confirms to the Referable trait can be declared via borrowed or inout.

borrowed x: T = some Referable[T, AnyLifetime]
inout y: T = some Referable[T, MutableLifetime]

When reading or writing from a borrowed or inout variable, a context manager will be opened to access the reference.

var x = 1
var a: Reference[Reference[Int]] = x  # Implicit conversion and allowed to auto-deref

borrowed b = a  # `b` acts like Int, auto-deref as far as possible
inout c = a  # `c` acts like Int, auto-deref as far as possible

c = 2  # Mutate Reference[Int], by calling the context manager methods __enter__ and __exit__ before and after the mutation.
print(x, b, c, a[][])  # prints 2 2 2 2

Using with no-auto-deref syntax:

var x = 1
var y = 2
var &a = Reference(Reference(x))
var &b = Reference(Reference(y))

inout &c = a  # Acts like Reference[Int]
inout &d = d
c, d = d, c # Swap the references, mutate Reference[Reference[Int]], calls __enter__ and __exit__
print(x, y, a[][], b[][], c[], d[]) # prints 1 2 2 1 2 1 

This allows users, unaware of reference types, to work with nested reference types:

var lst: List[Reference[Int]]

for inout x in lst:
    x += 1  # Mutate Reference[Int], *not* the list item; user doesn't need to know if it's an Int or Reference[Int].

for inout &y in lst:
    x = ... some Reference[Int] # Mutate Reference[Reference[Int]] the list item.

Additionally, functions with inout/borrowed args will work seamlessly with any Referable Type.

# User function unaware of all reference types and deref syntax
def do_something(inout x, borrowed y): ...


# Send it any Referable Type
var x: RWLock[Int]
var y: Arc[Int]
do_something(x, y) # User function only knows about Int, and automatically calls `__enter__`/`__exit__` without knowing exact types, acquiring and releasing the lock.

In the above code, the lock is acquired automatically without the user needing to be aware of it. In an ideal scenario without deadlock concerns, the function can simply auto-wait for the lock, allowing for a gradual introduction and adaptation to multi threaded code. This facilitates sharing the same user function for single-threaded with Reference type or multi-threaded with Arc/Mutex/RWLock types.

When a user requires more performance or is more advanced, they can always modify the function signature to specifically request the lock, such as for batching mutations with a single lock acquire-release. However, it's beneficial that users can learn about it gradually.

When sending non-Referable types to functions accepting borrowed/inout arguments, they will auto-promote to the built-in Reference type by default, similar to current behavior:

def do_something(inout x, borrowed y): ...

var x = 8
var y = 9
do_something(x, y)  # auto-promote builtin Reference, equivalent to:
do_something(Reference(x), Reference(y))

Regarding expressions returning Referable types being on the LHS, there may be ambiguity with __setattr__. One potential solution is to allow it only with the inout keyword, as follows:

var x = 8
inout Reference(x) = 9  
# allowed only with the `inout` keyword as a shorthand for:
inout temp = Reference(x)
temp = 9


struct SomeStruct:
    var a: Reference[Int]

var y: SomeStruct = ...
inout y.a = 8  # Mutate Reference[Int] since used with keyword inout
y.a = Reference... # Mutate the actual field a with __setattr__

var &z = Reference(x)
inout (z) = 10  # Specify an expression explicitly to prevent confusion with declaration and allow for potential shadowing in the future.  
inout z = 10  # Error, `z` is already declared.

[nmsmith]

Thanks for your feedback. If I'm understanding you correctly, you're mostly concerned about how the ref convention would generalize to "concurrent references", which manifests as Arc<Mutex<T>> in Rust.

I think the right thing to do here would be to first figure out what "concurrent references" would look like in Mojo, and more generally, what idiomatic concurrent programming looks like in Mojo. We might not end up with the same model as Rust. In particular, mutexes are a problematic concurrency primitive, owing to the risk of deadlock, so it would be good if the average Mojo user could work with something higher-level. (Deadlock is an "aliasing problem", similar to the problem solved by lifetimes.)

Once we understand all of this, we're more likely to know how it interacts with Mojo's argument conventions.

All that said, I'm merely an external contributor, so I don't have much insight into Mojo's future concurrency model. Perhaps somebody from Modular can provide a more informed response.

[Brian-M-J]

Mojo can dodge all these issues by adopting a concurrency model that integrates well with value semantics (so no need for references and lifetimes) and doesn't require users to work with locks directly (so no need to worry about Arc, Mutex etc.). An example of this is senders and receivers.

[rd4com]

Good job! about names, just an idea: SafeReference for Reference and UnsafeReference for UnsafePointer.

That way, maybe less confusion as to why Reference can be made with an UnsafePointer and so on ?

But this is just an idea, excellent job ! 🔥

[melodyogonna]

+1 on renaming Reference to "SafePointer" rather than just "Pointer". Due to C and C++, pointers, by themselves, generally have negative connotations. it'll be great to convey at a glance (to both low-level and high-level programmers) that this pointer is different and a lot safer.

[midrange22]

This is a great proposal!

I want to voice my support for the proposed keyword alternatives for inout, borrowed, and owned.
The proposed new keywords init, borrow, mut, and consume makes things much clearer when reading Mojo code, in my opinion.
The example shown in the proposal has sold me on the idea:

## Proposed
struct MyInt:
   fn __init__(init self, value: Int): ...

   fn __add__(borrow self, rhs: MyInt) -> MyInt: ...

   fn __iadd__(mut self, rhs: MyInt): ...

   fn __del__(consume self): ...

[taylorpool]

By virtue of the fact that Mojo already positions itself as a safe language, I believe that prepending Safe onto types such as Reference or Pointer would be redundant.

[melodyogonna]

I believe that adding "safe" to the name will instantly clarify things for many people. Otherwise, they might be confused by the abundance of unsafe operations at first glance.

As I mentioned earlier, when people think of pointers, the first thing that comes to mind is C/C++ pointers, which are unsafe. Even raw pointers in Rust are considered unsafe.

[martinvuyk]

I think we should have less names.

I'm not familiar with concrete implementations of languages so I might be completely off.

But to me it looks like we only really need 3 concepts:

• read only reference
• mutable reference
• reference whose lifetime will end (be consumed)

Why not have 3 types that describe exactly that and have auto-deref?
And have Reference be the generic type that has to be manually dereferenced.

struct MyInt:
   fn __init__(self: MutRef[Self], value: Int): ...

   fn __add__(self: Ref[Self], rhs: Self) -> Self: ...

   fn __iadd__(self: MutRef[Self], rhs: Self): ...

   fn map(self: ConsumeRef[Self]) -> Self: ...

   fn __del__(self: ConsumeRef[Self]): ...

And simpler to reason about lifetime notation (Lifetime should be able to be constructed from the Ref subtypes)

fn some_func(arg1: Ref[Int], arg2: MutRef[Int]) -> Ref[String]:
    if arg1 > arg2:
        return Reference[String, False, Lifetime(arg1)]("bigger")
    return Reference[String, False, Lifetime(arg1)]("smaller")


fn some_func2(arg1: Ref[Int], arg2: MutRef[Int]) -> Mutref[String]:
    if arg1 > arg2:
        return Reference[String, True, Lifetime(arg1)]("bigger")
    return Reference[String, True, Lifetime(arg1)]("smaller")


fn some_func3[
    a: Lifetime, b: Lifetime
](arg1: Reference[Int, False, a], arg2: Reference[Int, True, b]) -> Ref[String]:
    if arg1[] > arg2[]:
        return Reference[String, False, a]("bigger")
    return Reference[String, False, a]("smaller")

[remorses]

But ConsumeRef is not a ref, it's a value. And Ref is the default for arguments but not for result types, that's what's confusing imo.

[nmsmith]

@martinvuyk In Mojo, mutability is associated with lifetimes. A reference can be used to mutate its pointee if and only if its lifetime is mutable. This is why it makes sense to have just one Reference struct (plus the ref convention for auto-deref), rather than a different struct for each mutability.

Also, consume is used to declare that a function deinitializes a variable. It isn't directly related to references.

Regarding auto-deref, the problems with having structs auto-deref themselves was discussed in the thread for the former proposal, and it was also mentioned in the new proposal.

[soraros]

Don't have time to go into details, here are some potential thinking points:

• Not being able to get auto-derefed values from Tuple could be a bigger problem than people think. It's a special (and maybe the simplest) case of destructing (think ref pattern in Rust). It will be appear everywhere, once we have more pattern matching built out. ref as of now can be think of as customising the semantics of the binding form: var Ctor(name, ...) = f(...).
• We could/should write ref[...] instead of ref [...], "parametric convention marker" if you will. Let's don't start with cute syntax using from etc.
• Let's hold off the keyword renaming etc to later please. Not until we have more serious use of refs at least.

[lattner]

I agree on delaying the keyword renaming fyi, it's more to provide a vision than to suggest we do it all at once.

[dderooy]

Thanks for the great write up! A quick thought: alter instead of mut.
All together then: init, borrow, alter, consume.

[dderooy]

modify or mod would be a good key name too and goes well with modular 😉

[lattner]

Status update: the first three steps of this proposal are all implemented, and should roll out in the next nightly. I'm very happy with how they fit together.

[Brian-M-J]

I'm just hoping Mojo leans harder towards value semantics so I don't have to deal with all this, or at least won't have to deal with it in the overwhelming majority of situations.

[ivellapillil]

struct MyInt:
   fn __init__(init self, value: Int): ...

Could it mean that I can implement any method as a "constructor"/factory method and __init__ just becomes a convention?

E.g.

struct MyInt:
   @staticmethod
   fn new(init self, value: Int) -> ref Self: ...

# Usage

MyInt.new(1)

Or will we restrict the init calling convention to __init__?

P.S. I understand that __init__ is treated specially, but once we have the calling conventions, it need not be? init becomes the indicator for a constructor like operation (as it indicates that we pass an uninitialized thing and by end of the function we expect it to be initialized and optionally(?) returned.

[forFudan]

Thanks For the proposal.

I support init, read, mut. Not very sure about consume. reasons are as follows.

• mut would be more friendly for a Rust user.
• I prefer read because it literally indicates absolute immutable. Instead, borrow can still mean that you can make some changes to it, like a mutable ref.
• About consume, I am not sure whether it is the best name. The value is only consumed when a ^ sigil is attached. Otherwise, it is copied. So the default action is copy instead of consume. own seems better because it leaves users to decide whether it is copied or consumed (with the ^ sigil).

[martinvuyk]

I think duplicate i.e. dup as you said earlier aligns with what you're saying. I agree it is not clear yet in an "intuitive" way what owned does. How about clone ? It would either be a copy or the "real" one when used with the ^ sigil.

[ivellapillil]

Duplicate and clone are basically synonyms of copy... My main point is that copy does not happen always and therefore would be misleading.

[forFudan]

I think duplicate i.e. dup as you said earlier aligns with what you're saying. I agree it is not clear yet in an "intuitive" way what owned does. How about clone ? It would either be a copy or the "real" one when used with the ^ sigil.

I agree with @ivellapillil that dup or copy are not consistent with what has been defined in the function body, although they may be consistent with what users see.

Maybe a mid-way is good (using a verb that are not that direct), or completely remove owned keyword? The argument is passed as owned by default. When you want to pass a immutable ref, use borrowed.

[ivellapillil]

Maybe a mid-way is good (using a verb that are not that direct), or completely remove owned keyword? The argument is passed as owned by default. When you want to pass a immutable ref, use borrowed.

I think one of the insights in Mojo has been that most of the code is borrowed by default and we can be more ergonomic if we make it the default. It also allows some nice effects, that by default the compiler does not have to make potentially expensive copies. To prevent unnecessary copies, the developers have to be careful to mark their arguments with borrowed, which add to their cognitive load. Quite often developers do the "minimum" to get the code working, so if borrowed is not impacting the working of the function, they will leave it out. The nice thing about borrowed as default is that it forces them to use owned or inout if they want to mutate the argument. This means that they are marking an argument because it has direct impact to their function. Since borrowed is the more efficient compared to the others, making it the default actually makes the code faster by default - even if the developer did not think of it.

[forFudan]

Yes, it makes sense to make immutable reference default option.

But in that way, we still have to think about assigning a nicer, and less confusing keyword for owned. :D

[lattner]

This is all basically working now and the discussion converged, thank you for driving this @nmsmith !!