[proposal] Abstracting over async (the yields effect)

[nmsmith]

The proposal document can be found here.

I am proposing that we replace async with a yields T syntax. This allows us to abstract over whether a function is async, and more generally, it allows us to abstract over whether traits and even entire libraries are async. This solves the function/trait/library duplication problem that arises in languages that have async/await.

This can be seen as a counter-proposal to @owenhilyard's recent proposal on effect handlers. We are both attempting to solve the same problem, and my proposal is heavily inspired by Owen's. The main difference between our proposals is that I have taken the raises Never trick that @lattner presented here, and adapted it to async. The end result is really exciting, in my opinion.

I am keen to hear everybody's thoughts. 🙂

[martinvuyk]

I have proposed that we replace async with a yields T syntax, that allows us to abstract over whether a function/trait/library contains suspension points. A function that is not a coroutine (i.e. does not suspend) is equivalent to a function declared as yields Never.

@nmsmith I love this. Making it a function effect and being able to parametrize it would be awesome.

trait Iterator[T: AnyType]:
   alias yields_: Bool
   fn __next__(mut self) raises StopIteration, yields if yields_ -> T: ...

I'm personally not that fond of raising iterators or this conditional yield logic. I also don't think inserting comptime logic inside type signatures is a great idea.

I think I'd rather see something along the lines of:

trait Iterator[T: AnyType, generator: Awaitable]:
  fn __next__(mut self) yields generator -> T: ...
  # is the same as
  fn __next__(mut self) -> generator[T]: ...
  fn __has_next__(self) -> Bool: ...

We could try to make coroutine have a "false async mode". And then we try to add some kind of compiler decorator which sets the path to materialization

@nonmaterializable
struct Coroutine[T: AnyType, sync: Bool, auto_deref: Bool = True]:
    fn __init__(out self: Coroutine[_, sync=False], handle: AnyCoroutine): ...
    fn __init__(out self: Coroutine[_, sync=True], handle: AnyFunction): ...

  @materialization_entrypoint(auto_deref=auto_deref)
  fn __await__(owned self) -> T: ...

What this would allow is an "auto-deref" but for the await keyword

fn read_file[sync: Bool]() -> _FileGenerator[sync]:  ...
# or the alternative syntax
fn read_file[sync: Bool]() yields _FileGenerator[sync] -> List[Byte]:  ...

fn foo(path: String):
    var data: List[Byte]
    data = read_file[sync=False](path)
    data = read_file[sync=True](path)

IMO we should try to achieve something in this direction (even if the details of what I'm proposing here don't add up), where we don't end up having arbitrarily complex code executing inside type signatures.

As for @owenhilyard 's #3946 and also trying the same solution for raises. I don't know if my mini-proposal of using nonmaterializable and having a NoError error type as a workaround is possible. I'll leave it as food for thought.

[nmsmith]

Thanks for the feedback @martinvuyk.

I'm personally not that fond of raising iterators

Acknowledged. In my proposal, I just replicated how iterators work in Python. It was not my intention to propose that Mojo iterators raise exceptions. This is a discussion worth having, but it's outside the scope of this proposal.

we should try to achieve something [...], where we don't end up having arbitrarily complex code executing inside type signatures.

Mojo's metaprogramming model has always offered the ability to execute arbitrarily complex code at compile-time. This is a feature, not a bug. Also, I've only made use of a simple ternary-if operation, so I'm not sure what you're concerned about here.

What this would allow is an "auto-deref" but for the await keyword.

I'm not a fan of this idea. It sounds very complicated/magical.

[owenhilyard]

I think that this causes some awkward syntax, since we want the ability to have a coroutine who's component functions return Poll[T], where Poll is a sum type. There isn't a great way to specify "this will yield one sum type variant when you should keep polling, and another when it is finished". It leads to something like fn foo() yields Poll[T] -> Poll[T]. While I'm fine with that as a desugaring for async fn foo() -> T, I think that people will get very annoyed with that very quickly if they have to write it by hand. I think that, for functions which act as generators, it's a reasonable syntax since you do need the ability to specify return and yield types separately. For infinite generators, such as your prime numbers example, I would do fn prime_numbers() yields Int64 -> Never, so the compiler is informed that it is expected to never return. Creating this may necessitate something like Rust's loop keyword, signifying an infinite loop, with the yields inside of it.

I like the idea of using ternaries to control the values yielded, returned and raised, but I think I want to try to take it further if the compiler team is willing. Ideally, this shouldn't be special cased and there should be some way to get a token from the compiler which represents a type (which can later be used in reflection, but for now may as well be opaque), and make any expression which returns that mlir type legal in those positions, so you can have a function which figures out what to do instead of having massive nested ternaries.

I don't know if there is a "sane default" for yields except for None or Never, meaning it doesn't happen.

I strongly prefer "awaiting something without await does nothing", since I think it's easier to explain and makes the code more unified. The option of making awaits implicit introduces a lot of headaches for things like timeouts, where you want a future that will be done when time is up and another to do actual IO.

As far as function chaining, I agree that prefix await is a bit messy, and would prefer postfix await.

In IDEs, we can highlight the identifiers of coroutines differently

But github, discord, discourse and other places with regex-based highlighting cannot do that.

In languages with green threads, such as Go and Java, suspension points are not explicit. I haven't seen Go or Java users complain about this. To the contrary, they normally talk about the lack of function coloring (i.e. explicit suspension) as the primary advantage of green threads.

The reason that Go and Java can do this is because they have a runtime which can carry a lot of weight for them. Go uses what I call "the paint bucket approach to function coloring", and makes everything async. The compiler is technically free to suspend at any function call, even if you are holding locks that should be released quickly. There are substantial costs which come during performance analysis, and they result in users who want to write fast code having to do deep audits of the codebase and learn arcane rules for when the compiler suspends functions. You will also note that go doesn't have a way to get values out of a greenthread except via a channel. This causes a lot of headaches when you want to send out multiple requests and then gather results from all of them.

But we still need the latter type (let's call it Task) for modelling the execution state of a coroutine.

This is the coroutine frame.

Alternatively, tasks can be "detached" from their parent task by submitting them directly to an executor, e.g. executor.submit(task^). This would return a Future that can be awaited.

I disagree here, I would have normal coroutines used in async return an (excuse my Rust syntax) impl Future<Output = T>, which is effectively an alias for impl Coroutine<Yield = Poll<T>, Return=Poll<T>>. When you spawn a task, that is detaching it from you, such that it survives you being cancelled. Unless the return value is stored inline in the coroutine frame, I'm not sure we could support futures being returned from that, instead I would rather have a JoinHandle type, which explictly has that extra allocation to hold things in.

[martinvuyk]

we should try to achieve something [...], where we don't end up having arbitrarily complex code executing inside type signatures.

Mojo's metaprogramming model has always offered the ability to execute arbitrarily complex code at compile-time. This is a feature, not a bug. Also, I've only made use of a simple ternary-if operation, so I'm not sure what you're concerned about here.

In a way yes, but it will make function signatures get even more bloated (any function can be placed inside the ternary if else condition). Also AFAIK (I'm no compiler person) it's one thing to add code to check the default values for a function argument or parameter, and another for the type itself to be computed (AFAIK that is the purpose of Traits). If we were to allow inserting dynamic code in places where there are normally traits (or function effects), I think it's easier to read if we let a type have a function that returns the end result according to the parameters.

e.g. adapting from #3756

@value
struct Generator_1or2[parameter1: Bool, parameter2: Int]:
    @staticmethod
    fn __type__() -> AnyType:
        return Generator1 if Self.parameter1 else Generator2

    @staticmethod
    fn __trait__() -> AnyTrait:
        return SomeIntParamTrait[Self.parameter2 if Self.parameter1 else  0]

And each function gets added by code-gen (for types that have them) when on one such part of a type signature (parameter, type, or function effect space).

A crazy idea: extend this to include __yields__, __raises__, __captures__, and we have ourselves some quite readable advanced meta-programming.

What this would allow is an "auto-deref" but for the await keyword.

I'm not a fan of this idea. It sounds very complicated/magical.

We already use materialization for types like IntLiteral (and implicit constructors) when being passed to a function that expects an Int, this is just a reversed "implicit destructor" where the type gets consumed and controls to what it materializes to. The parameter "auto_deref" could also be called "nonmaterializable_at_comptime" or basically setting up an implicit barrier that anytime the type is instanciated it is consumed right after exiting from the function where it was created. It would be a "ghost return type".

The main goal of that workaround is to not have to make conditionals on when each effect gets applied and simply say that there is a "NoneEffect", so that the functions actually always return the same types parametrized for different cases.

[nmsmith]

Thanks for the feedback Owen! I'll reply to all of your main points.

I think that this causes some awkward syntax, since we want the ability to have a coroutine who's component functions return Poll[T], where Poll is a sum type. There isn't a great way to specify "this will yield one sum type variant when you should keep polling, and another when it is finished". It leads to something like fn foo() yields Poll[T] -> Poll[T].

I don't see why Poll needs to be explicit in the function signature. (For other readers: Poll is a Rust enum.) Let's consider the signature fn foo() yields A -> B. If you want the executor to receive a sum type of either A or B, we can surely make this happen without the sum type needing to show up in the signature. Isn't this phenomenon already implied by the fact that a coroutine may either yield or return, depending on the control flow? To me, it makes more sense for the executor to receive a Variant[A, B].

You seem to have made the assumption that the type appearing after yields must match the type that shows up in the signature of Rust's Future.poll method. I don't think we need to impose that constraint.

Ideally, [the ternary operator] shouldn't be special cased and there should be some way to get a token from the compiler which represents a type, and make any expression which returns that mlir type legal [after yields and raises].

Yes, it makes sense for Mojo to eventually have this kind of capability. That's outside the scope of this proposal though. 🙂

making awaits implicit introduces a lot of headaches for things like timeouts, where you want a future that will be done when time is up and another to do actual IO.

I don't understand the issue here. Setting a timeout for an I/O operation could be as concise as:
var data: String = await read_file.Task(path).timeout(1000)

...which would raise an exception if the operation times out.

Alternatively, if you want to do that trick where you "race" two tasks—one of which is a timeout—that might look like:
var data: Variant[String, None] = await race(read_file.Task(path), sleep.Task(1000))

The only difference from the conventional async/await model is that if you don't want to execute the coroutine immediately, you need to add .Task to construct an explicit task.

But github, discord, discourse and other places with regex-based highlighting cannot do that.

Yes, that's true. That's a legitimate downside of relegating suspension to a function effect. But given that the raises effect is also implicit in these places—and raises changes control flow—I'm not sure that this is going to be a big deal. In situations where a programmer needs to study a program's behavior carefully, they need a proper IDE regardless. Trying to make reading code on Github a pleasant experience is already a lost cause.

I would have normal coroutines used in async return an (excuse my Rust syntax) impl Future<Output = T>. [...] When you spawn a task, that is detaching it from you, such that it survives you being cancelled. Unless the return value is stored inline in the coroutine frame, I'm not sure we could support futures being returned from that, instead I would rather have a JoinHandle type, which explicitly has that extra allocation to hold things in.

All of this boils down to a disagreement on what a "Future" is. You're using the Rust definition, whereas I was using the C++ definition. In C++ a future is a pointer. If you use this mental model, then I think that section of my proposal makes more sense.

Also, I don't think a Task should be automatically detached from the task that spawned it. That prevents us from modelling subtasks, i.e. tasks whose lifetime is constrained by a parent task—and which have access to the parent task's state. It's very important that we have something like this. It's what "structured concurrency" is based on.

[melodyogonna]

I've gone back and forth with @owenhilyard about the merit of this functionality on Discord, I've been convinced it is massively useful - if we're successful in making the Mojo community adopt async genericity from the get-go.

I, however, have issues with the syntax and semantics proposed here.

1. I disagree with both removing the requirement to await coroutines and with having the compiler await them.
2. The syntax, while being useful for something the compiler can lower to, is not something people should write every time.
3. Environment, runtime-specific, compatibility concerns are brushed over.

I have a suggestion/proposal/improvement based on this that I'd like to submit for evaluation.

What if we introduce the concept of "partial async" or something similar, which describes a function signature that can be satisfied in either an async or non-async form?

Details

I suggest we add the ability to denote the asynchronicity of a function is optional - async?
Which we can use as following:

trait SomeTrait:
    async? fn method_name() -> Int: ...

This trait can be satisfied by async and non-async functions. Functions can also be defined to be partially async:

async? fn partially_async() -> Int: ...

The idea here is that functions which are explicitly declared to be partially async can be assumed to be safe to call in both sync and non-sync manner. However, functions that are explicitly fully async - async fn fully_async(): ... are designed to be used only in async environments. Likewise, functions without any effect are not async in any manner. The functions that are fully async or not async at all can be lowered into the syntax proposed here or the one Owen proposed.

Coroutines and Tasks

When a function is partially async, the compiler can decide whether it is a coroutine or not based on how it is called.

Tasks depend on the execution environment, there needs to be a guarantee that they're being created with a coroutine. I propose we cast a partially async function into fully async based on a model similar to type narrowing. Given a partially async function, we can verify it is async:

async? def do_something(partial_async_function):
  @paremeter
  if is_async(partial_async_function):
      x = partial_async_function() # creates a coroutine
      task = loop.create_task(x)
      value = await task
  else:
      value = partial_async_function()

see - https://mypy.readthedocs.io/en/stable/type_narrowing.html#user-defined-type-guards

Summary

1. 'await' can be used in async and partially async functions.
2. 'await' can be used for async and partially async functions.
3. Actions that require explicit coroutines can not happen with partially async functions unless it has been guaranteed to the compiler that it returns a coroutine.
4. 'await' can not be used in non-async functions.

I'm hoping we can have the sort of uniform generality we have with refs now.

[owenhilyard]

Tasks depend on the execution environment, there needs to be a guarantee that they're being created with a coroutine. I propose we cast a partially async function into fully async based on a model similar to type narrowing. Given a partially async function, we can verify it is async:

@paremeter
if is_async(partial_async_function):
    x = partial_async_function() # creates a coroutine
    task = loop.create_task(x)
    value = await task
else:
    value = partial_async_function()

I have a version of this in my effect handlers proposal, but you put it inside of the function.

async? fn partial_async_function(...):
    # common setup

    if async:
        ...
    else:
        ...

[melodyogonna]

Tasks depend on the execution environment, there needs to be a guarantee that they're being created with a coroutine. I propose we cast a partially async function into fully async based on a model similar to type narrowing. Given a partially async function, we can verify it is async:

@paremeter
if is_async(partial_async_function):
    x = partial_async_function() # creates a coroutine
    task = loop.create_task(x)
    value = await task
else:
    value = partial_async_function()

I have a version of this in my effect handlers proposal, but you put it inside of the function.

async? fn partial_async_function(...):
    # common setup

    if async:
        ...
    else:
        ...

Forgot to mention that the condition is inside a function, edited for clarity.

[nmsmith]

Hi @melodyogonna, thanks for your suggestion. I'll share my feedback below.

In the proposal document, I walk through a ton of different use cases for the yields T feature. From what I can tell, your async? feature doesn't fulfill all of those use cases. If you think it does, I would find it helpful to see the use cases addressed systematically. In the meantime, I am skeptical that your counter-proposal would be sufficient.

My second concern is that Mojo is going to have raises T at some point, and I've demonstrated the utility of raises if <cond>. If Mojo were to already offer that feature, then the cost of adding yields if <cond> would be very small (in terms of compiler + conceptual complexity). Certainly, the cost would be smaller than an async? feature that has a novel syntax and semantics. And raises T is undoubtedly more appropriate than raises?.

[nmsmith]

I do find at least one aspect of your design intriguing, @melodyogonna. Your is_async(f) syntax inspired me to investigate what would happen if we make the types that a function yields and raises associated types of the function's type. With such a feature, we can dramatically simplify the following signature from my proposal:

fn higher_order[T1: AnyType, T2: AnyType](f: fn () yields T1 raises T2) yields T1 raises T2:

Here's the idea:

• When using fn to declare a function type, the function is assumed to be non-yielding and non-raising. (As in today's Mojo.)
• When using any_fn to declare a function type, the type of value that a function yields and raises becomes an associated type of the function type. (Please ignore the keyword's name. If we agree that want this feature, we can bikeshed it later.)

This allows us to shorten the signature of higher_order significantly:
fn higher_order(f: any_fn ()) yields f.YieldType, raises f.ExceptionType:

As explained in the proposal, if f.YieldType is Never, then the function isn't a coroutine, and if f.ExceptionType is Never, then the function doesn't raise.

This design also happens to resolve a syntactic issue with function types. For the sake of readability, ideally effect declarations would be comma-separated, e.g. fn foo() yields X, raises Y, some_other_effect. But we can't include those commas in function types, because that would mess up the parsing of argument lists. By turning effects into associated types of a function type, accessible through the syntax f.ExceptionType, the parsing issue is neatly avoided.

[martinvuyk]

This allows us to shorten the signature of higher_order significantly:
fn higher_order(f: any_fn ()) yields f.YieldType, raises f.ExceptionType:

+1 on this. Also (a bit outside the main topic), if we extend it for docstrings overloading it would be great

fn timeit(f: any_fn ()) yields f.YieldType, raises f.ExceptionType:
  __doc__.summary = "Time any function."
  __doc__.args = "f: Any function."
  __doc__.raises = f.__doc__.raises
  __doc__.yields = f.__doc__.yields

[ivellapillil]

FWIW, Nim has effect system which seems to work pretty well. They use braces as syntax which we can also adopt if needed, and AFICS, is extensible to custom effects as and when needed.

https://nim-lang.org/docs/manual.html#effect-system

[melodyogonna]

Hi @melodyogonna, thanks for your suggestion. I'll share my feedback below.

In the proposal document, I walk through a ton of different use cases for the yields T feature. From what I can tell, your async? feature doesn't fulfill all of those use cases. If you think it does, I would find it helpful to see the use cases addressed systematically. In the meantime, I am skeptical that your counter-proposal would be sufficient.

While reasoning about my proposal yesterday I noticed a soundness problem. Specifically - calling a partially async function in sync context. If we assume that an API that accepts a partial async function dynamically (e.g trait) can be passed both a synchronous and an asynchronous function, and that coroutines must remain awaitable, then if the partial async function is called it'll just return a coroutine that does nothing. After thinking through different scenarios, it seems your idea to remove the necessity of making coroutines awaitable is the simplest solution. However, the functionality of lazy coroutines is very helpful, it is something we want. So perhaps we can have that option but in fully async environment. To that end, perhaps instead of having only eager coroutines or only lazy coroutines, we can support both?

[martinvuyk]

However, the functionality of lazy coroutines is very helpful, it is something we want. So perhaps we can have that option but in fully async environment. To that end, perhaps instead of having only eager coroutines or only lazy coroutines, we can support both?

Quoting myself:

We could try to make coroutine have a "false async mode". And then we try to add some kind of compiler decorator which sets the path to materialization

@nonmaterializable
struct Coroutine[T: AnyType, sync: Bool, auto_deref: Bool = True]:
    fn __init__(out self: Coroutine[_, sync=False], handle: AnyCoroutine): ...
    fn __init__(out self: Coroutine[_, sync=True], handle: AnyFunction): ...

  @materialization_entrypoint(auto_deref=auto_deref)
  fn __await__(owned self) -> T: ...

if the auto_deref parameter is there, it would mean we could control when coroutines are auto-awaited and when not.

This doesn't need to look exactly like what I proposed or even use materialization, but it would be necessary to sometimes switch no manual await

[melodyogonna]

@nmsmith
Another thing. I'm trying to make my model work with yours, however, it's being done with so many different assumptions that it's probably better as separate proposal.

Here are my thinking models.

1. Let's not mix generators and coroutines, let's make coroutines its own type. Also, coroutines can not "yield" a different type compared to what the function returns.
   if we try to translate this to your proposal.

async? def function() -> Int: ...

would look something like:

def function() yields -> Int: ...

which could be further be assumed to mean:

def function() -> Int | Coroutine[Int]: ...

2. Let's maintain the rule in Python that prevents generators from being used in a coroutine environment. And extend it so generators can't be used in a partially async function.
3. Whether the environment is async is determined by the function declaration rather than some compile-time parameter. so

fn read_file(path: String) yields if config['async'] -> String:

becomes

async? read_file(path: String) -> String:

when a function is fully async:

async read_file(path: String) -> String:

is desugared to something akin to... I don't really know how to represent it with your model. But if we go with Owen's model

async=async fn read[async: Bool = True](): ...

if partial async function is called, the environment it is executing in should be propagated by the function that called it.
4. We can then tie this down with your eager coroutine idea. In a partial async function, coroutines are eager and always desugar to coroutine.__await__ even if called without await. in a fully non-async environment, await can not be used, so coroutines always desugar to common coroutine.__await__ like in your model.
However, in a fully async environment, coroutine can be lazy.

Comparing this to current reference semantics.

async? def = ref[_] - the environment will be inferred
def = ref[Origin[False]] - the environment is fully sync. This will be propagated through all the call stack
async def = ref[Origin[True] - the environment is fully async. This will be propagated all the way through

5. I'm basically trying to maintain the current semantics of fully async and fully non-async execution in Python. So you can't call a fully async function in a fully sync environment. Instead, I introduce a bridge, which combines your ideas and Owen's in some way. If Mojo community is bootstrapped using this, we can have functions that can be called in both sync and async environments.

[melodyogonna]

Quoting myself:

We could try to make coroutine have a "false async mode". And then we try to add some kind of compiler decorator which sets the path to materialization

@nonmaterializable
struct Coroutine[T: AnyType, sync: Bool, auto_deref: Bool = True]:
    fn __init__(out self: Coroutine[_, sync=False], handle: AnyCoroutine): ...
    fn __init__(out self: Coroutine[_, sync=True], handle: AnyFunction): ...

  @materialization_entrypoint(auto_deref=auto_deref)
  fn __await__(owned self) -> T: ...

if the auto_deref parameter is there, it would mean we could control when coroutines are auto-awaited and when not.

This doesn't need to look exactly like what I proposed or even use materialization, but it would be necessary to sometimes switch no manual await

This is great.

[nmsmith]

@ivellapillil, I just looked into Nim's effect-tracking system, and I've noticed that the creator of Nim has described the system as a mistake and as having a bad cost-benefit tradeoff. So that's an immediate red flag.

That said, the curly-brace syntax that Nim uses in its function signatures has inspired me to consider whether Mojo effects should be bundled into a single data structure, rather than listed separately. (Note: Nim doesn't do this. Nim's {.foo.} braces are not a data structure.) By bundling effects together, it would be easier to write functions and traits that are generic over arbitrary effects.

Abstracting over arbitrary effects (not just raises and yields)

As a thought experiment, let's consider what would happen if we used Python's set literal syntax to bundle effects together:
fn foo() does {Exception[DivideByZero], Yield}:

The does keyword is just a placeholder, so let's avoid bikeshedding it for the moment. Also, let's gloss over what exactly Exception and Yield are. Maybe they are types; maybe they are some other kind of compile-time value.

If effects are bundled into a single value, and we have the any_fn keyword (proposed in my last post), the syntax for declaring that higher-order function is generic over all effects would reduce to:
fn higher_order(f: any_fn ()) does f.effects:

The compiler would need to have special logic for reasoning about the f.effects data structure. If it's possible for f.effects to include Exception or Yield, then the compiler must treat calls to f as altering control flow, etc.

Come to think of it, any_fn plays a very similar role to the async? fn syntax that Owen and Melody have been talking about. (And that Rust's keyword-generics initiative has proposed.) Here's an idea: let's consider having an effectful keyword (the name is a placeholder), that generalizes over all possible effects a function might have, including async. With this keyword, the signature of higher_order would be written:
fn higher_order(f: effectful fn ()) does f.effects:

effectful could also be used in traits, to declare that a method may have arbitrary effects:

trait Iterator[T: AnyType]:
    effectful fn __next__(mut self) does Exception[StopIteration] -> T: ...

Compare this to the definition of Iterator that I presented in my proposal document:

trait Iterator[T: AnyType]:
    alias yields_: Bool
    fn __next__(mut self) raises StopIteration, yields if yields_ -> T: ...

The definition based on effectful is both more concise, and more general, because it allows implementations of __next__ to have arbitrary effects. Here's what a function that takes an Iterator would look like:

fn get_integer[IterType: Iterator[Int]](mut iterator: IterType) does IterType.__next__.effects -> Int:
    try:
        return next(iterator)
    except StopIteration:
        return 0

The appearance of __next__ in the signature is a bit clunky, so maybe Mojo should have a built-in function named effects() that can return the effects associated with either a single function, or the union of all the effects of the methods available on a type:

fn get_integer[IterType: Iterator[Int]](mut iterator: IterType) does effects(IterType) -> Int:
    try:
        return next(iterator)
    except StopIteration:
        return 0

There's actually a small problem here: effects(IterType) would seemingly include the StopIteration effect, but we don't want get_integer to inherit that effect. We'd need to consider how to clarify that.

Advantages of this design (vs. my published proposal)

This design would put us in a good position to add additional kinds of effects to Mojo in the future. It would also allow us to support library-defined effects, which I know is something that @owenhilyard was interested in.

I can already think of another major class of effect that we haven't discussed: reading and/or mutating non-local state. Closures often do this:

fn main():
    counter = 0
    fn foo(x: Int) mut counter -> String:
        counter += x
        return bar(x)
        
    foo(5)               # Invoke 'foo'
    higher_order(foo)    # Let somebody else invoke 'foo'

The fact that foo mutates non-local state is very important. For one thing, it means foo must not be executed on multiple threads concurrently. Despite this, we still want functions like higher_order to be able to take a closure as an argument, and inherit its effects. If foo can mutate counter, then so can higher_order(foo).

This suggests that maybe the effectful keyword should prevent a function from being shared between threads. Maybe it can somehow be connected to theSend and Sync traits that Rust offers, and which have been contemplated for Mojo.

In summary, maybe raises, yields, and closures can be folded into one language feature: effects. Every effectful function has "associated effects", accessible via the syntax f.effects. This gives us the ability to define higher-order functions that inherit the effects of the functions they call. Notably, every generic function is implicitly a higher-order function: for each trait bound on its type parameters, it receives a set of function implementations.

Disadvantages of this design (vs. my published proposal)

As a consequence of this design being more general, it is also less explicit. In particular, by integrating whether a function raises into the effectful and does <effect> syntaxes, we might be making it harder to reason about the control flow of Mojo programs. Whenever a programmer invokes an effectful function, they would need to program defensively, on the assumption that the function may raise.

It's not clear whether this would be a problem in practice. Certainly a Python programmer wouldn't be bothered by this, because in Python, all functions can raise. The same is true of most other programming languages. Maybe as long as we make it clear that effectful implies raises, that will be sufficient.

Next steps

I think we should develop this design further, so that we can compare the two alternatives:

1. Supporting only two effects: raises and yields.
2. Supporting a whole menagerie of effects, and offering a way to abstract over all effects at once.

This concludes my brain dump. 🙂

[ivellapillil]

If we introduce custom effects, we would need custom effect handlers.

1. Where would those custom effect handlers live?
2. How would we hand over the function to them?
3. What all facilities we need so that the handlers can re-write the functions to handle the effects (e.g. machinery for Coroutine).
4. How would I tell the compiler that an effect is fully handled within a function and therefore should not be propagated (try-except that catches the top most exception).
5. The handler might need to analyse/operate upon the outer scope in addition to just the function (capture analysis in case of closure)

I think generic propagation of effects is easy - but challenges come in when we want to handle them.

Overall, I suspect that while generic effect handling would be super cool, it would be quite complex to pull off all kinds of effects (basically would need AST transformations, communication with the compiler etc.). At the very least it could slow down compilation.

We might be better to treat raises and async/yield and one or two more effects hard-coded in the compiler.

[nmsmith]

@ivellapillil I didn't mean to suggest that Mojo should have algebraic effects with custom handlers, etc. I was thinking a bit smaller: we could let programmers put custom data in the does clause, and this data is (forcibly) inherited by callers, etc. But if users don't have a way to stop propagation, maybe this is not sufficient in practice.

We might be better to treat raises and async/yield and one or two more effects hard-coded in the compiler.

Yeah, this is really the target I was aiming for: abstracting over a bunch of built-in effects. We could leave custom effects as something to investigate in the (distant) future.

[owenhilyard]

@nmsmith

As a thought experiment, let's consider what would happen if we used Python's set literal syntax to bundle effects together: fn foo() does {Exception[DivideByZero], Yield}:

I think you're part of the way to realizing why I proposed the Mojo Parameter store. I wanted key/value pairs with arbitrary Mojo objects, so that Exception could be List[T: Error], Yield could just be an AnyType, etc. So, I would prefer closer to fn foo() does {exception=[DivideByZero], yield=Never}.

If effects are bundled into a single value, and we have the any_fn keyword (proposed in my last post), the syntax for declaring that higher-order function is generic over all effects would reduce to: fn higher_order(f: any_fn ()) does f.effects:

I like this idea, but I also want to be able to do fn higher_order(f: any_fn ()) does {**f.effects, exception=[StopIteration, *f.effects.exception]}

The compiler would need to have special logic for reasoning about the f.effects data structure. If it's possible for f.effects to include Exception or Yield, then the compiler must treat calls to f as altering control flow, etc.

I think the special handling would actually be to keep the data structure strongly typed. does just evaluates a comptime expression to produce the data structure.

Come to think of it, any_fn plays a very similar role to the async? fn syntax that Owen and Melody have been talking about. (And that Rust's keyword-generics initiative has proposed.) Here's an idea: let's consider having an effectful keyword (the name is a placeholder), that generalizes over all possible effects a function might have, including async. With this keyword, the signature of higher_order would be written: fn higher_order(f: effectful fn ()) does f.effects:

effectful could also be used in traits, to declare that a method may have arbitrary effects:

trait Iterator[T: AnyType]:
    effectful fn __next__(mut self) does Exception[StopIteration] -> T: ...

Compare this to the definition of Iterator that I presented in my proposal document:

trait Iterator[T: AnyType]:
    alias yields_: Bool
    fn __next__(mut self) raises StopIteration, yields if yields_ -> T: ...

The definition based on effectful is both more concise, and more general, because it allows implementations of __next__ to have arbitrary effects. Here's what a function that takes an Iterator would look like:

I think I like the "compile-time data structure" approach more. And, remember that for compile time reasons we want to minimize the amount of information that can "travel" in multiple directions. An iterator may need some input, in the form of whether it's async, whether it can block, etc.

fn get_integer[IterType: Iterator[Int]](mut iterator: IterType) does IterType.__next__.effects -> Int:
    try:
        return next(iterator)
    except StopIteration:
        return 0

The appearance of __next__ in the signature is a bit clunky, so maybe Mojo should have a built-in function named effects() that can return the effects associated with either a single function, or the union of all the effects of the methods available on a type:

fn get_integer[IterType: Iterator[Int]](mut iterator: IterType) does effects(IterType) -> Int:
    try:
        return next(iterator)
    except StopIteration:
        return 0

I think making it clear it's a property of the function is a good thing. effects(IterType) doesn't indicate which function the effects are taken from. You can have a single struct with mixed sync and async methods, some of which may raise.

There's actually a small problem here: effects(IterType) would seemingly include the StopIteration effect, but we don't want get_integer to inherit that effect. We'd need to consider how to clarify that.

Advantages of this design (vs. my published proposal)

This design would put us in a good position to add additional kinds of effects to Mojo in the future. It would also allow us to support library-defined effects, which I know is something that @owenhilyard was interested in.

Library defined effects are something I absolutely want, as a way to propagate "how do I handle this situation?" through the stack. We don't want all of that to live in the compiler, so ideally we do library defined effects and have the standard library define "general" effects for common situations.

I can already think of another major class of effect that we haven't discussed: reading and/or mutating non-local state. Closures often do this:

fn main():
    counter = 0
    fn foo(x: Int) mut counter -> String:
        counter += x
        return bar(x)
        
    foo(5)               # Invoke 'foo'
    higher_order(foo)    # Let somebody else invoke 'foo'

The fact that foo mutates non-local state is very important. For one thing, it means foo must not be executed on multiple threads concurrently. Despite this, we still want functions like higher_order to be able to take a closure as an argument, and inherit its effects. If foo can mutate counter, then so can higher_order(foo).

I think turning the origin checker into an effect is something that would need to be discussed a lot more. Effects are opt-in to some degree, and that shouldn't be.

This suggests that maybe the effectful keyword should prevent a function from being shared between threads. Maybe it can somehow be connected to theSend and Sync traits that Rust offers, and which have been contemplated for Mojo.

I think Send and Sync are the right abstractions for this, since if a capture of a closure is !Send or !Sync, the closure inherits that, although a closure with a !Sync capture becomes !Send, since T: Sync is really &T: Send in Rust.

In summary, maybe raises, yields, and closures can be folded into one language feature: effects. Every effectful function has "associated effects", accessible via the syntax f.effects. This gives us the ability to define higher-order functions that inherit the effects of the functions they call. Notably, every generic function is implicitly a higher-order function: for each trait bound on its type parameters, it receives a set of function implementations.

Closures have to have a place to store captured information, which means they need to interact with allocations in a way most functions do not. I'm open to exploring the idea, but that could get messy.

Whenever a programmer invokes an effectful function, they would need to program defensively, on the assumption that the function may raise.

This is why I prefer the more explicit, data-structure-based method. You can get something like effectful with does {**f.effects}, but you don't have to do that if you want to be a bit more explicit about what effects you propagate. If you didn't handle a situation when you do that, the user gets a compiler error and knows to not use your library for that.

It's not clear whether this would be a problem in practice. Certainly a Python programmer wouldn't be bothered by this, because in Python, all functions can raise. The same is true of most other programming languages. Maybe as long as we make it clear that effectful implies raises, that will be sufficient.

I would much rather not force the cognitive load of defensively programming for things which cannot happen. There are also contexts, such as in some unsafe areas, where raising is a very bad idea, for instance if you are inside of signal handler and want to take a callback.

Next steps

I think we should develop this design further, so that we can compare the two alternatives:

1. Supporting only two effects: `raises` and `yields`.

2. Supporting a whole menagerie of effects, and offering a way to abstract over **all** effects at once.

I think we should aim for a general solution, but do a slow rollout. That way, we can try to roll out a full effect system and then if that doesn't work due to compile time or other reasons, we can lock that capability off and make the compiler provide the few effects we are willing to support, instead of allowing library definitions.

[nmsmith]

@owenhilyard

I wanted key/value pairs with arbitrary Mojo objects, so that Exception could be List[T: Error], Yield could just be an AnyType, etc. So, I would prefer closer to fn foo() does {exception=[DivideByZero], yield=Never}.

Hmm.. yes, maybe an effect Dict makes more sense than a Set. Good point. But note: if we're using Python's syntax for dictionary literals, we'd be writing {'exception': [DivideByZero], 'yield': Never}.

I also want to be able to do fn higher_order(f: any_fn ()) does {**f.effects, 'exception': [StopIteration, *f.effects.exception]}

I agree we would need the ability to add effects to an existing dictionary, but I find the above syntax a bit noisy. Maybe we could have an operator for this. Python's dict defines an | operator, but unfortunately it doesn't do deep merging. We could either make EffectDict.__or__ do a deep merge, or we could use a different operator, e.g. +:
fn higher_order(f: any_fn ()) does f.effects + {'exception': [StopIteration]}

I think the special handling would actually be to keep the data structure strongly typed. does just evaluates a comptime expression to produce the data structure.

I think what you mean is that as long as the compiler knows what the value of e is in the expression does e, then it will be able to tell whether e contains a raises or yields effect and therefore the compiler can do all of the special work it needs to do to manage these effects. I agree with that assessment.

My original comment was moreso asserting that the compiler will need to do "special work" to support exceptions and yielding etc, even if Mojo allows the effect dictionary to contain custom effects. Also, it's not clear to me how/whether custom effects would be useful in the absence of custom effect handlers. And even more importantly: I have no idea what benefits custom effects + custom handlers would bring to Mojo, and whether it is worth the complexity. Personally, I can only see a solid use case for the raises and yields effects right now.

I think I like the "compile-time data structure" approach more.

I think you've misunderstood the effectful fn syntax. It was meant to be semantically identical to any_fn. I just split the any_fn keyword into two components (effectful and fn), to make it syntactically similar to the async? fn syntax that has been brought up a few times. By doing this, I've hopefully demonstrated that async? is a special case of a more general/more principled feature.

When effectful fn foo appears in a trait definition, it's just shorthand for requiring the implementation of foo to have an effects field that enumerates its effects. That's all it does. In the absence of effectful (or a does clause), the implementation is required to have no effects.

effects(IterType) doesn't indicate which function the effects are taken from. You can have a single struct with mixed sync and async methods, some of which may raise.

That's true, but my rationale for proposing effects(MyType) is to give a function's author a way to declare that the function may invoke any method on the type, and therefore the function inherits all of those method's effects. This is a lot more concise than listing every method the function invokes in its signature. For many functions, that wouldn't be practical.

Library defined effects are something I absolutely want, as a way to propagate "how do I handle this situation?" through the stack. We don't want all of that to live in the compiler, so ideally we do library defined effects and have the standard library define "general" effects for common situations.

I don't see the use cases for custom effects, so maybe you should put together a separate proposal at some point (built atop the design(s) we've been discussing), that makes a compelling case for them. In the meantime, I'm probably going to operate on the assumption that we only need raises, yields, and (if I can make it fit in), read and mut operations upon variables captured by closures.

I think turning the origin checker into an effect is something that would need to be discussed a lot more.

Yes, whether closure captures can/should be modelled as effects requires a lot more investigation, I agree.

I would much rather not force the cognitive load of defensively programming for things which cannot happen.

I don't understand what your point is here. If you're invoking a function that has unknown effects, you have no choice but to operate on the assumption that the function may raise or yield. That's an inescapable consequence of writing effect-generic code.

I think we should aim for a general solution, but do a slow rollout. That way, we can try to roll out a full effect system and then if that doesn't work due to compile time or other reasons, we can lock that capability off.

Or conversely: we can choose a design that is capable of supporting additional effects, but we only put effort into supporting async and raises at first. This would allow us to get things done quicker. We don't want to be blocked on needing to come up with a full design for custom effects.