Ok wrapping: Improved support for writing code from an error handling mindset

[scottmcm]

Support a ? on function definitions and catch blocks that automatically wraps the result in an Ok, Some, Poll::Ready, etc. In particular, allow function authors to avoid the need for Ok(()) at the end of functions returning Result<(), E>.

Rendered

Posted as part of the error handling ergonomics initiative, rust-lang/rust-roadmap-2017#17
CC originating issue. rust-lang/rust#41414

[sfackler]

Might be worth noting here that C allows you to omit a return from main and it'll insert a return 0 for you for the same reason.

[strega-nil]

I don't believe this is true of C, but of C++. I'd have to check the standard to be sure though.

[strega-nil]

nevermind, it is also true of C.

[eddyb]

Apparently it does require C99, when compiling in a pre-C99 mode you get the "whatever was left around in the platform return register" behavior.

[est31]

While I don't share the view that writing Ok(()) or wrapping something else inside Ok() or Some is bad, I guess people exist who do have this view. As the cost this RFC incurs on the reader is fairly low thanks to the ? approach it choses over the other approaches (most of which suffer from ambiguity issues they introduce), I can gladly give it my 👍 as its not a big cost to the language and there are people who would benefit from this.

[steveklabnik]

My gut level response to this was 👎 but this line

This is analogous to the infallible case, where -> () methods don't need to explicitly return the () value.

Makes me much, much more okay with it.

[withoutboats]

I have pretty strong feelings about the syntax for this RFC, and I know @nrc has pretty strong feelings about the syntax in an opposite direction. Let me try to lay out my preference while I have ambushed everyone with the module RFC 😉

I personally think its very important that this have a function level signifier, and one thing I think is very important about that is that the function not appear to return Result. I think it will be really bad for learnability if the user sees a function thats -> io::Result<usize> which contains a return 0 statement.

Instead, I prefer this syntax:

(-> $ok_ty) throws $err_ty

$ok_ty - the Ok path (if ommitted, = ())
$err_ty - the Err path ty

So for example:

// Applied to Result
fn read(&mut self, buf: &mut [u8]) -> usize throws io::Error

Essentially we get something syntactically very similar to Java, but with these extremely significant differences:

• You need to mark throwing functions as ? to make them throw
• You can instead, always, deal with the "fallible computation" as an object, the Result object.

In sum, we "have our cake and eat it too" - we have the syntactic nicety of exceptions with the semantic nicety of errors as values.

This is also very nice for the "editing case" - you have a function that didn't used to do io, and so it was infallible, and now you introduced some fallible path to it. It needs to start throwing. What do you do?

• Add throws io::Error to the end of the signature
• Add the method call, using ?

No other changes necessary! This RFC supports that as well, but its a bit more clunky in that you have to replace the return type with a Result.

EDIT: I also think the throw sugar is essential to making this work. Under this RFC, the only way to generate an error in a throwing function is this, which IMO is incredibly unintuitive:

Err(Error::new(...))?

---

One thing this would mean is that this syntax does not support non-Result types. I pesonally feel okay with that, even prefer it, but others feel its very important that they be symmetric. This syntax was proposed as an option:

fn read(&mut self, buf: &mut [u8]) -> usize throws io::Error as Result<usize, io::Error>
fn pop(&mut self) -> T throws as Option<T>

In this world, an elided as would be expanded as Result, but an explicit one could take any Try type.

[withoutboats]

Also, syntactically what I've just proposed is extremely similar to what @glaebhoerl proposed in the original ? RFC so I'm sure they have more thoughts about how to do this.

[cramertj]

@withoutboats Nit: fn pop(&mut self) -> T throws as Option<T>-- should this maybe be fn pop(&mut self) -> T throws NoneError as Option<T>?

[stevenblenkinsop]

The odd thing about this syntax is that it makes it look like you're defining the type of function()? to be Result<T, E>, when it's actually T. This doesn't really say anything about the body of the function, either. If ? is just supposed to be a suggestive symbol in this case and not make logical sense, I guess that's fine and looks okay, but I'd prefer if there were some rational connection between the syntax and the effect.

The following syntax is conceptually closer to what's happening while retaining an explicit return type:

fn function() -> Result<T, E> catches { ... }

(either catches or catch works here). This would require defining catch to work like catch? in this proposal as well.

Edit: catches works better with a where clause:

fn function() -> Result<T, E> catches where ... { ... }`

[kennytm]

I think Ok-wrapping is great for ergonomics but I'm 👎 to the proposed syntax, since the RFC proposes to add the ? in the function signature to activate the feature (fn x()? -> Result<T, E> {), but the addition has no effect outside of the function. That is, you are changing the "external interface" of a function which has nothing to do with the external behavior.

Similarly, 👎 to fn x() -> Result<T, E> catches { or fn x() -> T throws E { or anything that activates the feature outside the {.

[sfackler]

@kennytm Function signatures already contain argument patterns which have no effect outside of the function body.

[kennytm]

@sfackler That's sunk cost fallacy 😉 Besides, the argument pattern (except mut) do get printed out in rustdoc, so they do get an exposure in the external interface.

[sfackler]

It's only a sunk cost if you think it's a cost in the first place.

[withoutboats]

In rustdoc at least this syntax could be normalized. I don't agree that its important functions have only externally relevant info, or unimportant that they have info that is only internally relevant.

[est31]

@kennytm do you have alternate proposals except doing this without any syntax (e.g. as coercion, or "whenever there would be an error"), or not doing this at all?

[nrc]

I think I actually ended up pretty close to @withoutboats on the syntax. My initial feeling was that from the caller's perspective, the function returns a Result, so that is what should be in the signature. But they made the argument (iirc) that T throws E gives you exactly the same info as Result<T, E>, just in a different format. That is persuasive to me. One aspect I like about current error handling is that we're not tied to Result and that one can express different error handling techniques in a single type. However, Result is by far the common case. Therefore, I'm happy with @withoutboats' syntax suggestion with the extension that it is viewed as sugar for a more explicit form that can be used with custom error types: ... -> T throws E as MyErr<T, E> where the sugary form (... -> T throws E) is desugared to ... -> T throws E as Result<T, E>. I also agree that throw is an essential part of this whole thing.

[cramertj]

One thought: I don't really like that T throws E hides that the return type is just a plain old Result enum, but thinking of T throws E as an elided version of T throws E as Result<T, E> makes me feel a bit better.

[kennytm]

@est31: Whenever the function is -> Result<(), E> or -> Option<()> or -> impl Try<Ok=()> in general,

1. Replace every return; by return Try::from_ok(());
2. If the last statement ends with a semicolon, or is a block that returns (), insert a Try::from_ok(()) after that statement.

(Basically combining "Handle only the Ok(()) case" and "Omit the explicit marker on the functions/blocks")

No idea about -> Result<T, E> in general, I'd consider "do nothing" as the RFC has demonstrated "coercion"/"omit the marker" is not acceptable.

[stevenblenkinsop]

My main other concern is that, given

fn function() -> T throws E { ... }

if you write function(), it doesn't actually throw, it just returns a Result. Semantically, the function is effectively putting its body in a catch block, which is why I suggested the catches syntax. An alternative which combines syntaxes a bit is to acknowledge that function()? does in fact throw E. So perhaps you could have:

fn function()? -> T throws E { ... }

The issue I see with this is that, while it is logical, it might not be terribly usable, since you always have to remember to add a ? when you use a throws clause. The ? doesn't actually do anything other than make the syntax read correctly.

[est31]

@kennytm that would be a breaking change because right now code like this is possible:

fn foo() -> Result<(),()> { ... }
fn bar() -> Result<(),()> { return foo(); }

Its a pattern that is present, and you either have to disallow it (doing a breaking change), or you'd have to allow ambiguities which would be horrible for readers of code.

[kennytm]

@est31 I don't see how that is breaking, I wrote return;, not return expression;.

[withoutboats]

@est31 An important difference (which makes this backcompat) is that in @kennytm's proposal, you can return both () and Result<(), E> within the same function, whereas in the main proposal, you have to always return the same type in the same function.

[est31]

@kennytm I see. If only return; and <end of block> got wrapping to Ok(()) (basically changing the default from () to Try::from_ok(())), its a neat idea!

[scottmcm]

This would require defining catch to work like catch? in this proposal as well.

Thanks for bringing that up, @stevenblenkinsop. That is, after all, how catch is defined per accepted RFC.

I've added catch semantics as an unresolved question in the RFC, since it's worth discussing explicitly.

Personally, I'm intrigued by the notion that "ok wrapping" could become essentially "function-level catch" (distinct from just wrapping the body in catch{} by the fact that it wraps returns as well).

[stevenblenkinsop]

Personally, I'm intrigued by the notion that "ok wrapping" could become essentially "function-level catch" (distinct from just wrapping the body in catch{} by the fact that it wraps returns as well).

Yeah, I like how it describes what's actually happening. Unfortunately, it's since occurred to me that both fn function() -> Result<T, E> and fn function() -> Result<T, E> catches do in fact "catch" any thrown errors, just in subtly different ways corresponding to your original proposed distinction between catch and catch?. This means that saying that a function catches doesn't adequately distinguish the two forms on its own.

I see two solutions. The first might be too drastic, but I'm including it because I think it's an elegant solution on its own.

1. Deprecate the existing implicit catching at function boundaries, and instead require all uses of ? to appear either in an explicit catch block or in a function annotated with catches. In both cases, the body of a catch and the value returned by the body of a catching function are invariantly ok-wrapped. The advantages of this would be:

   • Catching is explicit everywhere, making the story around catching simpler—"if an error might be thrown (as marked by ?), it must be caught".
   • The type after the -> always matches the type received by the caller.
   • In catching code, all potential sources of errors are explicitly marked (either ? or throws if it's added). Errors can't be hidden in tail position or in return statements.
   • We don't have two subtly different ways of writing code using ?.
      

   The downside is that every current use of ? would become deprecated. This should be an easy change to automate, but it might not be the best use the evolution budget. Still, if the premise is that the existing way to use ? isn't as ergonomic as it should be, and the language should be changed to add a subtly different way to use it that's more ergonomic, then we probably want to remove the existing syntax at some point even if we aren't admitting it yet.

2. Use the throws syntax from above, but replace throws with catches:

   fn function() -> T catches E as Result<T, E> { ... }

   I feel like this better describes what happens when you call function than throws, because the whole point of the ? design is that calling a function doesn't throw; using the ? operator on the result does.

I suppose you could interpret throws to mean the body throws—with the understanding that all thrown errors are caught at function boundaries—but I suspect the intuition people have about throws annotations on functions is that they describe a potential result of calling the function rather than a detail of how it's implemented.

A potential precedent in favour of throws might be Swift, which uses throws along with explicit propagation (using try). The difference is that try is part of the function call syntax for throwing functions in Swift, whereas in Rust, ? is an additional operation applied to the result of the function, so throwing isn't a property of the function itself.

[burdges]

I think return x without the Ok(x) sounds problematic anyways. Imagine functions like fn foo<R: Results<Foo>>(&self, ...) -> R where Foo: Results<Foo> and Result<Foo,Error>: Results<Foo> where you must nw disambiguate.

[RalfJung]

@Ixrec

Overall, I'm just not convinced Ok(value) is a big enough eyesore to justify these costs.

I think it's not just an eyesore. Currently, Rust fails to entirely deliver to the promise of "propagating failure is explicit via ?", as demonstrated by this example code from the RFC:

fn main() -> Result<(), io::Error> {
    let mut stdin = io::stdin();
    let mut raw_stdout = io::stdout();
    let mut stdout = raw_stdout.lock();
    for line in stdin.lock().lines() {
        stdout.write(line?.trim().as_bytes())?;
        stdout.write(b"\n")?;
    }
    stdout.flush()
}

This makes it look like the flush in the end is an infallible operation, while it is not.
I think "propagating failure is explicit via ?" is a great principle, and really implementing it means we have to provide a way to write functions such that code like the above is not accepted. If we adapt the ? on the signature proposed in the RFC, or the catch right before the body, or the throws annotation -- any of these will be a signal saying that in this function, really, "propagating failure is explicit via ?". Just solving the implicit Ok(()) fails to deliver this, and hence I think it is not good enough.

I somewhat prefer throws because it makes it so that the type after the -> matches the type that is handed to return, and also because it emphasizes the relationship to "true exceptions". (Part of me wonders when/if we will see a version of the Rust ABI which says that "functions returning Result<T, E> actually return T and provide E by unwinding... ;) ). But ultimately, I would happily adapt any of the proposals that provide "full Ok wrapping", preferably alongside a lint that warns about functions returning Result without using this new syntax (i.e., functions that could still hide fallible calls without there being any ? nearby).

Since what I said above is entirely motivated around Result, I agree with @withoutboats that making this entire RFC work only for Result is a worthwhile compromise if it gets us better ergonomics. For Option I feel like the more explicit expression-based handling is way less of an irritation. I am actually not convinced at all that this sugar is even a good idea for Option. I haven't worked with futures, so I cannot speak to them.

[scottmcm]

any of these will be a signal saying that in this function, really, "propagating failure is explicit via ?"

Well said.

I am actually not convinced at all that this sugar is even a good idea for Option.

Here's an example hitting the ?-motivating problems on Option instead of Result: https://github.com/rust-lang/rust/pull/42428/files#diff-667f8101ac1fe1b2f63b7c2ef0691502L1948

Today:

    fn get(self, slice: &str) -> Option<&Self::Output> {
        if let Some(end) = self.end.checked_add(1) {
            (self.start..end).get(slice)
        } else {
            None
        }
    }

With ?-for-Option:

    fn get(self, slice: &str) -> Option<&Self::Output> {
        (self.start..self.end.checked_add(1)?).get(slice)
    }

But the get there is fallible, so fails the "propagating failure is explicit via ?" property.

So, with this RFC:

    fn get(self, slice: &str)? -> Option<&Self::Output> {
        (self.start..self.end.checked_add(1)?).get(slice)?
    }

Edit: Another example where ? is useful on Option, working with iterators. In stable today:

fn fold1<T, I, F>(mut it: I, f: F) -> Option<T>
    where I: Iterator<Item=T>, F: FnMut(T,T)->T
{
    if let Some(init) = it.next() {
        Some(it.fold(init, f))
    } else {
        None
    }
}

In the future with #1859 and trying another syntactic variant of this RFC:

fn fold1<T, I, F>(mut it: I, f: F) -> Option<T>
    where I: Iterator<Item=T>, F: FnMut(T,T)->T
?{
    let init = it.next()?;
    it.fold(init, f)
}

[RalfJung]

But the get there is fallible

True. However, somehow, I am much more comfortable seeing Option<T> as a piece of data here that is propagated outwards. After all, when using Option, None is considered to be expected possibility of failure. There is a reason that Result is #[must_use] but Option is not. So this code does not bother me nearly as much as the one involving flush:

    fn get(self, slice: &str) -> Option<&Self::Output> {
        (self.start..self.end.checked_add(1)?).get(slice)
    }

I am not saying we shouldn't have have sugar that does Some-wrapping. I am just saying that I think this is much less important than getting sugar for Ok-wrapping, and it is worth sacrificing Some-wrapping if it makes Ok-wrapping significantly more ergonomic. If we can have both, even better :)

[nox]

I am strongly against this RFC, in the long term this hurts readability because it makes things nonobvious.

Even when seen "from an error handling mindset", when writing fallible code I very much want to convey explicitly that I'm returning a value that represents success.

If I see a function ending with a ;, I will very much assume that this function returns (), this RFC breaks this assumption.

[glaebhoerl]

This makes it look like the flush in the end is an infallible operation, while it is not.

I guess this is an "agree with the conclusion but not the reasoning" thing for me? It's a basic thing in Rust that a block evaluates to the result of the last expression inside it, and hence its type is also that of its last expression, and here we have a function which returns something of type Result<(), io::Error>, and whose last expression is of type Result<(), io::Error> as it has to be, which is a completely boring and unsurprising instance of the rule. Would you expect this to work differently for Result than for every other type?

[RalfJung]

@glaebhoerl Everything you say makes sense if you think of the Result as data that is passed along. And that's a fine view to take in some situations, I am sure. However, we also make Result the way to do error handling in Rust, and this encourages a different view where a function returning Result<T, E> really actually returns T, but things may go wrong and if they do, E tells us what happened. Using the entire Result as data accidentally is a problem because it breaks this view of things.

Would you expect this to work differently for Result than for every other type?

No, I don't think that would be reasonable. But I want a way opt in to the view saying that these are T-returning functions with exceptions; I want a way to tell the compiler that this is how I think so that accidental use of Result as data does not happen.

Learning that error handling can be "reduced" to this kind of data passing is eye-opening, and I love Rust for fully buying into this. However, in many cases the other "mode" of thinking about error handling more like exceptions in C++/Java is useful. I actually think learning both views helps people understand error handling in Rust and in general better. It also helps understanding the relationship between Rust's Result and traditional exceptions, as it clearly shows how they are really just two different ways to look at exactly the same thing.

[withoutboats]

@rfcbot fcp postpone

Let me say first that I am really excited about this space of designs, and want to see something along the lines of this RFC in the language someday! However, one thing I think we've discovered over the last few weeks is that there are a lot of different things we can do here, and its going to take some iteration and learning to figure out the path forward.

Given that we've got a lot of really high priority & high traffic RFCs going right now, and the impl block coming up in less than a month, I think we can't hope to merge this before the block begins. For that reason I'm proposing we postpone this for the time being, but its something I hope to return to personally in 2018, I think possibly using an attribute to test it out or an eRFC similar to async/await.

[rfcbot]

Team member @withoutboats has proposed to postpone this. The next step is review by the rest of the tagged teams:

• @aturon
• @eddyb
• @nikomatsakis
• @nrc
• @pnkfelix
• @withoutboats

No concerns currently listed.

Once these reviewers reach consensus, this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[kennytm]

For the next checkpoint (2018) I strongly suggest we start from "throwing functions" instead of this RFC.

[scottmcm]

Thanks, @withoutboats; I agree we don't seem to be on track to resolve the diametrically-opposed opinions around this area within the schedule target.

Two more partially-formed thoughts I wanted to include in the record for later:

1. Maybe the signature concerns can be resolved by moving the sigil so it looks more like part of the body:

fn fold1<T, I, F>(mut it: I, f: F) -> Option<T>
    where I: Iterator<Item=T>, F: FnMut(T,T)->T
?{
    let init = it.next()?;
    it.fold(init, f)
}

(This is slightly ambiguous because where clauses can end in +, but it's resolvable with 1-token lookahead)

2. For the "how many arguments to a type constructor" discussion, we do have a pattern of using type aliases to make one-argument Result constructors, so maybe there's a way to use those:

fn write(&mut self, buf: &[u8]) -> usize in io::Result { ... }
fn get(&self, i: usize) -> &T in Option { ... }

(There's also been talk before of defaulting E on Result to Box<Error> or something; that would mean that ordinary Result could be used as a one-argument type constructor here as a default until the code needs to use a particular, possibly-custom error type.)

[tomwhoiscontrary]

I don't see why allowing a general coercion from T to Result::Ok would be a bad thing. The RFC says:

Coercions have properties that are undesirable from this feature however:

• Coercions are transitive, so this would enable things like i32 to Result<Option, WhateverError>.
• Coercions apply in far more locations than are ? boundaries. Notably function arguments are a coercion site, so adding a "try coercion" would mean the "into trick" would happen on everything taking an Option. It would also allow heterogeneous-appearing array literals like [4, Ok(4), 7, Err(())].

But those both seem rather desirable to me.

This example is definitely painful:

A function such as the following compiles, while being almost certainly not what was desired:

fn some_default<T: Default>() -> Option<T> { Default::default() } // Produces None, never using the bound

But to me, that mostly says that allowing Default::default() is a mistake - shouldn't that have to be T::default()?

@RalfJung's example is also painful:

fn main() -> Result<(), io::Error> {
    // snipped
    stdout.flush() // looks infalliable
}

And somewhere i saw something like this (although not quite this, which wouldn't compile anyway):

fn foo() -> Result<(), u32> {
  if everything_is_awesome() {
    Ok(())
  } else {
    Err(911); // the semicolon means this returns Ok
  }
}

To me, what those examples really show is that the implicit return of a trailing expression is a mistake. Sadly, that ship has long since sailed.

Would it be possible to introduce a general coercion, but then not apply it in some particular cases where it's likely to introduce ambiguity? I can imagine that writing the rules for when it would not apply would be a nightmare. And for every case where you avoid a confusion coercion, you introduce a confusing lack of coercion.

[RalfJung]

To me, what those examples really show is that the implicit return of a trailing expression is a mistake. Sadly, that ship has long since sailed.

For my example, I don't see how trailing expression-return makes any difference. This is just as bad:

fn main() -> Result<(), io::Error> {
    // snipped
    return stdout.flush(); // looks infalliable
}

[tomwhoiscontrary]

return stdout.flush(); // looks infalliable

I think the explicit return draws attention to the fact that this returns something, suggesting it is not infalliable. I realise that at this point we are offering subjective opinions on a well-settled of Rust syntax, though!

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[phaux]

How about

fn regular_func() -> T {…}
fn throwing_func() -> T ? E {…}
let regular_closure = || {…};
let throwing_closure = ||? {…};

Edit: other types of functions that we might eventually have

fn function()        -> T           {…}
// returns T

fn throwing_fn()     -> T ? E       {…}
// returns impl Try<Success=T, Error=E>
  
fn generator()       -> T * Y       {…}
// returns impl Generator<Yield=Y, Return=T>
  
fn throwing_gen()    -> T * Y ? E   {…}
// returns impl Generator<Yield=Y, Return=impl Try<Success=T, Error=E>>
  
fn async_fn()        -> ^ T         {…}
// returns impl Future<Item=T, Error=()>
  
fn throw_async_fn()  -> ^ T ? E     {…}
// returns impl Future<Item=T, Error=E>
  
fn async_gen()       -> ^ T * Y     {…}
// returns impl Observable<Item=Y, Error=(), Complete=T>
  
fn throw_async_gen() -> ^ T * Y ? E {…}
// returns impl Observable<Item=Y, Error=E, Complete=T>

let function        = || {…};
let throwing_fn     = ||? {…};
let generator       = ||* {…};
let throwing_gen    = ||*? {…};
let async_fn        = ||^ {…};
let throw_async_fn  = ||^? {…};
let async_gen       = ||^* {…};
let throw_async_gen = ||^*? {…};

So basically the symbols mean:

• -> = returns
• ? = throws
• * = yields
• ^ = awaits

[strega-nil]

@phaux that's surprisingly nice syntax. I wonder if it could actually work?

[rfcbot]

The final comment period is now complete.

[aturon]

This RFC is being closed as postponed. While the lang team is very interested in pursuing improvements to Result-oriented programming along these lines, more design work and experimentation is going to be needed, so we're punting until after the impl period.

Thanks @scottmcm!