Champion "Null-conditional await"

[MadsTorgersen]

• Proposal added: proposals/null-conditional-await.md
• Discussed in LDM
• Decision in LDM
• Finalized (done, rejected, inactive)
• Spec'ed

Design Meetings

https://github.com/dotnet/csharplang/blob/main/meetings/2022/LDM-2022-08-31.md#await

[dmitriyse]

As I am understand it's port from dotnet/roslyn#7171

[yaakov-h]

awaits e if it is non-null, otherwise it results in null

What if I have Task<T> where T is a struct/scalar, or Task without a result type?

[alrz]

awaiting Task yields void so you can't assign it. for a value type I believe it'll be T?.

[gafter]

Confirming @alrz 's response,

If e is of type Task, then await? e; would do nothing if e is null, and await e if it is not null.

If e is of type Task<K> where K is a value type, then await? e would yield a value of type K?.

[ljw1004]

I think this proposal doesn't quit hit the key scenario, for the same reason as we made the last-minute change to "short circuit ?."

Let's review that short-circuit first. We initially had users put ?. all the way through, because we said every single ?. was evaluated from left to right. If x() returned null, then x()?.y() would return null, and so you had to put ?.z else otherwise it would throw.

  x()?.y()?.z

But this wasn't so nice... (1) it created "question mark pollution" everywhere, (2) it THREW AWAY legitimate null-checking information -- in the case where x() returns non-null, and you want to assert that y() will never return null, so you want to write x()?.y().z. The left-to-right order threw away your ability to distinguish that.

Let's now look at the proposed await? operator. And pretend for a moment that await could be written in a fluent left-to-right syntax, rather than being forced to the left...

  await? x()?.y();
  x()?.y()?.<<AWAIT>>;

The proposal as it stands requires you to write await?. This has the same problems as left-to-write conditional evaluation: it peppers your code with too many question marks, and it throws away the legitimate check you might want to make that y() never returns a null Task.

...

KEY SCENARIO. All of that discussion is theoretical. Let's get concrete. The places where I've wanted this feature (and I've wanted it a lot) have almost all been to do with ?. operators on the right hand side. I reckon this is the key use-case for this scenario:

   await? customers.FirstOrDefault()?.loadedTask;

PROPOSAL2: Let's make the same short-circuit ?. evaluation order apply to await as well. So, if the await operand involved a ?. that short-circuited the remainder of the expression, then it can also short-circuit any await operator that operates on the expression.

Written out like that, the proposal feels partly natural but partly irregular/unpredictable. And to avoid the unpredictability, that's why I end up preferring @bbarry's original suggestion:

PROPOSAL3: Let's say that await always does the null-check, on the grounds that (1) awaiting null feels naturally like a no-op, (2) using the await operator is a kind of goofy way to put in a "non-null" assertion into your code, and if you really wanted a non-null check, then there are much better ways to write it.

Note: I've spent the past two months immersed in the Flow language. I really like how it lets me write clean code with a minimum of annotations and it figures them all out for me. I guess that await? feels extra busy in the light of that experience.

[gafter]

@ljw1004 But isn't your proposal an incompatible change? What if someone has come to depend on the code throwing a NullReferenceException? </joking>

[alrz]

Since return null would still create a Task<T> in an async method I also think making await a no op in case the expression is null is ok. However, it should not always check for null, only when it's being used with null-conditional member access. Though that might not be always the case.

[svick]

@alrz

However, it should not always check for null, only when it's being used with null-conditional member access.

I would find it confusing if await a?.b; gave different result than var task = a?.b; await task;. Something similarly confusing already happens with ?. followed by ., but in that case there is a good reason for it. What is the reason here?

[alrz]

I think I gave the wrong impression. I was just pointing out an issue that would arise if we want to deliberately avoid await? syntax. I think it's not "extra busy" as @ljw1004 claims. And I don't think the compiler should "always" emit null check when we're using await to make awaiting a null a no op.

[gafter]

@ljw1004 I think there is a compatibility issue with your proposal.

If we add the feature that await automatically does the null check (whether or not that depends on the syntax of the operand), then people will write code depending on that. But that code will also compile cleanly with an earlier version of the compiler, where it will produce code that throws an exception.

I think that is a fatal flaw.

[bbarry]

What if said change was tied to a CLR version upgrade (say for default interface methods). A compiler targeting the new CLR could let await null be a no-op. and an older compiler wouldn't be able to target the new framework.

[jnm2]

@alrz Always checking for null is no worse than what every using statement does today. It seems like the most natural thing in the world to me.

[jnm2]

@gafter Just brainstorming. To solve the old compiler problem, it could be useful to plan to add the capacity to opt in to new compiler behaviors in the csproj SDK which translate to csc.exe switches, in the same vein as <AllowUnsafeCode> and <CheckForOverflowUnderflow>, but which would cause old compilers to error. Here's what that might look like:

Default template, allows await null

(SDK sets <AllowAwaitNull>true</AllowAwaitNull>)

Csproj, does not override AllowAwaitNull so it stays true:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net462</TargetFramework>
  </PropertyGroup>
</Project>

Class1.cs

class Class1
{
    public async Task Foo(Bar x) => await x?.WhenX;
}

Old compiler gets passed -CompatSwitch:AllowAwaitNull and is smart enough to refuse to build at all (whether or not you await) because it doesn't recognize the AllowAwaitNull switch.

New compiler emits the null check because it recognizes the switch.

In the rare scenario where you need to compile using an older compiler:

(SDK sets <AllowAwaitNull>true</AllowAwaitNull>)

Csproj, overrides AllowAwaitNull so that it can build against older compilers:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net462</TargetFramework>
    <AllowAwaitNull>false</AllowAwaitNull>
  </PropertyGroup>
</Project>

Class1.cs

class Class1
{
    public async Task Foo(Bar x) => await x?.WhenX;
}

Old compiler does not get passed -CompatSwitch:AllowAwaitNull and acts as it always has, and an analyzer sees the await x?.y and warns that you may be awaiting null.

New compiler skips the null check because it was not given the switch, and an analyzer sees the await x?.y and warns that you may be awaiting null.

[svick]

@bbarry @jnm2 I'm not sure this is a problem that can be solved with technical solutions. I think it's an undesirable situation when you find some code e.g. on Stack Overflow, you copy it to your project and it's broken in a subtle way, because you're using a different version of the compiler (compiler error is fine, exception isn't). And neither of your solutions changes that.

[ljw1004]

@gafter agreed it's a fatal flaw. That's a shame.

[jamesqo]

I wonder if it would make much of a performance difference to return null instead of Task.CompletedTask for tasks that complete synchronously. I'm working on an app right now that awaits a huge (thousands) amount of tasks, most of which are completed. I could check !IsCompleted on each task before awaiting it, but it would feel cleaner to do await? task.

[yaakov-h]

@jamesqo Isn't that what ValueTaskis for?

[jamesqo]

@yaakov-h ValueTask<T> is only available as a counterpart to the generic Task<T> type. There is no non-generic ValueTask.

[jamesqo]

To be fair though, I'm not 100% sure await? x will provide much benefit over awaiting a completed task. From what I can see GetAwaiter(), and IsCompleted and GetResult() on that awaiter reduce to just a few flag checks for a completed task. Maybe it wouldn't make so much of a perf difference to have await?.

[Joe4evr]

I'm not 100% sure await? x will provide much benefit over awaiting a completed task.

It's more for cases like var x = await foo?.SomeTask(); If foo turns out to be null, then the await still attempts to call ((Task<T>)null).GetAwaiter().GetResult() which turns into a cryptic NRE.

[jamesqo]

@Joe4evr Yes, I know; I wasn't criticizing the feature, I meant performance benefit. Sorry for any confusion.

[CyrusNajmabadi]

@Xyncgas see #35 (comment)

[CyrusNajmabadi]

Can I pay you $200 to do it, I can pay you before you finish

I appreciate the gesture. But i'd rather the money go to better causes :)

[taori]

@CyrusNajmabadi Are there any guidelines on how a feature like this would be implemented or a branch in your memory which adds a feature (so i can learn from file changes of the PR)? I usually only write analyzers/quickfixes, but i suppose with some example on how to add such a feature i would be capable of doing this. I've cloned dotnet/roslyn and jumped back and forth to get an idea, but i guess if there are any recommended docs to read on this, implementing this feature would be a much more feasible task.

[CyrusNajmabadi]

@333fred any thoughts?

[333fred]

@taori I would suggest coming and talking to us on discord, in one of the communities listed here: https://github.com/dotnet/csharplang/blob/main/Communities.md. This issue isn't likely to be a good place for the type of back and forth real time discussion you'd probably want.

[legistek]

I agree with those suggesting to simply change await to not throw on null, just as with using.

The only code that could be negatively impacted, it seems to me, would be code that relied on the throwing of a null ref exception instead of actually checking for null. We shouldn't let past bad practices impede future progress, especially since, as was pointed out, there is much more appetite for breaking changes now than there was several years ago, and language versions are opt-in anyway.

FWIW, I would change foreach in the same way.

[jnm2]

@legistek Changing await to not throw on null isn't feasible when awaiting Task<int> because you'd need to get an int? out of that, and this would change the meaning of existing code such as:

var someInt = await foo.SomeTaskOfInt;
// someInt is currently `int`, not `int?`

Would every await of a value-typed result start returning Nullable<T>? Almost all the time, this would not even be helpful, let alone how confusing of a change in meaning this would be in the language.

I don't think replacing throwing with using default(int) is a good idea, either.

[Rekkonnect]

I largely agree to this, you cannot break existing behavior that much. await was initially designed following a condition that was taken when designing foreach; ensure that the thing is not null, and throw if it is. Such a design has great advantages, and is the healthiest possible. You cannot simply assume that in any case, the awaited task is null, requiring that you handle the result being null too. In the case of foreach, not much changes if you ignore the entire iteration if the iterated collection is null.

We have already crossed that path. The foreach and await account for nullability. Silently changing foreach behavior will only impact bad practices, like relying on the thrown exception. This will, however bring a slight inconsistency to the table, with await no longer taking a similar path. The healthiest addition would be foreach? and await?, which explicitly indicate the implementation detail.

[legistek]

@legistek Changing await to not throw on null isn't feasible when awaiting Task<int> because you'd need to get an int? out of that, and this would change the meaning of existing code such as:

var someInt = await foo.SomeTaskOfInt;
// someInt is currently `int`, not `int?`

Would every await of a value-typed result start returning Nullable<T>? Almost all the time, this would not even be helpful, let alone how confusing of a change in meaning this would be in the language.

I don't think replacing throwing with using default(int) is a good idea, either.

Good point, I hadn't thought of returning non-nullable value types.

My first response would have been to go with default, but I do see how that would be a pretty significant behavior change.

[CyrusNajmabadi]

I agree with those suggesting to simply change await to not throw on null, just as with using.

The only code that could be negatively impacted, it seems to me, would be code that relied on the throwing of a null ref exception instead of actually checking for null.

I disagree with this, for the same reason I think that . should continue to throw with a null receiver.

Yes, it will almost always throw when it encounters a bug in the code. But that's a virtue. It tells me when I've done something very wrong. Nulls should not be papered over. They should either be explicitly expected, or they should fail catastrophically. Silently ignoring them is just a way to lead to even worse bugs happening

[legistek]

Nulls should not be papered over.

Ok but what's the logic for allowing using (null) then?

await was initially designed following a condition that was taken when designing foreach; ensure that the thing is not null, and throw if it is. Such a design has great advantages, and is the healthiest possible. You cannot simply assume that in any case, the awaited task is null, requiring that you handle the result being null too.

This is true but it's also worth pointing out that foreach and (I believe?) await predate ?..

Before ?., if you said await foo.bar(), you had to check foo first or were rolling the dice. So there was no legitimate reason to ever be awaiting a null value.

But with ?. the temptation is to write await foo?.bar() to skip the null check. This is what I (for one) want to be able to do.

Regarding the value type question, I assume if we used await? with a value type, the resulting type would be nullable right?

In other words assume myClass.foo() returns a Task<int>. If I said:
var result = await? myClass?.foo()
result would be an int? right?

[Rekkonnect]

This is true but it's also worth pointing out that foreach and (I believe?) await predate ?..

Before ?., if you said await foo.bar(), you had to check foo first or were rolling the dice. So there was no legitimate reason to ever be awaiting a null value.

But with ?. the temptation is to write await foo?.bar() to skip the null check. This is what I (for one) want to be able to do.

That is correct, and as the language has been evolving towards that direction, I believe it's safe to say, such proposals like await? and foreach? have supporting grounds.

Regarding the value type question, I assume if we used await? with a value type, the resulting type would be nullable right?

In other words assume myClass.foo() returns a Task<int>. If I said: var result = await? myClass?.foo() result would be an int? right?

Yeah, that would be the most sensible design. You still have the GetValueOrDefault method anyway, if you would like the approach of defaulting to default(TStruct) instead of (TStruct?)null.

[CyrusNajmabadi]

Ok but what's the logic for allowing using (null) then?

Legacy. Doing it again, i would absolutely not do that.

[CyrusNajmabadi]

(I believe?) await predate ?..

Sure. But it postdates our belief that null-tolerance is not a virtue, which is why alter versions of the language embraced explicit null-handling as the mechanism to use, not being null-tolerant.

[CyrusNajmabadi]

But with ?. the temptation is to write await foo?.bar() to skip the null check. This is what I (for one) want to be able to do.

Sure. But i don't. And i'd much rather just say: write await? foo?.bar() if your preference is to be null-resilient for tasks.

Regarding the value type question, I assume if we used await? with a value type, the resulting type would be nullable right?

Correct.

[legistek]

Yeah ok. Well like I said I can live with await? as I hadn't considered the nullable value type issue, and it sounds like await? solves that then.

I still would let foreach blow past the null, but I guess I just like to live dangerously.

Ok but what's the logic for allowing using (null) then?

Legacy. Doing it again, i would absolutely not do that.

Interesting! A true believer then. :)

[SebastianSchumann]

In the meantime we have .Net 8.
ConfigureAwait has been implemented with ConfigureAwaitOptions. If you specify SuppressThrowing, an ArgumentOutOfRangeException is thrown if the task is not terminated correctly and should return a value.

Wouldn't that also be an option for this await implementation?

I could imagine something like this:

class Foo
{
    public async Task DoAsync() { }

    public async Task<int> GetValueAsync() => 42;
}

// usage
Foo? foo = null;
await foo?.DoAsync();        // simply does nothing - no exception is thrown
await foo?.GetValueAsync();  // simply does nothing - no exception is thrown

var x = await foo?.GetValueAsync(); // an ArgumentOutOfRange is thrown

Same behaviour for ValueTask? and ValueTask<T>?.

[Rekkonnect]

What you're saying is that the nullability annotation of the type should control whether the task is allowed to be null and thus suppress throwing the null exception. Keep in mind that Task is a reference type, and the annotation is part of the metadata, not within the actual type system. The awaited literal's type (Type?)null is still a null used in a place where Task (or any implementing an async iterator) is expected. During runtime you have no idea about the type being converted into Type?, this would be entirely a C# detail implemented in a very hacky way, in my opinion.

[SebastianSchumann]

What you're saying is that the nullability annotation of the type should control whether the task is allowed to be null and thus suppress throwing the null exception.

No. My example code should only illustrate how await might work. I am aware that the nullable annotation is only metadata and you forward a null reference.

But the compiler "knows" whether the task has a return value or not. If not no exception should be thrown. If the result value is used in any way (assignment to variable, forward to using, ...) an ArgumentOutOfRangeException should be thrown in the same way like ConfigureAwait(ConfigureAwaitOptions.SuppressThrowing) will do.

I edited my sample code to be more clear.

[HaloFour]

@SebastianSchumann

But the compiler "knows" whether the task has a return value or not.

This would silently change the behavior of existing programs and affect the return type of the expression. I don't think that would be palatable. The Task<T> could always be null, even if not annotated as an NRT.

[SebastianSchumann]

@HaloFour
Yes you're right, this will silently change the behaviour for existing programs. But the difference is throwing a NullReferenceException vs. an ArgumentOutOfRangeException. I can accept that argument. In that case await? might be a better choice.

Btw:
I don't understand what NRTs have anything to do with this feature. I know that a reference type could be null even if it's not annotated as an NRT. If you will change the line Foo? foo = null to Foo foo = null. This will have absolutely no effect for the usage and the rest of my sample code. The ?.-operator has nothing to do with NRTs. I'm aware of this. My sample code should only show how await should behave if a null-reference has been specified.

[alrz]

To my surprise there are many hits for await on conditional member access most of which are invalid?

https://github.com/search?q=%2Fawait%5Cs%2B%5Cw%2B%5C%3F%5C.%5Cw%2B%2F%20lang%3AC%23&type=code (not a comprehensive query)

Would it make sense if the compiler adds a null check in the presence of a null-conditional access with the current syntax? It seems like that's sort of the expected behavior given that many hits. (edit: actually there's already a null check there, it could only await on the success branch, so it still throws on a null task)

[julealgon]

It's interesting this popped up again as I'm in the middle of converting large amounts of APIs from sync to async in one of our largest solutions.

I must say this is super dangerous when performing such conversions. I've hit a few cases here where we had "callback/event" properties typed as Action that I was forced to convert to Func<CancellationToken, Task> to make sure those callbacks were awaitable and could then call async methods inside them.

This caused code to change from this:

myClass.SomeCallback?.Invoke();

To this:

if (myClass.SomeCallback is not null and var someCallback)
{
    await someCallback.Invoke(cancellationToken);
}

But that was after I had made the changes and before sending the PR for review that I noticed there could be a problem and switched to the explicit defensive check.

It would've been nice if "it just worked" and I could keep it like:

await someCallback?.Invoke(cancellationToken);

Which is exactly what @alrz is proposing:

Would it make sense if the compiler adds a null check in the presence of a null-conditional access with the current syntax? It seems like that's sort of the expected behavior given that many hits.

I'd even go as far as to suggest this null check should be done regardless of the presence of null coalescing operators in the right-hand side. It would make await behave similar to how using behaves in the presence of null. This works just fine, for example, with not even a compiler warning or hint anywhere:

using ((IDisposable?)null)
{
    Console.WriteLine("Hello world");
}

If await behaved the same way I think it would be consistent and not surprising.