Use of await and yield

[TheNavigateur]

Use of await and yield, that seems the most natural to me, with explanation.

[gilbert]

This is really nice. Does this mean it's possible grammar-wise?

What would happen in this case? Syntax error?

var result = 10 |> g |> await;

[TheNavigateur]

@gilbert Possible? Requires review, but I'm pretty sure it is.

Your example? Yes, an error because a function expression would be expected, given the last |>.

Your thoughts?

[azz]

I mentioned in #53 (comment), but I think this could be confusing.

userId |> await fetchUserById |> getAgeFromUser

await in this context should asynchronously resolve the expression fetchUserById, which assuming it is a function would be a no-op, getAgeFromUser would then receive a promise as an argument. Otherwise doesn't it violate Tennent's Correspondence Principle?

Something like

userId |> fetchUserById |> getAgeFromUser(await ?)

makes more sense to me because it would expand to

userId |> fetchUserById |> promise => getAgeFromUser(await promise)

which is the same as

getAgeFromUser(await fetchUserById(userId))

[TheNavigateur]

It's not a normal type of closure (|> only allows a function expression after any await/yield, or just a function expression), so I'm not sure it necessarily violates any principle or required pattern. I'd be interested to see a demonstrable problem with doing so, even if it does.

Furthermore, it reads fairly naturally to me, in order:

userId |> await fetchUserById |> getAgeFromUser

"Pass userId to and wait for fetchUserById, pass result to getAgeFromUser"

I'm not sure how your alternative should be read and understood?

[gilbert]

Some more examples to consider:

var A = fetch(url) |> handleFetchPromise;
var B = await fetch(url) |> handleFetchPromise;
var C = ( await fetch(url) ) |> handleFetchPromise;
var D = await (fetch(url) |> handleFetchPromise);

Which are equivalent, which are not?

[TheNavigateur]

Personally, I would have B equivalent of D (that is, |> having higher precedence than await on both sides of it).

Your and anyone else's thoughts?

[ljharb]

Given await's existing low precedence, i'd expect C to be necessary to await the result of the fetch, whereas B and D would be equivalent.

[TehShrike]

I think this change would make the pipe operator too complicated. I would rather see the pipe operator stay simple and easy for people to build a mental model of, and see the async case handled by some new Promise.waterfall proposal.

[TheNavigateur]

Rejecting the ability to await async function results in a pipeline would make it less capable than ordinary () syntax for functions. Are you ok with that?

[ljharb]

There's no need for a "waterfall" when arrays have a reduce method, and async functions can use loops; I'm not sure why that would be helpful.

[TehShrike]

@TheNavigateur it would make the pipe operator more complex, which I am not ok with. I'd rather people had to do handleFetchPromise(await fetch(url)) than remember the difference between await fetch(url) |> handleFetchPromise and ( await fetch(url) ) |> handleFetchPromise

[TheNavigateur]

@TehShrike Your primary concern is not affected by this change, and would exist anyway: await fetch(url) |> handleFetchPromise and ( await fetch(url) ) |> handleFetchPromise would still be valid without the change and its meaning would still depend on the final precedence decided between |> and await. Developers would still have to double check the precedence before presuming anything about either case.

But the larger question is: you want to force people to have to do () for async pipelining, while allowing |> only for sync pipelining. That's definitely interesting. Personally, that would make me abandon |> altogether, for only one main reason: I would want only 1 or the other in my entire codebase, for the sake of consistency and quick readability. Secondarily, if I wanted to transition one or more sync pipelines into async, it would not be as easy as if I just stuck with () syntax throughout every system. Are you comfortable that there might be others that would abandon |> altogether, for the same reason?

[TehShrike]

If your understanding of await is nothing more than "makes promises behave like values", then you're dealing with a leaky abstraction. async builds on generator functions to make promises easier.

The pipe operator, in JS or any other language, is great for composing a bunch of transformation functions. Adding generator/promise awareness on top of that is not necessary.

[TheNavigateur]

async await doesn't just make promises easier. It abstracts away promises altogether, such that a developer can do async programming without even necessarily knowing anything about promises. async functions are declared with the return value as the resolution value of the promise, e.g. return 5, such that async functions can be treated as any other function except only for the fact that it is asynchronous.

Let's examine your claim that this is a "leaky" abstraction: I would claim it is a rock-solid abstraction. However, perhaps you can give an example of where this isn't the case.

[littledan]

Maybe this is a job for Promise.prototype.then().

[mAAdhaTTah]

I know most of the discussion here is around await, but I also find the inclusion of yield here a bit weird, mostly given that generators can have new values passed in by the caller. If you yield in the middle of a composition, the next function in the chain could (will always?) end up with an entirely different value. I suppose that's expected / intended, but it seems weird to compose a pipeline whereby a value can be swapped out by outside code. Do we really want to enable that?

Nothing screams "don't do this!" about including yield (if we're going to include await), but I think the overall complexity of the proposal ramps up quickly when including both.

[TheNavigateur]

@littledan Usage example?

@mAAdhaTTah I'm simply proposing making it as capable as the existing () syntax. You can yield/await inside existing () based pipelines, so anyone doing so can easily transition to the |> operator without any loss. If not supported (while await would seem to be the more common case) they would be forced to continue to use (), which could cause them to abandon |> altogether, to preserve consistency throughout their code.

[jridgewell]

Throwing my 2¢ in:

var A = fetch(url) |> handleFetchPromise;
var B = await fetch(url) |> handleFetchPromise;
var C = ( await fetch(url) ) |> handleFetchPromise;
var D = await (fetch(url) |> handleFetchPromise);

This is how it parses today in Babel (and it all makes sense to me).

• A does what you think
• B and C are exactly equal, await the fetch response only handleFetchPromise(await fetch(url))
• D awaits the return value of handleFetchPromise(fetch(url))

I think the more interesting cases are awaiting after a |>:

var E = fetch(url) |> await handleFetchPromise;
var F = fetch(url) |> await handleFetchPromise |> next;
var G = fetch(url) |> handleFetchPromise |> await next;
var H = await (fetch(url) |> handleFetchPromise);

Currently, babel awaits for the function reference that will be invoked (not the actual invocation of it). This is not what I'd want when writing it, and I agree with the PR's proposed semantics. Specifically, it'd be a syntax for |> await or |> yield which reads "await/yield the return value before invoking the next with its final value".

• E is the return value of handleFetchPromise invoked with the unwrapped response: handleFetchPromise(await fetch(url))
• F is next(handleFetchPromise(await fetch(url)))
• G is next(await handleFetchPromise(fetch(url)))
• H is the same as D, just highlighting that you can await on the last call in two places.

We can wait on the expression handleFetchPromise by wrapping in parenthesis:

var I = fetch(url) |> (await handleFetchPromise);

This breaks the |> await syntax, making it clear you want to invoke the unwrapped handleFetchPromise: (await handleFetchPromise)(fetch(url))

[littledan]

@jridgewell That's a coherent proposal, and pretty intuitive. I suppose we could specify that by putting a nolookahead for await and yield after |>, and then giving special behavior to the sequence |> await and |> yield. I'm still wondering, though, would that case be served well by .then()? For example, your example E could be written as

var E = fetch(url).then(handleFetchPromise);

Maybe it's not so easy to understand then, though. The main difference in semantics is that |> await would always wrap in a native promise, and then would just treat the receiver as a thenable directly; I doubt this will affect most cases in practice. Are there any practical downsides to using .then() here? Is it that much easier to understand |> await?

[gilbert]

@jridgewell Thanks for your input. If I understand correctly, one thing that strikes me as odd is the relationship between B and E:

var B = await fetch(url) |> handleFetchPromise;
var E = fetch(url) |> await handleFetchPromise;

// Are B and E equivalent?
// What about J?

var J = await fetch(url) |> await handleFetchPromise;

I think it would be ideal for an await to attach to what immediately follows it, as opposed to what precedes it. Using this rule we get the following:

var B = await fetch(url) |> handleFetchPromise;
//=> handleFetchPromise(await fetch(url))

var E = fetch(url) |> await handleFetchPromise;
// => await handleFetchPromise( fetch(url) )

var F = fetch(url) |> await handleFetchPromise |> next;
//=> next(await handleFetchPromise( fetch(url) ))

var G = fetch(url) |> handleFetchPromise |> await next;
//=> await next( handleFetchPromise(fetch(url)) )

var H = await (fetch(url) |> handleFetchPromise);
//=> await handleFetchPromise(fetchUrl(url))

var J = await fetch(url) |> await handleFetchPromise;
//=> await handleFetchPromise(await fetch(url))

And your rule about "breaking the |> await syntax" would still apply.

[TheNavigateur]

@gilbert Yes, B and E should be equivalent, in my opinion. J should be different (awaits the fetch(url) instead of synchronously passing the promise).

As for your examples from E to J, that's what the PR recommends.

Your B example is inconsistent with the rest only with the fact that await is gluing with a higher precedence than |>, whereas in the rest of the examples |> is having higher precedence. I would have it effectively the same as E so that |> consistently has higher precedence

[gilbert]

@TheNavigateur B is consistent with how it parses in Babel today, as @jridgewell explained. It's also consistent with the F example I wrote, which behaves the same way.

[TheNavigateur]

@gilbert B is not exactly consistent with F:

F doesn't glue await to the expression after it, i.e. function itself, it glues to the |> operator to imply awaiting the result of the function (think awaiting the function vs awaiting the function's result)

Whereas, B glues await to the expression after it, more strongly than |> glues to that expression (i.e. it puts await on a higher precedence than |>, unlike all the other examples.

Basically, if we put |> on a higher precedence than await, all the possibilities resolve consistently.

[gilbert]

You might be misreading the code I wrote, so let me compare @jridgewell's suggestion to my suggestion side by side:

// @jridgewell's
var F = fetch(url) |> await handleFetchPromise |> next;
//=> next(handleFetchPromise(await fetch(url)))

// @gilbert's
var F = fetch(url) |> await handleFetchPromise |> next;
//=> next(await handleFetchPromise( fetch(url) ))

(our Bs are the same)

The F example I wrote does make await glue to the handleFetchPromise expression after it, as opposed to the fetch(url) expression before it. But I might be misunderstanding what you mean by glue.

[jridgewell]

I'm still wondering, though, would that case be served well by .then()?

I think in general, yes, anything can be written as #then chains. But it faces the same issues as normal promise chains (sharing variables, complexity, etc). I think directly supporting sync syntax through await is a great goal for this (or a pretty quick follow up) proposal.

I think it would be ideal for an await to attach to what immediately follows it, as opposed to what precedes it.

That makes more sense than my proposal, I like it. In this case, E and H are equivalent, which I think is fine.

[TheNavigateur]

@gilbert @jridgewell

it would be ideal for an await to attach to what immediately follows it, as opposed to what precedes it

In

x |> await f

await is not attaching to what immediately follows it. It is not attaching to the function, but rather to the |> operator, to await the result of the function f given the argument x.

So I think simply giving |> a higher precedence than await is the key that resolves all possibilities, IF you are comfortable that

await x |> f

is the equivalent of

await (x |> f)

(I am)

Otherwise, you would need to give a higher precedence to await when it appears just after a |>, vs lower otherwise.

I think that simply giving |> higher precedence than await consistently is simpler to conceptualize.

[azz]

I think we should either come up with a case where await is useful with |> or forbid it until we find one.

When replacing promise chains without introducing unnecessary bindings?

async function rejectIssues(user) {
  return getIssues()
   .then(filterByCreator(user))
   .then(assignLabelsToIssues(["wontfix"]))
   .then(closeIssues)
}

Current translation:

async function rejectIssues(user) {
  return closeIssues(
    await assignLabelsToIssues(["wontfix"])(
      await filterByCreator(user)(
        await getIssues()
      )
    )
  );
}

With pipeline operator:

async function rejectIssues(user) {
  return getIssues()
    |> filterByCreator(user) |> await
    |> assignLabelsToIssues(["wontfix"]) |> await
    |> closeIssues;
}

[littledan]

Or, alternatively, with the other pipeline operator PR, as well as the partial application proposal rather than the "elixir option",

async function rejectIssues(user) {
  return getIssues()
    |> await filterByCreator(?, user)
    |> await assignLabelsToIssues(?, ["wontfix"])
    |> closeIssues;
}

[TheNavigateur]

@littledan Advantage of |await> is that there are no precedence related anomalies like you mentioned before with |> await. Do you not like it?

[azz]

I'm fine with |> await |>, await f(?), and even |await> (though I said that more as a joke than anything else), but I'd be really careful with |> await f as it makes it complex to refactor the code. #66 (comment)

await readFile("foo.txt")
 |> stripBom
 |> await writeFile("foo.txt")

If await writeFile("foo.txt") is moved to a different line it means something else.

const file = await readFile("foo.txt")
  |> stripBom;

await writeFile("foo.txt") // WRONG!
// actually you want:
//   await writeFile("foo.txt")(file)

Worse, it causes complications when moving some code into a pipeline.

/** @returns {Function} */
async function getResolver() {
  const { type } = await readConfig();
  return resolvers[type];
}

// original
(await getResolver())(query);

// attempt 1
query |> await getResolver(); // wrong, this is `await getResolver()(query)`
// attempt 2
query |> await getResolver; // wrong, this is `await getResolver(query)`
// attempt 3
query |> (await getResolver()); // correct

[Volune]

I'm not comfortable with |> await, not that I dislike it, but it feel kinda ambiguous for the reader.
x |> await f is not the same as x |> (await f), but x |> await await f is the same as x |> await (await f). Yes that would be terrible code. But my point is, it's not clear for the reader what is evaluated before calling the function (evaluating f or whatever expression that is) and what is evaluated after (await).

|await> feels like yet another operator, with letters inside. Also I wonder how the parser will work when parsing x|awai>y or x|await>y, both being valid

That made me think of maybe another alternative: |> await:

async function rejectIssues(user) {
  return getIssues()
    |> await: filterByCreator(user)
    |> await: assignLabelsToIssues(["wontfix"])
    |> closeIssues;
}

(Replace the : with whatever is convenient for the grammar and the reader)

[littledan]

@azz I'm not sure what you mean in your example. Are you imagining that pipelining puts the left hand of the operator into the first argument position of the right hand, or are you imagining that something like readFile is based on manual currying?

[azz]

I was assuming the functions were curried.

[TheNavigateur]

@Volune For me, since await: would only be valid after |>, just like |> await would have special meaning unto itself, both variants are already effectively self contained operators unto themselves, so for me the construct had might as well be an unambiguous looking operator, e.g. |await>, |->, etc., which I think causes minimum semantic collision / confusion with other similar looking expressions

[littledan]

Yes, it's true that the |> await f proposal makes a strange distinction, where it has completely different semantics from |> (await f). I guess that's the cost of it; this is why I was more in favor of |> await |> above until realizing those syntactic issues.

I'm a bit uncomfortable with |> await: just because : has so many meanings already: It's for labels, and also for object literals, and also for types in TypeScript. This doesn't really have to do with any of those three.

[TheNavigateur]

@littledan Do you envisage any disadvantages with |await> f?

[littledan]

@TheNavigateur I think it's a little odd that you can't use it in a trailing way. Trailing await seems just as useful to me as one in the middle of a pipeline.

[TehShrike]

I would rather see the pipe operator not support await (at least not right away) than see it land with surprising edge cases or extra syntax.

[littledan]

Should we ban the cases where await immediately follows |> to allow for future expansion later?

[benjamingr]

I still think it's easiest not to support async/await in the middle of a pipe chain and then add support later. Note that functions can always be lifted to take async params, so code like:

let res = (await readRequest()) 
  |> await parseContent
  |> await retrieveFromCache
  |> await capitalizeOnServer
  |> JSON.stringify

Can be (assuming await has a lower precedence than |>, otherwise add parans):

function lift(fn) {
  return async function lifted(...args) {
    let unlifted = await Promise.all(...args);
    return await fn(...args); // assume no `this`, can change to `.call` according to proposal
  }
}

// assuming `lift` was called on all the above functions:
let res = await readRequest()
  |> parseContent
  |> retrieveFromCache
  |> capitalizeOnServer
  |> stringify

Which means you never have to await in a promise pipeline.

Now would be a good time to point out that I see |> as a tool mainly to be used on pure functions - since |> does not have any error semantics (good thing) I'm not sure how error handling is supposed to work. Where a(b(c(x))) looks very reasonable to me await a(await b(await c(x))) is pretty full of mines and can lead to difficult code since often async flows relate to acquiring and releasing resources.

---

F# had a similar proposal - let's try summoning @WallaceKelly for their opinion about it.

[TheNavigateur]

@littledan |await> allows a trailing await perfectly:

const
    user =
        lastComment
        |> userIdFromComment
        |await> fetchUserById

[TheNavigateur]

An alternative, is to allow |> await |> even with the ASI hazard that necessitates a semicolon in the case of a dangling await, and let that be communicated to developers as a caveat.

Otherwise I think an operator like e.g. |await> minimizes semantic ambiguity / similarity / collision with other constructs when reading it, while allowing the last function call to be awaited without any ASI hazards, thereby avoiding all the problems with the alternatives

[littledan]

OK, I will prepare a PR which simply bans |> await in async functions then.

[TheNavigateur]

@littledan Did I satisfactorily address or miss your point about the trailing await?

[littledan]

I'm not sure communication is enough. I will raise the issue to TC39 and present a range of options.

[benjamingr]

I think it's important that we keep the possibility of extension in the future.

As for yield, I can see how it would be useful in a pipeline but not in real world scenarios and I think we should consider banning it in the middle of a pipeline (for starters) too.