Asynchronous coffeescript made easy, part II

[matehat]

Hi folks,

I think we're slowing getting there. Here's an update:

I gave a shot at replacing the ~ character in positional arguments with defer (following weepy interesting suggestion). It makes it more obvious, as gfxmonk pointed out, that it's not just continuing execution and it appears in the called function arguments since, this way, we can have only one special keyword/character for the whole thing. I also kept the ~> arrow, because that gives us more flexibility than just finding the closest enclosing function for yield, especially useful for a programming language where anonymous functions are used that much.

Good news: a huge test suite based on gfxmonk's branch (and beyond) is passing on my branch. There are still one or two issues, though. You can take a look if you wish to see example usage.

http://github.com/matehat/coffee-script/blob/deferred/test/test_deferred.coffee

I'd also like to hear your thoughts on the whole yield/~> issue. Keep in mind, though, that the choice of character, ~>, is really just a suggestion. The core question is about making the invokation of implicit callbacks transparent, while enforcing, with a different keyword, the fact that it's calling and not returning, all the while making the callback container explicit, somehow. I suggested it to be specified with a special character. It could really be anything, don't take a position just on how it looks and what glasses you need to wear when hacking on it.

And nothing in there is magic, you can take my words for it ;)

P.S. gfxmonk I'd like to hear what you think about my implementation to date. Do you find it satisfying, or you think it's missing a point somewhere?

[matehat]

Just a thought : would you guys prefer something like :

return_arg: (x) ->> process.nextTick -> yield x
return_call: (x) ->> process.nextTick -> yield x()

single_def: (x) ->> yield return_arg(x, defer)

callback_called: false
sys: require 'sys'
single_def "a", (result) ->
  callback_called: true
  ok(result is "a")
  return_arg undefined, -> ok callback_called is true

That is, to replace ~> with ->>? I think it is far more obvious and distinguishable than ~>.

[StanAngeloff]

First of all, great work on the deferred branch. It would be an awesome feature. Here are my thoughts on it so far:

yield is a very bad choice. I have been working on the Mozilla platform for a while and I have several files using the native yield operator introduced in JavaScript 1.7. Since we are dealing with callbacks here, and not with generators, I reckon we should choose something different. I can think of resume and continue on top of my head, e.g., resume x.
It is also worth looking at C# as well -- yield behaves similarly to JavaScript. It comes handy when you want to exit execution of a method, perform some calculations and return to continue the loop -- pretty much the definition of a generator.

As for the ~> syntax, I reckon it's too similar to -> and will get missed quite a lot. In my view, it should ideally be -> decorated with a keyword, so in your case using yield:

return_arg: (x) yields -> process.nextTick -> yield x

With regards to using ~ in the code this confused the hell out of me, LOL. It could be replaced with another keyword, perhaps same as the above... so the final output can read something like:

readFirstFile: (dir, yield, error) yields ->
  [err, files]: fs.readdir dir, yields
  return error(err) if err
  [err, out]: fs.readFile files[o], yields
  return error(err) if err
  yield content

[matehat]

Great comments. I agree we can find something better than yield for all this and a decorator-oriented alternative to the ~> is great. What do you think about defer in place of ~, as it is currently the case in the branch?

I personally think resume could be real nice for the decorator and the way to call the implicit callback, though I think the keyword used on deferred function calls should be different, so we don't get too confused on the order of execution of it all. So it'd go like :

readFirstFile: (dir, error) resumes ->
  [err, files]: fs.readdir dir, defer
  return error(err) if err
  [err, out]: fs.readFile files[o], defer
  return error(err) if err
  resume content

[StanAngeloff]

LOL again, I was going to post this originally, but thought you had good reasons not to use defer from gfxmonk's branch. I like it too much !

[matehat]

:) Well, as posted initially, that's the keyword currently used in place of ~.

Hmm, now wondering what's the gain in putting a decorator like resumes over using a straight callback argument like resume :). What do you think about ->>? That would make it like :

readFirstFile: (dir, error) ->>
  [err, files]: fs.readdir dir, defer
  return error(err) if err
  [err, out]: fs.readFile files[o], defer
  return error(err) if err
  resume content

[StanAngeloff]

I personally find ->> not so useful -- it does not tell you it's adding in an implicit callback to the definition.. but then again, does it need to? One of the goals here is to make it as close to the procedural model as possible so maybe the decorator-keyword is a no-no. We also have the _ph_at arrow => which without reading the docs has no meaning either. I can't decide really, there are cons and pros in any case, but I am leaning towards a decorator-keyword.

[matehat]

Well, I introduced implicit callback as an alternative to "translate returns into callback invokation" which I think is not a good option. I agree that the implicit callback issue is far from being a necessity, since explicitness does not cost much in coffeescript (think (->) being a complete function definition :). I only thought that a shorthand for simple use cases could benefit for iterator-like use cases, as well as for the whole asynchronous context.

That said, the thing with the decorator alternative is that it gets in the way when writing small wrappers. For instance, the following would be too verbose if we wrote it with decorators, instead of special arrows :

handleerr: (fn, args...) ->>
  if ([err, out]: fn args..., defer)[0] then throw error(err)
  else resume out

safe_readdir:  handleerr <- {}, fs.readdir
safe_readfile: handleerr <- {}, fs.readFile

readFirstFile: (dir) ->>
  try:
    files: safe_readdir  dir, defer
    resume safe_readFile files[o], defer
  catch err
    puts "Error while reading first file: $err"

(A lot is going on behind this : the equivalent compiled javascript is 51 LOC! )

[timbertson]

gfxmonk I'd like to hear what you think about my implementation to date. Do you find it satisfying, or you think it's missing a point somewhere?

I've had a brief look at your code, and it's much nicer than my hacky progress. I don't yet know what it covers or is missing, but it looks pretty great. Thanks for working on it, I haven't had enough time to make any progress lately.

The features, however, I still disagree with.

I the use of a keyword to stand in for an argument is mostly an aesthetic one, though I'm still confident that having the defer keyword out the front of a call reads better than placing something inside the argument list. But that's presumably easy enough to change if we decide to.

Similarly, I don't think we need another operator to specify that the function requires a callback (~> or ->>). It's just another thing to forget when coding, I think we should simply infer it from the use of defer. But again, this is mostly cosmetic.

The most important issue I think is having the yield (or whatever it gets called) construct as distinct from a return. I understand that you don't want to restrict the use of this feature to the single case of returning asynchronous values, but the way I see it:

• we don't need to plug any other holes in javascript with this feature. The case of yielding to callbacks multiple times is already perfectly well serviced by anonymous functions
• having the ability to actually return from an async function (as distinct from yielding something) is outright dangerous, as it is the equivalent of stopping the entire execution of a program silently. If you accept that we shouldn't use this feature for iteration (see above point), then there is no good reason to actually return - rather you should return into your callback, or you should raise an error.

Sorry to be so opinionated, but I have been trying to write an actual application with a heavily asynchronous datastore for a few months. I started with native javascript, then tried narrative JS, then strands, then async.js, and then I even wrote a macro-laden version of parenscript in order to solve the problem of asynchronous returns. So I feel like I have a pretty good grasp of what, exactly, needs to be solved. It isn't yet-another iteration mechanism, it is simply an issue of returns.

in regards to those other libraries, narrativeJS (and strands) were too ambitious, and the original author admits they were going about solving the problem in the wrong way. My approach is almost identical (in functionality) to that provided by async.js, it's just a while lot more simpler to write and use. So it's not just me who has these specific ideas about what is required.

[tim-smart]

What would the difference be between me going:

myFn: (arg, cb) ->
  cb compute_result arg

and

myFn: (arg) ->
  yield compute_result arg

If there isn't any difference, why are you using yield?

[matehat]

Tim-Smart: Is there a difference between returning a value from a function and requiring a pointer to be in the arguments, eventually used as the place to store some value? Or is there a difference between list[1:-1] and list[1:len(list)-1] in python? That's never a matter of whether if it can be done in another way, but one about the form we wish our code to have. yield calls an implicit callback, which could be passed explicitly, of course, nothing's new about it.

The point is around the fact that many of the functions we use and produce in asynchronous/event-based contexts, and many others, requires a callback to be passed as the last argument. In those cases, which represents a big portion of them, instead of putting a name at the end of the argument list and remembering it ever time you need to invoke it, we suggested to make that transparent to the programmer, so it becomes straightforward. Most programmers are used to bigger conceptual challenges that those implied in learning that yield would call an implicit callback.

gfxmonk initially proposed to just used the return statement, and swap it for a call to the implicit callback, then return null, as in most of those cases, you don't want to continue execution in the same function. I was opposed to this, because I thought we would need to assume that an occurrence of defer in a function implies that the function should never return otherwise, in any context, and I think that's a unnecessary restriction. Also, it adds complexity to an otherwise really simple keyword (return) which is that it does not always behave in the same way. That's where I proposed to use another keyword, which could be used in a more general context, then adding the whole deferring magic as a particular use case of that.

gfxmonk: I can assure you a swapping of the current state of my branch to what you wish it was would be a matter of minutes. I can't say the same about our divergence of opinions :) I just hope we can find an agreement to get things moving forward...

I truly believe an implicit callback mechanism would make our code lighter, though I agree, not more potent. It does also provide a really convenient and flexible plug for the defer magical machinery. I tend to not restrict flexibility because we think people shouldn't need it in any other way. Anyway, people doing asynchronous programming are supposed to know what they're doing. We shouldn't need to impose some implied (and not obvious) conventions based on our sense of good practice.

Also, about the defer keyword holding the position of the implied callback in the arguments, I thought it was nice because it would allow us to not have to remember another character, ~. It is not a necessity. I could really put it back upfront and even remove the positional placeholder as it would not occur so frequently, as you pointed out.

[matehat]

Briefly, to be clearer about an implicit callbacks (read yield or whatever), the good thing about it is its striking semantic when used in large pieces of code. When you have an explicit callback at the end of the argument list, it is not obvious, at first, when a called function is the callback.

Also, about the ~> character, I'd be happy to avoid that entirely and just infer the yielding function from the closest embodying function, as some people suggested, though it will restrict the potential of its use, if people are opposed to it.

[matehat]

(Disclaimer: I'm sorry about bringing this forward all the times, but I'd really like it if we can get a consensus sometime, so we can get something into trunk. I'm currently developing a open-source server based entirely on Node.js and all its async features, so it would be nice if we could be able to start actual use of a async-friendly syntax).

After some chat with jashkenas on irc, we concluded the feature is one that's really hard to wrap out our head around, so this stuff really needs to not be obscure, if we want it to be in core language. The feature should appear useful to anyone, for other purpose like in ajax jQuery for instance.

The defer keyword appearing in front of the function call is fair enough, but I think it could also be in place of a position argument in case the implicit callback was to be put anywhere else than at the end. That way, we get the leading keyword, keep the positional flexibility and don't have to learn another character, ~. So the following two statements would be allowed :

[error, files]: defer fs.readdir "lib/"
[error, out]:   readSomeFile files[0], defer, lastarg

Now for the way to yield values to the implicit callback. I still don't think we should mess with the return keyword. Normally returning from within one of the deeply nested function is not always a dangerous situation and might even sometimes appear as a useful one. I just can't think of how we could present, without putting confusion in the minds of people, the fact that return is not behaving the same way in the asynchronous magic we are proposing. In the following example :

readFileAsync: ->
  for file in (defer fs.readdir "lib/")[1]:
    return (defer fs.readFile file)[1]

I don't see how we could explain that this will fire an implicit callback more than once, without confusing people. We could just settle on that good old "invoke the callback" syntax for now,

readFileAsync: (cb) ->
  for file in (defer fs.readdir "lib/")[1]:
    cb (defer fs.readFile file)[1]

So now, we're back with simplicity, no more keywords than defer and no special character. What do you think?

[weepy]

I like it. After seeing all the ideas - I've really come to the conclusion that it should be as simple as possible, and as little deviation from standard Coffee.

In fact I'd even go further and say that only one of your two examples:

[error, files]: defer fs.readdir "lib/"
[error, out]:   readSomeFile files[0], defer, lastarg

should be allowed. I prefer the second as it's most inline with normal function calling, but either way we need a standard otherwise it's confusing (for no tangible gain).

It seems we're close !

[hen-x]

+1 for only adding defer for now, since it seems like calling async functions, rather than defining them, is the main problem that needs solving. For the same reason, the syntax of passing defer in argument position makes sense to me. Syntactically, it is more obvious that a callback is being passed (albeit an automatically-generated one), which makes it symmetrical with the shape of an async function signature. Explicit callback functions are an essential part of CS, and they aren't going to disappear. While it is nice to have a syntax that makes them easier to use, I think it is a mistake to altogether cloak their presence as an argument in a function call.

[timbertson]

I'm glad to hear the code itself isn't too divergent, even if our opinions are ;)
Would it be worth actually implementing some sort of switch between our two behaviours? Not in the long term of course, but perhaps for the short term so that if anyone who isn't myself or matehat wanted to try out writing some actual code, we could get feedback from people on which approach is more useful (and least confusing).

I truly believe an implicit callback mechanism would make our code lighter, though I agree, not more potent. It does also provide a really convenient and flexible plug for the defer magical machinery. I understand you haven't met many such situations, in your experience, but I tend to not restrict flexibility because we think people shouldn't need it in any other way.

I believe that by making yield separate from return, you would be breaking half of the equation. Iterators and continuations should not be confused, and I think yield (or even resume) does exactly that. I maintain that "return" is exactly what you are doing (and the only thing you ought to be doing) when you call your continuation, so we might as well re-use that for the sake of familiarity, simplicity and conciseness. If you're not returning, it's not a continuation.

As an aside, if you think we're dealing with arbitrary callback arguments, rather than only with continuations then that would be a departure from the original issue / feature suggested. There's nothing wrong with that, but the original feature is still the one I am wanting to address, because I don't think coffeescript needs an additional keyword as sugar for calling an implicit callback.

Anyway, people doing asynchronous programming are supposed to know what they're doing. We shouldn't need to impose some implied (and not obvious) conventions based on our sense of good practice.

• People doing asynchronous callbacks are more and more just regular folks who have no choice (it's made for them by the libraries they use, and even the HTML5 spec!). I'd like to make it as much like "regular" programming as is reasonable.
• I agree that my solution is less flexible, but I see that as perfectly acceptable because javascript is already plenty flexible (with anonymous functions). You say there's nothing wrong with being able to do something in more than one way, and in some cases that's true. But should we really be introducing all new syntax for doing something the existing language is perfectly capable of doing (pretty cleanly, I might add) already?

[timbertson]

Oh, and I forgot to mention:

Anyway, people doing asynchronous programming are supposed to know what they're doing. We shouldn't need to impose some implied (and not obvious) conventions based on our sense of good practice.

These conventions are not simply based on our sense of good practice. These conventions are existing conventions in every single piece of asynchronous javascript (and many other languages) code I have ever seen, aside from window.setTimeout() - which is probably just because it's an old API. For that reason, they should be obvious to everyone who's used async functions. And they will be obvious to new users just as soon as we document it ;)

[timbertson]

wow, not sure what happened there. When I posted those two comments, I hadn't seen past http://github.com/jashkenas/coffee-script/issues#issue/287/comment/170651 . weird caching? Thanks matehat for splitting out the yield syntax into another issue - they're not unrelated, but it'd be good to discuss the merits of this and that issue separately.

Anyways, I think it's a good idea to have defer as either a prefix or an argument placeholder as you suggest, that way at least we only add one more keyword. Although as pointed out, it could cause more confusion. Sorry I wasn't on IRC when this was discussed, I've popped in a couple times but nothing much has been happening when I do - maybe because of timezones or something.

One thing I noticed is that the following code is confusing because it "returns" nore than once:

readFileAsync: ->
  for file in (defer fs.readdir "lib/")[1]:
    return (defer fs.readFile file)[1]

in my proposal (replacing return), the actual generated code would include both a call to the callback, followed by a return. That is:

readFileAsync: (_cb) ->
  for file in (defer fs.readdir "lib/")[1]:
    fs.readFile file, (_result) ->
        _db(result[1])
        return null'
    return null;

that is, every return statement is still the end of your function - code past a "return" will still never execute. I comepletely agree that without this, using "return" to mean "call my continutaion" would be a confusing misnomer.

[timbertson]

If you still can't come at the return idea (I may have failed to explain it fully before now, so please read the above and let me know if you still object), I'm okay with it not being in the mainline (yet). It's a small enough change that I'm happy to champion it as distinct from the "defer" continuation issue, as you have started to do with the "yield" syntax. Hell, maybe it'll become obvious that it does or doesn't make sense once people start actually using these features. The continuation-generation is really the hardest (and most useful) part of this whole business, so it's better to get that settled, and then we can see which (if any) of our suggested improvements are still worth adding on top of that functionality.

To sum up (because I have a habit of long and winding replies), I'd love if we could get the defer functionality in master (is this an option, jashkenas?). It sounds like both my return-replacing functionality as well at matehat's yield operator should not go in at this point, but hopefully without either of these we can still start using the defer keyword and see where it takes us :)

Thanks for being patient matehat, I know I've been pretty opinionated on this matter but it's something I really want us to get right, and it's kinda hard to convey ideas with text alone...

[matehat]

gfxmonk: I'm happy with your approval of it moving without either of our divergent ideas for the moment. Sorry for being so opinionated as well :)

Yeah, after some thought, I've come to decide to split the yield issue into its own as I wanted people to consider it in a more general perspective, so to not confuse it altogether with the defer issue. If it makes sense, it will for more than the asynchronous context. Also, I've recently seen it as the way to mix in both of our suggestion. The fact that a small indicator in a function definition specifies its way to return back to the implicit callback could distinguish that from the more usual case you are proposing with the override of return into a invocation.

So, once we get approval of the current state, the next step would be to discuss overriding the return statement, as you pointed out. I think it will be clearer for most at that point.

[jashkenas]

Getting either of your branches merged into master is definitely possible. I just need to see what's going on (as soon as it's on master people will start to use it as their standard CoffeeScript install...)

So the plan should be this: make sure you're passing all of the tests, and then post a comment or a ticket that links to the diff (or Github compare view) of your branch, explain the extent of the changes, and show a couple examples of CoffeeScript that uses defer, along with the corresponding JavaScript output.

If it's ready, we can take it from there.

[matehat]

UPDATES

It's moving quite nicely and the codebase is quite stable. We should discuss some of
ambiguous points in order for things to be clear before hard-coding non obvious behavior. Here is a quick list of those (feel free to post more if you think of any) :

1. defer func(i) for i in list : we expect this to aggregate a list of results taken from a deferred call. It's integrity is everything but easy to garantee. Should we allow this or throw an exception and make the user handle aggregations by hand?
2. if defer func(1); ... else if defer func(2) : there is two possibilities, here. We either wait for all deferred calls to return before evaluating the whole if tree, or we proceed as vanilla js would, not evaluating unnecessary conditions. The latter should be prioritized, but that involves breaking switch cases structure fundamentally.

[timbertson]

great news :)

1). I would hope this can work, in a generic way, along the lines of:

defer func(i) for i in list;

would become, basically:

(function(cb) {
    var _idx = 0;
    var results = []
    var iterate = function() { 
        if(_idx >= list.length) {
            cb(results); return;
        }
        _idx += 1;
        func(i, function(result) {
            results.push(result);
            iterate();
        }
    };
    iterate();
})(__current_continuation);

It's awkward, but that's the beauty of not having to write it manually ;)

Is that possible? I haven't peered too closely at the current implementation of while loops, but I would expect it would have to be somewhat similar (with an iterate() method per-iteration, wrapped in a global "loop" function that could be broken out of at any time by simply calling the continuation)

2. the latter sounds good, how exactly does it break the switch structure? I'm not very familiar with the switch structure, but I though it got transformed early enough to if blocks that we shouldn't even have to worry about it specifically.

[timbertson]

hmm, I just realised that you can't assume an integer index into an arbitrary object. Possibly one would need to construct an actual list of iterable property names up-front, and then iterate through that list (instead of the object directly).

[weepy]

I haven't looked in detail at the, but from a outsiders point of view, : does anyone ever really want to do:

defer func(i) for i in list;

?

I'd argue that edge cases should not be used if they complicate the code base (perhaps people do use it? ). Might be best to force the programmer to do it a consistent way since it must be doing some quite big code munging ?

[timbertson]

It's more common than you might thnk. In my RSS app with an async datastore, I often find myself using exactly this pattern. e.g:

feed_items = defer items.get(id) for id in feed.entry_ids

where the function signature of items.get is get(key, callback)

It can be serviced with a mapAsync function that takes care of the iteration, but it would be cool to have coffeescript deal with it. I don't think it's an edge case, simply an awkward one ;)

[matehat]

gfxmonk: makes sense. BTW, we can distinguish between the integer-array/key-object cases with the use of in or of in the comprehension, that's no problem. Coffeescript already does that.

So, for aggregations, we can guarantee integrity and ordering and it shouldn't be so hard to implement, which I think I'd do through the new utility functions mechanism. Another ambiguous question is whether non-aggregating loops should have that sequential order machinery. Should we always wait for the preceding iteration to return before firing the next one, knowing no aggregation is performed, or just fire them all through a usual for loop? while loops need to be fired sequentially, through some kind of tail recursing formula, since some state needs to be reevaluated in every iteration. That argument does not currently hold in coffeescript in the case of for loops.

[StanAngeloff]

I think you are starting to face some of the concurrency questions the guys designing the Rx library for .NET had to solve not long ago.

[timbertson]

matehat: good point. For loops that don't (obviously) depend on ordering, spawning them off immediately and collating the results as they arrive is probably a good idea. The only real trick there is the bookkeeping to know when you are "done", and also the possibility of iterations depending on previous ones - you obviously can't protect against it 100%, but are there any normal cases where firing off for iterations simultaneously would cause trouble?

[weepy]

The Do lib is interesting : http://github.com/creationix/do
It's probably not directly applicable, but might inspire !

[matehat]

The 2 mentioned libraries provide a way for developers to decide whether they want chained async iterations or parallel async iterations. Both ways can be made so as to keep aggregates topologically correct. The problem is that we can't provide that choice for people to choose themselves. We should make a decision whether we want to decide for them, or we don't do smart (read: restrictive) stuff and let people handle it.

[timbertson]

perhaps it might be best to leave it out for now then (i.e, fail on using a defer in a for loop). An async-aware library is probably a good idea, and keeps us from having to code too many decisions into the language itself.

Are any cases other than the for loop have this problem? As you said, while loops have an explicit ordering requirement. I think if/else branches should be evaluated sequentially, because the short-circuiting behaviour is pretty ingrained in all programming these days - it would be surprising to break that...

[matehat]

The if/else and switch/case should not be problematic and, as you said, we can just make it the way people will expect it to behave. There doesn't seem to be any other issue for now, at least that I can think of.

For the for loop and while loop, as you said, it's pretty logical for the while loop to behave sequentially (i.e. wait for the deferred call to return before reevaluating the condition). We should probably just throw exception when people use it in a for loop, but maybe since the sequential case is taken care of by the while loop, we could make it explicit that the for loop, in async context, is run in parallel. That's just an idea...

[timbertson]

Yeah a parallel for seems fair enough - it's just a default, if you need synchronous calls you can always do so with a while loop (or a library function)

[matehat]

ok, I should push an up-to-date commit soon, when I get time (should be no later than the end of this week)

[jashkenas]

If you don't have enough to read already ... here's an archive of papers on continuation implementation, for the record:

http://library.readscheme.org/page6.html

[weepy]

Syntax idea, instead of

[err, data] = defer get "/", {}
async(x, y)
[err, data] = defer more a, b
setTimeout(defer, 100)
final()    

use a loong arrow

err, data = get "/", {}, -->
async(x, y)
err, data = more a, b, -->
setTimeout -->, 100
final()

I think it neatly encapsulates the idea of 'look to the next line' and also that it's a bit like a function. I've also snuck in an implicit array for multiple assignment ;-)

[jashkenas]

More fun continuation-related reading for the adventurous:

http://okmij.org/ftp/Computation/Continuations.html

[weepy]

hmmm i was just thinking - is there really any benefit to this async stuff ? I mean really the main problem with the JS version is it's ugly and there's lots type type. In CS - it's really just an indentation - which really nicely encapsulates the idea of a async process. Since the methods suggested so far will run up to the end of the function, it means it's not especially useful since you cannot do something after the end of the async declaration?
E.g. cannot convert

get "/", {}, (err, data) -->
  async(x, y)
do_more()

[matehat]

Hi, I'm sorry I haven't been able to make the project progress anyhow lately. I've been quite busy in my company's own projects and since things are going quite well and since I need to rush on a few things there, I won't be able to work much here... So, if anyone wishes to take the lead on that particular branch, my own deferred branch is up-to-date with latest trunk and all tests (except 1 or 2) pass. The special behavior of for and while loops with continuations hasn't been implemented though.

Closing the ticket. Anyone wishing to take the lead can create a part III :D

[timbertson]

Thanks for pushing your work, matehat. I'm away for the weekend, but I'm definitely happy to continue this work. I'll make a part III when I've got something ready for merging, as I think discussion has mostly slowed for now (anyone who disagrees should feel free to continue here or make their own III).

[timbertson]

oh sorry weepy, missed your post.

yes, it will run to the end of the function. That's by design - if you want to do something after the async call, you should just use a callback.

"really just an indentation" is still a big deal when you take this to a logical conclusion of say 8 async calls in a row.

And the biggest boon is when you have an if block, or a loop interacting with callbacks. This is not at all trivial to hand code (I've tried), and is something where compiler help would be greatly appreciated.

Once I've got this feature working, I can definitely provide some examples which hopefully prove its usefulness :)