Add support for AsyncResource

[Sebmaster]

I kinda hijacked the domain architecture to implement AsyncResource, but it makes the changes very consistent and keeps the context stuff transparent to the rest of the code.

Fixes #1403.

[Sebmaster]

I sadly can't quite figure out why this is breaking some tests.

A simple old = cb; cb = function(){return old.apply(this, arguments);} in the bindContext function breaks everything. Something's special about this original function object in some cases.

Is there like some hidden property bluebird attaches to them?

[Sebmaster]

Alright, found the specific compare to reflectHandler and fixed that.

There's one thing which is kinda "unimplemented" which is emitDestroy, but I don't think I understand the lifecycle of the objects enough to determine where we'd need to put that.

[benjamingr]

CC @addaleax

[Sebmaster]

Should drop the domain null checks here, they never do anything now.

[jkrems]

Is this stripped somehow in the browser build? E.g. will this expression make it into js/browser/bluebird.js? If so, it might cause problems while bundling in apps using bluebird. Maybe we'd want to put this logic into a completely separate file that's "guarded" by a file-level browser override so it's excluded from bundling? E.g. something like this:

// src/getContext.browser.js, gets loaded directly in browser environments
module.exports = function getContext() { return {}; }

// src/getContext.js, gets loaded in not-definitely-browser environments
var getFallbackContext = require('./getContext.browser.js');
var getContext;
// [...]

// src/promise.js
var getContext = require('./getContext');

[Sebmaster]

Yes, see the exclude in the browserify instructions, require will just return an empty object in that case (and even that won't be executed due to the isNode check before).

[jkrems]

Ah, gotcha. It compiles to _dereq_('async_hooks') which just never gets executed. 👍

[benjamingr]

Another attempt to solicit feedback from @addaleax about this

[addaleax]

@benjamingr Sorry, the reason I haven’t commented so far is: The async_hooks API sucks for these use cases at this point, and we’re going to fix that, but that hasn’t happened yet. Every resource needs a destroy hook, otherwise everything is just a memory leak, and for JS resources that would be done through GC tracking which isn’t there yet.

So, uh, I have this on my radar… if anybody wants to make the necessary improvements inside of Node themselves, please go for it, and know that I’m available (same handle on twitter/irc) for any questions, help, guidance, whatever :)

[Sebmaster]

@addaleax nodejs/node#16153 would be the ticket for that, I assume?

@benjamingr Do you want to leave this open until the changes in node land?

[addaleax]

@Sebmaster Yeah, thanks for looking that up. 😄 I’ve labeled that as a good first issue.

Also, I guess it doesn’t hurt to leave this open or even merge this without GC support – async_hooks is experimental for a reason, and this is just plainly a bug in core.

[addaleax]

As for the actual diff here, I think there might be too much bluebird-specific stuff in it for me to grok easily?

[Sebmaster]

or even merge this without GC support – async_hooks is experimental for a reason, and this is just plainly a bug in core.

In that case we'd need at least a config option to enable AsyncResource, I think (and disable by default). Can't start leaking asyncIds on a random minor bluebird update.

[benjamingr]

Thanks for chiming in @addaleax

I'd think we'd rather explicitly not support async_hooks than give users something that can leak memory at this point.

I'm ok with leaving this open until this gets fixed in core

[Sebmaster]

New node release is out and has support for automatic destroy call on GC, so limiting async_hooks support to 9.3.0 and up. Should be ready now.

[AndreasMadsen]

You should consider calling it bluebird.Promise to not confuse it with the builtin PROMISE.

[AndreasMadsen]

@addaleax @Sebmaster master do we want to expose an emitPromiseResolve for the promiseResolve hook?

[suguru03]

Excuse me for cutting in.
I realized a performance issue.
I tested ./bench doxbee, it was approximately two times slower than before.
I just started learning async_hooks and AsyncResource, so I'm not sure if it is required or not. If it is not, I would like to suggest using Bluebird.config and the default is false.

file                                       time(ms)  memory(MB)
callbacks-baseline.js                           214       27.03
promises-suguru03-aigle.js                      304       48.38
callbacks-suguru03-neo-async-waterfall.js       312       35.63
promises-bluebird-generator.js                  328       37.34
promises-cujojs-when.js                         511       66.89
promises-tildeio-rsvp.js                        528       81.32
promises-lvivski-davy.js                        593       91.18
promises-then-promise.js                        597       72.99
promises-native-async-await.js                  631      108.68
generators-tj-co.js                             660       80.39
promises-bluebird.js                            681       84.18
promises-ecmascript6-native.js                  696      104.88
callbacks-caolan-async-waterfall.js             788       80.94
promises-dfilatov-vow.js                        904      134.86
promises-obvious-kew.js                        1205      115.54
promises-calvinmetcalf-lie.js                  1228      116.02
streamline-generators.js                       1281       82.92
promises-medikoo-deferred.js                   1392       96.83
observables-pozadi-kefir.js                    1669      143.13
streamline-callbacks.js                        2010      105.96
observables-Reactive-Extensions-RxJS.js        3293      187.86
promises-kriskowal-q.js                       10433      305.79
observables-caolan-highland.js                14689      434.57
observables-baconjs-bacon.js.js               30249      751.06

[AndreasMadsen]

@Sebmaster @suguru03 This makes sense. AsyncResource listens on the garbage collector which is a rather slow operation.

@addaleax We should consider only calling registerDestroyHook if there are actual listeners, that should give a big speedup in the default case. I'm not a huge fan of this, as it makes late enabling of async_hooks a little odd. Although, we kinda have the same situation for native promises – also not a fan of that :p

[Sebmaster]

Alright, fixed all the feedback, added a config option for enabling AsyncResource.

@AndreasMadsen Sure we could emit the event, however it seems like looping through the proper AsyncResource is kinda difficult right now, since it's not really promise-instance-specific, but just some param which gets weaved through the API.

I think there might be a behaviour difference here now. Native promises generate a single AsyncResource for the whole promise lifecycle(?), while this current implementation creates one AsyncResource per .then call, to use the domain infrastructure. Maybe that should change...

[kjin]

Hi @Sebmaster -- any update on this PR? It's been almost a year since it has first opened, so I'm sure some of the concerns about it landing (with regards to the async_hooks API) might have been alleviated in the meantime. On the other hand, if there are still concrete things that still need to be done in Node core, maybe it would be good to mention what those things are now.

[miensol]

I'd love this PR to land in bluebird as well 👍 It would enable transparently passing tracing contexts in libraries that do not support it directly but rely on bluebird. This would be a huge help in some cases where one needs to parse and correlate e.g. log messages.

[p-bakker]

Just had to completely toss out Bluebird from our codebase and replace it with native Promises (combined with https://github.com/asfktz/Awaity.js) due to Bluebird not playing nice with other libraries that are based on async_hooks

[Sebmaster]

So I just restarted work on this to finally get this in.

Test suite passes, I think the performance issues have been fixed in core (was not able to reproduce the 10x doxbee bench slowdown with v10.9.0 with hooks in Bluebird enabled - I think because no hooks are listening) and I think the native vs bluebird Promise behaviour should not be an issue; it's acceptable with regards to the async_hooks spec.

@benjamingr Can I get some concrete maintainer guidance on what you'd like to see to merge this?

[benjamingr]

@Sebmaster this would need a review from someone who is very familiar with AsyncResource and someone who is a maintainer. I can do the bluebird code part and maybe @addaleax or @AndreasMadsen has time to take a look at the AsyncResource stuff?

Then I need to ping @petkaantonov IRL to do a release :D

[petkaantonov]

continues here #1597