Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

async-resource-vs-destroy benchmark is broken locally #33591

Closed
addaleax opened this issue May 27, 2020 · 12 comments
Closed

async-resource-vs-destroy benchmark is broken locally #33591

addaleax opened this issue May 27, 2020 · 12 comments
Labels
benchmark Issues and PRs related to the benchmark subsystem.

Comments

@addaleax
Copy link
Member

(Extracted from #33575 (comment))

What steps will reproduce the bug?

$ ./node benchmark/async_hooks/async-resource-vs-destroy.js 
Error [ERR_STREAM_DESTROYED]: Cannot call write after a stream was destroyed
    at write_ (_http_outgoing.js:689:11)
    at ServerResponse.end (_http_outgoing.js:783:5)
    at /home/addaleax/src/node/master/benchmark/async_hooks/async-resource-vs-destroy.js:152:13
    at FSReqCallback.readFileAfterClose [as oncomplete] (internal/fs/read_file_context.js:63:3)
Emitted 'error' event on ServerResponse instance at:
    at emitErrorNt (_http_outgoing.js:666:43)
    at processTicksAndRejections (internal/process/task_queues.js:85:21) {
  code: 'ERR_STREAM_DESTROYED'
}

Bisecting pointed to 28e6626 (#33131).

/cc @ronag

@addaleax addaleax added the benchmark Issues and PRs related to the benchmark subsystem. label May 27, 2020
@ronag
Copy link
Member

ronag commented May 27, 2020

Benchmark is broken in that it does not wait for the requests to finish before closing the server, alternatively does not catch the error caused by ending a closed response.

Looks to me like the benchmark does not wait for all the requests to finish. Hence, once it's dispatched all the requests it closes the server and the response is closed, once the setTimeout fires it will try to end() a closed response and errors.

Is it possible to make close async and have it wait a bit before closing the server? Or just to be quick:

@addaleax
Copy link
Member Author

@ronag Oh, yeah, sorry, should have commented that the suggestion you posted didn’t work, even with large timeouts. :/

I also think closing the server handle itself shouldn’t really affect the individual streams?

@ronag
Copy link
Member

ronag commented May 27, 2020

I also think closing the server handle itself shouldn’t really affect the individual streams?

Closing the server could/should close the response? The problem here is that the response has been closed. It worked before the commit you refer to because it incorrectly ignored this state.

What platform are you on? I can't reproduce it.

@addaleax
Copy link
Member Author

I also think closing the server handle itself shouldn’t really affect the individual streams?

Closing the server could/should close the response?

… that sounds like a huge bug? But I’m also fairly sure that that’s not what happens.

The problem here is that the response has been closed. It worked before the commit you refer to because it incorrectly ignored this state.

Hm okay, in that case I guess the problem is just being exposed by it?

What platform are you on? I can't reproduce it.

x64 Linux, Ubuntu 18.04

@ronag
Copy link
Member

ronag commented May 27, 2020

… that sounds like a huge bug? But I’m also fairly sure that that’s not what happens.

Given the error you encounter I suspect it is hitting this 28e6626#diff-feaf3339998a19f0baf3f82414762c22R208-R211 or 28e6626#diff-feaf3339998a19f0baf3f82414762c22R678

Hm okay, in that case I guess the problem is just being exposed by it?

That is my belief.

What platform are you on? I can't reproduce it.

Cool. I'll see if I can reproduce by running it inside docker.

@addaleax
Copy link
Member Author

@ronag onServerResponseClose() and resOnFinish() are per-request/per-socket, not per server, though. The server closing shouldn’t have an effect beyond new connections no longer being accepted.

@ronag
Copy link
Member

ronag commented May 27, 2020

The server closing shouldn’t have an effect beyond new connections no longer being accepted.

From what I can see the server _handle will be closed (i.e. _handle.close()). What that actually does in practice I don't know and will have to defer to you.

For whatever reason, it seems for you the response and/or socket is prematurely closed which is why you are getting an error.

addaleax added a commit to addaleax/node that referenced this issue May 29, 2020
In the benchmark, because it performs asynchronous operations before
writing its HTTP replies, the underlying socket can be closed by the
peer before the response is written. Since 28e6626, that means
that attempting to `.end()` the HTTP response results in an uncaught
exception, breaking the benchmark.

Fix that by checking whether the response object has been destroyed
or not before attempting to call `.end()`.

nodejs#33591
@addaleax
Copy link
Member Author

@ronag That makes sense, but I think it also means #33131 should have been semver-major.

The reason why the benchmark is failing is that the socket being closed, it because httpAllowHalfOpen isn’t set and thus when the client closes the readable side of the underlying socket, the writable side will be .end()ed as well. I’ve opened #33642 to address that.

Tbh, it feels wrong to me that you can’t write code like await someAsyncThing(); res.end('done');. It’s a pitfall that I probably wouldn’t think of when writing an HTTP server.

(I get that your PR aligned the behavior with the one for streams, but for most Duplex streams, allowHalfOpen is true, so it’s a lot harder to get this wrong there usually.)

Aside, should #33131 have updated .writable as well, in addition to .destroyed? Right now fixing the benchmark means that if (!res.writable) res.end(); won’t work.

@ronag
Copy link
Member

ronag commented May 29, 2020

@ronag That makes sense, but I think it also means #33131 should have been semver-major.

Agreed. Was not aware of this side effect at the time.

Aside, should #33131 have updated .writable as well, in addition to .destroyed?

I agree writable should also be updated.

@ronag
Copy link
Member

ronag commented May 29, 2020

#33643

@ronag
Copy link
Member

ronag commented May 29, 2020

Tbh, it feels wrong to me that you can’t write code like await someAsyncThing(); res.end('done');. It’s a pitfall that I probably wouldn’t think of when writing an HTTP server.

I think this can be fixed in a way which is consistent with the semantics of streams. I'll add this to my list.

@ronag
Copy link
Member

ronag commented May 30, 2020

Aside, should #33131 have updated .writable as well, in addition to .destroyed?

#33655

addaleax added a commit that referenced this issue Jun 6, 2020
In the benchmark, because it performs asynchronous operations before
writing its HTTP replies, the underlying socket can be closed by the
peer before the response is written. Since 28e6626, that means
that attempting to `.end()` the HTTP response results in an uncaught
exception, breaking the benchmark.

Fix that by checking whether the response object has been destroyed
or not before attempting to call `.end()`.

#33591

PR-URL: #33642
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Juan José Arboleda <[email protected]>
codebytere pushed a commit that referenced this issue Jun 18, 2020
In the benchmark, because it performs asynchronous operations before
writing its HTTP replies, the underlying socket can be closed by the
peer before the response is written. Since 28e6626, that means
that attempting to `.end()` the HTTP response results in an uncaught
exception, breaking the benchmark.

Fix that by checking whether the response object has been destroyed
or not before attempting to call `.end()`.

#33591

PR-URL: #33642
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Juan José Arboleda <[email protected]>
ronag added a commit to nxtedition/node that referenced this issue Jun 20, 2020
ronag added a commit that referenced this issue Jun 21, 2020
Refs: #33591

PR-URL: #33654
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
codebytere pushed a commit that referenced this issue Jun 30, 2020
In the benchmark, because it performs asynchronous operations before
writing its HTTP replies, the underlying socket can be closed by the
peer before the response is written. Since 28e6626, that means
that attempting to `.end()` the HTTP response results in an uncaught
exception, breaking the benchmark.

Fix that by checking whether the response object has been destroyed
or not before attempting to call `.end()`.

#33591

PR-URL: #33642
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Juan José Arboleda <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
benchmark Issues and PRs related to the benchmark subsystem.
Projects
None yet
Development

No branches or pull requests

3 participants