http: fix request with option timeout and agent #21204

killagu · 2018-06-08T04:43:46Z

When request with both timeout and agent, timeout not
work. This patch will fix it, socket timeout will set
to request timeout before socket is connected, and
socket timeout will reset to agent timeout after
response end.

Update agent doc, add timeout option.

Fixes: #21185

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

When request with both timeout and agent, timeout not work. This patch will fix it, socket timeout will set to request timeout before socket is connected, and socket timeout will reset to agent timeout after response end. Update agent doc, add timeout option. Fixes: nodejs#21185

XadillaX

👍

vsemozhetbyt · 2018-06-09T11:09:16Z

CI: https://ci.nodejs.org/job/node-test-pull-request/15362/

Trott

Relevant test failure in CI.

https://ci.nodejs.org/job/node-test-commit-arm/16725/nodes=debian8-docker-armv7/console

07:23:39 not ok 762 parallel/test-http-client-timeout-option-with-agent
07:23:39   ---
07:23:39   duration_ms: 2.837
07:23:39   severity: fail
07:23:39   exitcode: 1
07:23:39   stack: |-
07:23:39     assert.js:270
07:23:39         throw err;
07:23:39         ^
07:23:39     
07:23:39     AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
07:23:39     
07:23:39       assert(Math.abs(duration - HTTP_CLIENT_TIMEOUT) < 20)
07:23:39     
07:23:39         at ClientRequest.req.on.common.mustCall (/home/iojs/build/workspace/node-test-commit-arm/nodes/debian8-docker-armv7/test/parallel/test-http-client-timeout-option-with-agent.js:40:5)
07:23:39         at ClientRequest.<anonymous> (/home/iojs/build/workspace/node-test-commit-arm/nodes/debian8-docker-armv7/test/common/index.js:451:15)
07:23:39         at ClientRequest.emit (events.js:182:13)
07:23:39         at Socket.emitRequestTimeout (_http_client.js:660:40)
07:23:39         at Object.onceWrapper (events.js:273:13)
07:23:39         at Socket.emit (events.js:182:13)
07:23:39         at Socket._onTimeout (net.js:445:8)
07:23:39         at ontimeout (timers.js:471:11)
07:23:39         at tryOnTimeout (timers.js:341:5)
07:23:39         at listOnTimeout (timers.js:315:5)
07:23:39   ...

https://ci.nodejs.org/job/node-test-commit-arm/16725/nodes=ubuntu1604-arm64/console

07:15:19 not ok 794 parallel/test-http-client-timeout-option-with-agent
07:15:19   ---
07:15:19   duration_ms: 2.909
07:15:19   severity: fail
07:15:19   exitcode: 1
07:15:19   stack: |-
07:15:19     assert.js:270
07:15:19         throw err;
07:15:19         ^
07:15:19     
07:15:19     AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
07:15:19     
07:15:19       assert(Math.abs(duration - HTTP_CLIENT_TIMEOUT) < 20)
07:15:19     
07:15:19         at ClientRequest.req.on.common.mustCall (/home/iojs/build/workspace/node-test-commit-arm/nodes/ubuntu1604-arm64/test/parallel/test-http-client-timeout-option-with-agent.js:40:5)
07:15:19         at ClientRequest.<anonymous> (/home/iojs/build/workspace/node-test-commit-arm/nodes/ubuntu1604-arm64/test/common/index.js:451:15)
07:15:19         at ClientRequest.emit (events.js:182:13)
07:15:19         at Socket.emitRequestTimeout (_http_client.js:660:40)
07:15:19         at Object.onceWrapper (events.js:273:13)
07:15:19         at Socket.emit (events.js:182:13)
07:15:19         at Socket._onTimeout (net.js:445:8)
07:15:19         at ontimeout (timers.js:471:11)
07:15:19         at tryOnTimeout (timers.js:341:5)
07:15:19         at listOnTimeout (timers.js:315:5)
07:15:19   ...

Looks like the test contains a somewhat arbitrary setTimeout(). Ideally, the test should be rewritten without it. But a minimal workaround may be to move the test out of the parallel directory and into the sequential directory.

Trott · 2018-06-09T12:00:59Z

You can probably replicate the test failures locally by running multiple copies of the test simultaneously:

tools/test.py -j 50 --repeat 100 parallel/test-http-client-timeout-option-with-agent

Trott · 2018-06-09T12:02:28Z

test/parallel/test-http-client-timeout-option-with-agent.js

@@ -0,0 +1,48 @@
+'use strict';


Would you be willing to add a comment here explaining what this is testing?

Trott · 2018-06-09T12:03:42Z

test/parallel/test-http-client-timeout-option-with-agent.js

+  req.on('timeout', common.mustCall(() => {
+    timeout_events += 1;
+    const duration = Date.now() - start;
+    assert((Math.abs(duration - HTTP_CLIENT_TIMEOUT) < 20));


Regarding the request for a comment above, it would be useful in particular to know if what this is all about. 20 seems like a magic number and this sort of fuzzy duration check is a bit of a red flag in tests too. Curious if there's another way to do it that still manages to check whatever the test is testing for.

Trott · 2018-06-09T12:05:31Z

@nodejs/testing @nodejs/http

killagu · 2018-06-11T02:14:38Z

@Trott Thank you for the response. The test comment have been added, and the magic number have been removed. Please review it again.

tests seem reliable now, unblocking

Trott · 2018-06-11T03:28:46Z

CI: https://ci.nodejs.org/job/node-test-pull-request/15381/

XadillaX · 2018-06-13T04:42:37Z

New CI: https://ci.nodejs.org/job/node-test-pull-request/15431/

ryzokuken · 2018-06-13T20:34:44Z

@Trott can this be landed now?

Trott · 2018-06-14T05:24:43Z

@Trott can this be landed now?

Theoretically, yes, but in practice, I try to ping for more reviews rather than land if something only has one review. Especially a semver-minor (because we'll be stuck supporting a new option forever). So, in that spirit:

/ping @nodejs/http @nodejs/streams @nodejs/tsc

mcollina

LGTM

targos · 2018-06-22T13:16:15Z

New CI: https://ci.nodejs.org/job/node-test-pull-request/15566/

mcollina · 2018-07-05T15:21:51Z

CI: https://ci.nodejs.org/job/node-test-pull-request/15740/

addaleax · 2018-07-13T16:49:35Z

Landed in 949e885, thanks for the PR! 🎉

When request with both timeout and agent, timeout not work. This patch will fix it, socket timeout will set to request timeout before socket is connected, and socket timeout will reset to agent timeout after response end. Fixes: #21185 PR-URL: #21204 Reviewed-By: Khaidi Chu <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Ujjwal Sharma <[email protected]>

Notable changes: * console: * The `console.timeLog()` method has been implemented. (#21312) * deps: * Upgrade to libuv 1.22.0. (#21731) * Upgrade to ICU 62.1 (Unicode 11, CLDR 33.1). (#21728) * http: * Added support for passing both `timeout` and `agent` options to `http.request`. (#21204) * napi: * Added experimental support for functions dealing with bigint numbers. (#21226) * process: * The `process.hrtime.bigint()` method has been implemented. (#21256) * Added the `--title` command line argument to set the process title on startup. (#21477) * trace_events: * Added process_name metadata. (#21477)

Notable changes: * console: * The `console.timeLog()` method has been implemented. (#21312) * deps: * Upgrade to libuv 1.22.0. (#21731) * Upgrade to ICU 62.1 (Unicode 11, CLDR 33.1). (#21728) * http: * Added support for passing both `timeout` and `agent` options to `http.request`. (#21204) * napi: * Added experimental support for functions dealing with bigint numbers. (#21226) * process: * The `process.hrtime.bigint()` method has been implemented. (#21256) * Added the `--title` command line argument to set the process title on startup. (#21477) * trace_events: * Added process_name metadata. (#21477) * Added new collaborators * codebytere - Shelley Vohr PR-URL: #21851

Notable changes: * console: * The `console.timeLog()` method has been implemented. (#21312) * deps: * Upgrade to libuv 1.22.0. (#21731) * Upgrade to ICU 62.1 (Unicode 11, CLDR 33.1). (#21728) * http: * Added support for passing both `timeout` and `agent` options to `http.request`. (#21204) * inspector: * Expose the original console API in `require('inspector').console`. (#21659) * napi: * Added experimental support for functions dealing with bigint numbers. (#21226) * process: * The `process.hrtime.bigint()` method has been implemented. (#21256) * Added the `--title` command line argument to set the process title on startup. (#21477) * trace_events: * Added process_name metadata. (#21477) * Added new collaborators * codebytere - Shelley Vohr PR-URL: #21851

fengmk2 · 2018-10-20T16:20:22Z

Should this bugfix backport to Node 8?

mcollina · 2018-10-21T09:06:56Z

I would prefer not to, it caused some
breakage.

nodejs-github-bot added http Issues or PRs related to the http subsystem. net Issues and PRs related to the net subsystem. labels Jun 8, 2018

killagu force-pushed the fix-timeout branch from 5592223 to 33e62a1 Compare June 8, 2018 05:40

XadillaX approved these changes Jun 8, 2018

View reviewed changes

Trott previously requested changes Jun 9, 2018

View reviewed changes

Trott reviewed Jun 9, 2018

View reviewed changes

jasnell requested a review from mcollina June 10, 2018 20:46

update tests

4add8f6

Trott added 5 commits June 10, 2018 20:23

minor edit to doc text

7ab62fb

fixup! minor edit to comment text

9773531

squash! minor edit to comment

3b30906

squash! minor edits to comments

1cf617e

squash! minor edits to comments

2c5f0c0

ryzokuken added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Jun 13, 2018

Trott added the semver-minor PRs that contain new features and should be released in the next minor version. label Jun 14, 2018

mcollina approved these changes Jun 14, 2018

View reviewed changes

jasnell approved these changes Jun 14, 2018

View reviewed changes

ryzokuken approved these changes Jun 22, 2018

View reviewed changes

addaleax closed this Jul 13, 2018

targos removed the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Jul 17, 2018

targos mentioned this pull request Jul 17, 2018

v10.7.0 proposal #21851

Merged

ChALkeR mentioned this pull request Jul 20, 2018

10.7.0 broke the Yarn tests #21907

Closed

griffinmyers mentioned this pull request Aug 3, 2018

Set socket connecting state to always be false nock/nock#1182

Merged

kobelb mentioned this pull request Jul 29, 2020

doc: add net.Socket#timeout #34543

Closed

2 tasks

keithchew mentioned this pull request Jul 18, 2024

net.Socket has no setConnectionTimeout(timeout) method #5757

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

http: fix request with option timeout and agent #21204

http: fix request with option timeout and agent #21204

killagu commented Jun 8, 2018

XadillaX left a comment

vsemozhetbyt commented Jun 9, 2018

Trott left a comment

Trott commented Jun 9, 2018

Trott Jun 9, 2018

Trott Jun 9, 2018

Trott commented Jun 9, 2018

killagu commented Jun 11, 2018 •

edited

Loading

Trott commented Jun 11, 2018

XadillaX commented Jun 13, 2018

ryzokuken commented Jun 13, 2018

Trott commented Jun 14, 2018

mcollina left a comment

targos commented Jun 22, 2018

mcollina commented Jul 5, 2018

addaleax commented Jul 13, 2018

fengmk2 commented Oct 20, 2018

mcollina commented Oct 21, 2018

http: fix request with option timeout and agent #21204

http: fix request with option timeout and agent #21204

Conversation

killagu commented Jun 8, 2018

Checklist

XadillaX left a comment

Choose a reason for hiding this comment

vsemozhetbyt commented Jun 9, 2018

Trott left a comment

Choose a reason for hiding this comment

Trott commented Jun 9, 2018

Trott Jun 9, 2018

Choose a reason for hiding this comment

Trott Jun 9, 2018

Choose a reason for hiding this comment

Trott commented Jun 9, 2018

killagu commented Jun 11, 2018 • edited Loading

Trott commented Jun 11, 2018

XadillaX commented Jun 13, 2018

ryzokuken commented Jun 13, 2018

Trott commented Jun 14, 2018

mcollina left a comment

Choose a reason for hiding this comment

targos commented Jun 22, 2018

mcollina commented Jul 5, 2018

addaleax commented Jul 13, 2018

fengmk2 commented Oct 20, 2018

mcollina commented Oct 21, 2018

killagu commented Jun 11, 2018 •

edited

Loading