Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport 48969 to v18.x staging v2 #49183

Conversation

mhdawson
Copy link
Member

@mhdawson mhdawson commented Aug 15, 2023

This is the set of commits needed to backport 48969 to v18.x

The list of PR's addressed in the commits include:

I added a blank Backport-PR-URL: to the commits that required a merge

48969 also required a tweak to the test even though it applied cleanly. That is in the final commit in this PR.

@ShogunPanda I tried to include 48464 a few ways, but the obvious merge to me resulted in many failures. It would be good to have somebody with more context do backport for that one.

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/crypto
  • @nodejs/net

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. v18.x Issues that can be reproduced on v18.x or PRs targeting the v18.x-staging branch. labels Aug 15, 2023
@nodejs-github-bot
Copy link
Collaborator

ShogunPanda and others added 4 commits August 15, 2023 15:16
PR-URL: nodejs#46587
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
PR-URL: nodejs#48189
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
And switch from `google.com` to `nodejs.org`.

PR-URL: nodejs#47029
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Paolo Insogna <[email protected]>
Reviewed-By: Tobias Nießen <[email protected]>
Fixs two issues in `TLSWrap`, one of them is reported in
nodejs#30896.

1. `TLSWrap` has exactly one `StreamListener`, however,
that `StreamListener` can be replaced. We have not been
rigorous enough here: if an active write has not been
finished before the transition, the finish callback of it
will be wrongly fired the successor `StreamListener`.

2. A `TLSWrap` does not allow more than one active write,
as checked in the assertion about current_write in
`TLSWrap::DoWrite()`.

However, when users make use of an existing `tls.TLSSocket`
to establish double TLS, by
either
  tls.connect({socket: tlssock})
or
  tlsServer.emit('connection', tlssock)
we have both of the user provided `tls.TLSSocket`, tlssock and
a brand new created `TLSWrap` writing to the `TLSWrap` bound to
tlssock, which easily violates the constranint because two writers
have no idea of each other.

The design of the fix is:
when a `TLSWrap` is created on top of a user provided socket,
do not send any data to the socket until all existing writes
of the socket are done and ensure registered callbacks of
those writes can be fired.

PR-URL: nodejs#48969
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Paolo Insogna <[email protected]>
@mhdawson mhdawson force-pushed the backport-48969-to-v18.x-staging-v2 branch from 0a79941 to afa5fd7 Compare August 15, 2023 19:27
@mhdawson
Copy link
Member Author

Based on discussion with @ruyadorno squashed last commit into the PR it belonged with and removed the blank Backport-PR-URL: lines

@mhdawson mhdawson added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 15, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 15, 2023
@nodejs-github-bot
Copy link
Collaborator

@mhdawson
Copy link
Member Author

This is likely a real issue. @ShogunPanda before I go looking in the haystack do you have any context that would point me in the direction of which PR might have affected it.

=== release test-dns-ipv6 ===
Path: internet/test-dns-ipv6
test_resolve6
test_reverse_ipv6
node:internal/errors:496
    ErrorCaptureStackTrace(err);
    ^

Error: getHostByAddr EINVAL 2001:4860:4860::8888
    at node:internal/dns/promises:280:14
    at new Promise (<anonymous>)
    at createResolverPromise (node:internal/dns/promises:267:10)
    at ResolverBase.getHostByAddr (node:internal/dns/promises:299:12)
    at test_reverse_ipv6 (/home/runner/work/node/node/test/internet/test-dns-ipv6.js:[72](https://github.com/nodejs/node/actions/runs/5870931290/job/15919021443?pr=49183#step:6:73):36)
    at next (/home/runner/work/node/node/test/internet/test-dns-ipv6.js:22:7)
    at process.processTicksAndRejections (node:internal/process/task_queues:[77](https://github.com/nodejs/node/actions/runs/5870931290/job/15919021443?pr=49183#step:6:78):11) {
  errno: -22,
  code: 'EINVAL',
  syscall: 'getHostByAddr',
  hostname: '2001:4[86](https://github.com/nodejs/node/actions/runs/5870931290/job/15919021443?pr=49183#step:6:87)0:4860::[88](https://github.com/nodejs/node/actions/runs/5870931290/job/15919021443?pr=49183#step:6:89)88'
}

Node.js v18.17.2-pre
Command: out/Release/node /home/runner/work/node/node/test/internet/test-dns-ipv6.js

@

@mhdawson
Copy link
Member Author

Based on

  pull_request:
    types: [opened, synchronize, reopened, ready_for_review]
    paths: [test/internet/**]

in the action it looks like the internet tests are only run when the internet tests are passed. I'm not sure that makes sense as if this PR introduces a failure, the tests were only run because of an unrelated change to a diffferent test in /test/internet. It also makes looking at history hard as you need a PR that changed the internet tests to compare to.

@mhdawson
Copy link
Member Author

mhdawson commented Aug 15, 2023

Running in an unbuntu 22 container just the one test passes:

root@cf00edeb225c:/io.js# tools/test.py test/internet/test-dns-ipv6
[00:00|% 100|+ 1|- 0]: Done

All tests passed.

I'm guessing since IPV6 is not configured for the container?

@mhdawson
Copy link
Member Author

Looking at the address that is being complained about

Error: getHostByAddr EINVAL 2001:4860:4860::8888
    at node:internal/dns/promises:280:14
    at new Promise (<anonymous>)
    at createResolverPromise (node:internal/dns/promises:267:10)
    at ResolverBase.getHostByAddr (node:internal/dns/promises:299:12)

There seems to be an extra ':' between the last two parts. Wondering if that is the issue.

@mhdawson
Copy link
Member Author

The file from which the address comes has the extra colon -

DNS6_SERVER: '2001:4860:4860::8888',

@mhdawson
Copy link
Member Author

Nevermind I guess that the double colons is what is expected and it is a properly formed address based on https://www.ipqualityscore.com/free-ip-lookup-proxy-vpn-test/lookup/2001:4860:4860::88881

@mhdawson
Copy link
Member Author

In my non-containered machine where IPV6 is at least partically configured (but not fully configured which is why it did not use it for initial tests), the test fails even after I remove all of the commits in this PR.

@mhdawson
Copy link
Member Author

I removed all of the commits in the staging branch that are not in the release 18.x branch and still got the same failure on my local machine.

@mhdawson
Copy link
Member Author

PR to see if test failure re-occurs in CI to rule out my environment being the cause of the failures I'm seeing - #49191

@mhdawson
Copy link
Member Author

The test-dns-ipv6 test fails in the same way on the shipped v18.x branch with the only change being adding a blank line to a different test in internet to trigger the tests running - see #49191.

Therefore, I believe that failure is not related to the commits in this PR.

@mhdawson
Copy link
Member Author

mhdawson commented Aug 16, 2023

This was the failure in the initial CI run on PPC, don't believe it is related to the changes in this PR

Test Result (1 failure / +1)
parallel.test-inspector-exception

Going to resume build.

@nodejs-github-bot
Copy link
Collaborator

@mhdawson
Copy link
Member Author

From some initial bisection, the test-dns-ipv16 test starts failing as of 18.17.0

@mhdawson
Copy link
Member Author

I believe my investigation confirms that the failure in test-dns-ipv16 is not related to the content in this PR and should not be a blocker.

I'll open an issue for the pre-existing test-dns-ipv16 but I think @ruyadorno this PR should be ready based on the rest of the CI passing.

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Contributor

@ShogunPanda ShogunPanda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@ShogunPanda
Copy link
Contributor

@mhdawson Thanks for handling all this pain. I'll try to backport #48464 in 18.x next week.

@ruyadorno
Copy link
Member

ruyadorno commented Aug 17, 2023

I believe my investigation confirms that the failure in test-dns-ipv16 is not related to the content in this PR and should not be a blocker.

@mhdawson is there a need to open an issue to track the test-dns-ipv16 failure?

ruyadorno pushed a commit that referenced this pull request Aug 17, 2023
PR-URL: #46587
Backport-PR-URL: #49183
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
ruyadorno pushed a commit that referenced this pull request Aug 17, 2023
PR-URL: #48189
Backport-PR-URL: #49183
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
ruyadorno pushed a commit that referenced this pull request Aug 17, 2023
And switch from `google.com` to `nodejs.org`.

PR-URL: #47029
Backport-PR-URL: #49183
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Paolo Insogna <[email protected]>
Reviewed-By: Tobias Nießen <[email protected]>
@ruyadorno
Copy link
Member

Landed in 0beb5ab...3eeca52

ruyadorno pushed a commit that referenced this pull request Aug 17, 2023
Fixs two issues in `TLSWrap`, one of them is reported in
#30896.

1. `TLSWrap` has exactly one `StreamListener`, however,
that `StreamListener` can be replaced. We have not been
rigorous enough here: if an active write has not been
finished before the transition, the finish callback of it
will be wrongly fired the successor `StreamListener`.

2. A `TLSWrap` does not allow more than one active write,
as checked in the assertion about current_write in
`TLSWrap::DoWrite()`.

However, when users make use of an existing `tls.TLSSocket`
to establish double TLS, by
either
  tls.connect({socket: tlssock})
or
  tlsServer.emit('connection', tlssock)
we have both of the user provided `tls.TLSSocket`, tlssock and
a brand new created `TLSWrap` writing to the `TLSWrap` bound to
tlssock, which easily violates the constranint because two writers
have no idea of each other.

The design of the fix is:
when a `TLSWrap` is created on top of a user provided socket,
do not send any data to the socket until all existing writes
of the socket are done and ensure registered callbacks of
those writes can be fired.

PR-URL: #48969
Backport-PR-URL: #49183
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Paolo Insogna <[email protected]>
@ruyadorno ruyadorno closed this Aug 17, 2023
@mhdawson
Copy link
Member Author

@ruyadorno should have mentioned I opened #49203 to track the test-dns-ipv16 failure

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. v18.x Issues that can be reproduced on v18.x or PRs targeting the v18.x-staging branch.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants