src: quic #44325

jasnell · 2022-08-21T03:50:18Z

Updated alternatiive to #38233 ...

Significant changes here. This focuses entirely on the internal API surface and does not expose any public API yet. That is intentional to allow us to work through this and was agreed on at the last collaborators summit.

The tests here are woefully incomplete and will be evolved over time.

It's a very big PR. If folks want help getting started on how to review I'm happy to walk through it. The prior PR bit rotted badly and had to be nearly completely redone.

Expect bugs. This will continue to be a work in progress for a bit.

/cc @mcollina @splitice

mcollina · 2022-08-21T07:34:53Z

I think we should better off in tagging this "dont-land" on all lines. Wdyt?

jasnell · 2022-08-21T14:00:28Z

Yeah likely a good idea

jasnell · 2022-08-21T14:02:38Z

To-do list:

Investigate build failures on various CI platforms
- Build failure on macos
- other build failure on Windows
- ngtcp2 build failure on Windows
Socketaddress bug
Keep expanding tests to identify where the bugs are
Fix those bugs
Fix annoying format-cpp failures
ASAN failure

splitice · 2022-08-22T07:10:18Z

I'll likely get involved once there is a public API. I'm no internals expert (more of an internals breaker)

lib/internal/socketaddress.js

lib/internal/event_target.js

src/node_http_common.h

tniessen · 2022-09-12T20:57:17Z

I've opened a few small PRs with commits from this PR (or adaptations thereof) in the hope that those can land, potentially making it easier to review this PR.

src: avoid copy when creating Blob #44616 replaces d5df573 (#44325) and f6b6cb7 (#44325).
lib,test: fix bug in InternalSocketAddress #44618 replaces 24ef1f8 (#44325) (and adds a test case).
deps: update to ngtcp2 0.8.1 and nghttp3 0.7.0 #44622 replaces 8efe1dc (#44325) and d1a397b (#44325) (and updates to the latest tagged versions).

The first two should be able to land without requiring any changes here. The third will conflict with the commits here and will also require a small change due to an API change in ngtcp2 (see #44622 (comment)).

Note that these changes only affect existing code in Node.js and do not add any features. Additions should remain in this PR.

src/crypto/crypto_common.cc

splitice · 2022-10-14T01:52:39Z

@jasnell how close is this to getting a JS API?

I have the possibility of some time that can be allocated in late november / early december where I can do a similar process to the previous pull request on a JS API. Probably finding a few bugs along the way.

Adds an optional third argument to `defineEventHandler()` to specify an event name that is separate from the name used for the property. This will be used, for instance, by the quic implementation to support generating the `on...` events where the event name itself is hyphenated. For instance `defineEventHandler(target, 'streamreset', 'stream-reset')` Separated out from nodejs#44325 Signed-off-by: James M Snell <[email protected]>

jasnell · 2022-10-16T22:52:03Z

PR #45032 replaces commit 9156797 in this PR.
PR #45033 replaces commit 4fa7e02 in this PR.
PR #45045 replaces commit 9c48c17 in this PR.

Separated out to help continue reducing the size here.

jasnell · 2022-10-16T23:10:17Z

@tniessen ... smaller now :-)

GeoffreyBooth · 2022-12-05T17:04:22Z

I reviewed the src/crypto changes, not the other parts.

So between your review and @Qard's, does that cover most or all of it?

Along those lines, how much of what's unreviewed is in new versus old areas? Like if the bulk of this PR is for a new API, and everyone not using that API is unaffected, then that entire new API could pretty much land with cursory review (since it's low risk) and all we need to focus reviewing effort on is the parts that overlap with existing/stable APIs.

ronag · 2022-12-05T17:14:13Z

Any chance to split the javascript file into multiple files?

Qard · 2022-12-05T18:13:32Z

I'm still not done reading through all of it. Like I said, it's taking a lot of time. 😅

bricss · 2022-12-05T22:08:33Z

lib/internal/quic/quic.js

+
+    const opts = {
+      ...options,
+      depth: options.depth == null ? null : options.depth - 1


Perhaps, it might be better to check for NaN here, before arithmetics

bricss · 2022-12-05T22:13:21Z

lib/internal/quic/quic.js

+      alpn,
+      servername,
+      preferredAddressStrategy,
+      undefined, // Connection ID Factory, not currently used.


bricss · 2022-12-05T22:27:56Z

Alrighty, codebase all LGTM @jasnell you da men 🦾
Fortunately/unfortunately, I don't have much wisdom in catching C++ memory leaks
Watta about merging this as it is with alpha-beta-gamma experimental flags all over the place aka docs
And applying gradual enhancement approach to all of this? 🤔

splitice · 2022-12-05T22:43:29Z

I've got the ability to run my tests that caught memory leaks last time again once there's a js API.

I'm no native expert. I only know enough native js to break things or make small fixes (currently).

bnoordhuis · 2022-12-06T10:42:53Z

So between your review and @Qard's, does that cover most or all of it?

I haven't looked at src/quic/crypto.* (1500 lines) at all yet. Maybe someone else has?

Qard

I've left some notes about the JS part.

Also, much of the tests seem to not really assert anything and just validate that it doesn't crash. Perhaps in place of a proper public API we could have more in-depth tests as examples of how all this fits together? I assume when a public API is made we could probably delete much of the internal tests, so could also use that to provide code examples with lots of comments explaining the use.

Qard · 2022-12-05T19:00:06Z

lib/internal/quic/quic.js

+  sendInfoHeaders(headers) {
+    if (this.#inner === undefined)
+      throw new ERR_INVALID_STATE('The stream has been closed.');
+    this.#inner.sendHEaders(QUIC_HEADERS_KIND_INFO,


Suggested change

this.#inner.sendHEaders(QUIC_HEADERS_KIND_INFO,

this.#inner.sendHeaders(QUIC_HEADERS_KIND_INFO,

Qard · 2022-12-05T19:00:19Z

lib/internal/quic/quic.js

+    } = options;
+    validateBoolean(terminal, 'options.terminal');
+
+    this.#inner.sendHEaders(QUIC_HEADERS_KIND_INITIAL,


Suggested change

this.#inner.sendHEaders(QUIC_HEADERS_KIND_INITIAL,

this.#inner.sendHeaders(QUIC_HEADERS_KIND_INITIAL,

Qard · 2022-12-05T19:06:24Z

lib/internal/quic/quic.js

+    this.#endpoint = endpoint;
+    this.#inner[kOwner] = this;
+    this.#stats = new SessionStats(this.#inner.stats);
+    this.#state = new SessionState(this.#inner.state);


Not really a fan of having two so similarly-named things so close together. Very easy to misread/mistype. Might be a good idea to rename one of these. 🤔

Qard · 2022-12-05T19:13:30Z

lib/internal/quic/quic.js

+    this.#inner = options[kCreateInstance]();
+    this.#inner[kOwner] = this;
+    this.#state = new EndpointState(this.#inner.state);
+    this.#stats = new EndpointStats(this.#inner.stats);


Again, very similarly named things in the same place is easily misread.

jasnell · 2022-12-19T19:50:43Z

The changes to src/crypto have been moved out to #45912 to allow for incremental/independent review.

jasnell · 2023-03-27T00:34:11Z

I have pulled off more of the implementation bits into a separate PR here: #47263

The plan is to keep peeling pieces off into separate smaller PRs as much as possible.

Faolain · 2024-07-17T14:05:41Z

I have pulled off more of the implementation bits into a separate PR here: #47263

The plan is to keep peeling pieces off into separate smaller PRs as much as possible.

Do we know what else is needed here?

splitice

Most of my commits are very out of sync with your new work and would require re-testing however the ones highlighted in this review look to be matching with the current source.

There is also some comits related to keeping objects alive that look like they should still apply but would require testing.

We saw lots of crashes if the endpoint was allowed to close while streams were bound etc in the old implementation and generally resolved this by adding references between objects as required (e.g that the outbound source requires its backing packet)

I can go about validating these when theres a JS API I bet.

splitice · 2024-08-01T02:10:48Z

src/quic/session.cc

+
+    pos = nullptr;
+
+    if (++packetSendCount == maxPacketCount) {


Is this how you plan to deal with the stream blocked = infinite resends?

I did it a different way in my fork. In my fork I ensured that a stream that returned NGTCP2_ERR_STREAM_DATA_BLOCKED would not be requeued within this call (HalleyAssist@a2c6fbe)

splitice · 2024-08-01T02:14:32Z

src/quic/session.cc

+    DEBUG_ARGS(session_, "Stream Data: %s", stream_data);
+
+    if (!packet) {
+      packet = CreateStreamDataPacket();


This packet needs to retain the outbound source or node may crash

See HalleyAssist@60d9bbc for how I did it

splitice · 2024-08-01T02:16:30Z

src/quic/session.cc

+void Session::SendConnectionClose() {
+  CHECK(!NgCallbackScope::InNgCallbackScope(*this));
+
+  if (is_destroyed() || is_in_draining_period() || state_->silent_close) return;


This fix is the same as I found in HalleyAssist@f505f7a 👍

splitice · 2024-08-01T02:20:20Z

src/quic/session.cc

+}
+
+int DefaultApplication::GetStreamData(StreamData* stream_data) {
+  if (stream_queue_.IsEmpty()) return 0;


This short circuit bypasses stream_data->stream.reset(stream);

My notes don't say why but this was an issue.

nodejs-github-bot added dont-land-on-v14.x needs-ci PRs that need a full CI run. quic Issues and PRs related to the QUIC implementation / HTTP/3. labels Aug 21, 2022

jasnell mentioned this pull request Aug 21, 2022

quic: add quic #38233

Closed

jasnell added semver-minor PRs that contain new features and should be released in the next minor version. experimental Issues and PRs related to experimental features. labels Aug 21, 2022

jasnell force-pushed the new-new-quic branch from 125eebb to 4f25588 Compare August 21, 2022 17:08

ruyadorno added dont-land-on-v16.x dont-land-on-v18.x PRs that should not land on the v18.x-staging branch and should not be released in v18.x. labels Aug 22, 2022

jasnell force-pushed the new-new-quic branch 2 times, most recently from 5c0053e to 161f525 Compare August 27, 2022 18:32

tniessen mentioned this pull request Sep 12, 2022

src: remove node_bob.h and node_bob-inl.h #44607

Closed

tniessen reviewed Sep 12, 2022

View reviewed changes

lib/internal/socketaddress.js Outdated Show resolved Hide resolved

lib/internal/event_target.js Outdated Show resolved Hide resolved

src/node_http_common.h Outdated Show resolved Hide resolved

This was referenced Sep 12, 2022

src: avoid copy when creating Blob #44616

Merged

lib,test: fix bug in InternalSocketAddress #44618

Merged

deps: update to ngtcp2 0.8.1 and nghttp3 0.7.0 #44622

Merged

tniessen reviewed Sep 15, 2022

View reviewed changes

src/crypto/crypto_common.cc Show resolved Hide resolved

espoal mentioned this pull request Oct 10, 2022

Add experimental support for io_uring #34388

Closed

jasnell mentioned this pull request Oct 16, 2022

lib: add ability to add separate event name to defineEventHandler #45032

Merged

jasnell force-pushed the new-new-quic branch from 2652d56 to 2e1802f Compare October 16, 2022 23:09

jasnell requested a review from tniessen October 16, 2022 23:10

bricss reviewed Dec 5, 2022

View reviewed changes

lib/internal/quic/quic.js

alpn,

servername,

preferredAddressStrategy,

undefined, // Connection ID Factory, not currently used.

Copy link

bricss Dec 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

huh? 🤔

Qard requested changes Dec 7, 2022

View reviewed changes

jasnell mentioned this pull request Dec 19, 2022

src: update src/crypto/* in preparation for quic #45912

Closed

jasonwilliams mentioned this pull request Dec 24, 2022

HTTP2/3 Support nodejs/undici#399

Closed

mscdex mentioned this pull request Mar 3, 2023

Support for QUIC protocol on official release #46921

Closed

agilan2001 mentioned this pull request Mar 4, 2023

feat: add quic support mqttjs/MQTT.js#1298

Closed

jasnell mentioned this pull request Mar 27, 2023

quic: implement various utilities classes to be used by the quic impl #47263

Closed

jasnell marked this pull request as draft March 27, 2023 00:34

jasnell mentioned this pull request Mar 29, 2023

quic: add additional utilities for quic (part 2) #47289

Closed

metcoder95 mentioned this pull request Apr 17, 2023

feat: Add H2 support nodejs/undici#2061

Merged

20 tasks

targos removed the dont-land-on-v14.x label May 3, 2023

Eomm mentioned this pull request Jun 20, 2023

QUIC implementation fastify/fastify#2379

Open

darrachequesne mentioned this pull request Jun 21, 2023

QUIC support socketio/socket.io#3769

Closed

targos removed the dont-land-on-v16.x label Sep 27, 2023

CommanderStorm mentioned this pull request Feb 11, 2024

Support HTTP/3 louislam/uptime-kuma#4475

Closed

CommanderStorm mentioned this pull request Mar 7, 2024

Support for selecting HTTP client version (HTTP/1.0, HTTP/1.1, HTTP/2, HTTP/3) louislam/uptime-kuma#4559

Open

splitice reviewed Aug 1, 2024

View reviewed changes

robertsLando mentioned this pull request Aug 8, 2024

feat: add quic support mqttjs/MQTT.js#1925

Draft

alexander-akait mentioned this pull request Aug 20, 2024

Please support server: 'quic' webpack/webpack-dev-server#4976

Closed

jasnell closed this Sep 28, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

src: quic #44325

src: quic #44325

jasnell commented Aug 21, 2022 •

edited

Loading

mcollina commented Aug 21, 2022

jasnell commented Aug 21, 2022

jasnell commented Aug 21, 2022 •

edited

Loading

splitice commented Aug 22, 2022

tniessen commented Sep 12, 2022 •

edited

Loading

splitice commented Oct 14, 2022

jasnell commented Oct 16, 2022 •

edited

Loading

jasnell commented Oct 16, 2022

GeoffreyBooth commented Dec 5, 2022

ronag commented Dec 5, 2022

Qard commented Dec 5, 2022

bricss Dec 5, 2022

bricss Dec 5, 2022

bricss commented Dec 5, 2022 •

edited

Loading

splitice commented Dec 5, 2022

bnoordhuis commented Dec 6, 2022

Qard left a comment

Qard Dec 5, 2022

Qard Dec 5, 2022

Qard Dec 5, 2022

Qard Dec 5, 2022

jasnell commented Dec 19, 2022

jasnell commented Mar 27, 2023

Faolain commented Jul 17, 2024

splitice left a comment

splitice Aug 1, 2024

splitice Aug 1, 2024

splitice Aug 1, 2024 •

edited

Loading

splitice Aug 1, 2024

	this.#inner.sendHEaders(QUIC_HEADERS_KIND_INFO,
	this.#inner.sendHeaders(QUIC_HEADERS_KIND_INFO,

src: quic #44325

src: quic #44325

Conversation

jasnell commented Aug 21, 2022 • edited Loading

mcollina commented Aug 21, 2022

jasnell commented Aug 21, 2022

jasnell commented Aug 21, 2022 • edited Loading

splitice commented Aug 22, 2022

tniessen commented Sep 12, 2022 • edited Loading

splitice commented Oct 14, 2022

jasnell commented Oct 16, 2022 • edited Loading

jasnell commented Oct 16, 2022

GeoffreyBooth commented Dec 5, 2022

ronag commented Dec 5, 2022

Qard commented Dec 5, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bricss commented Dec 5, 2022 • edited Loading

splitice commented Dec 5, 2022

bnoordhuis commented Dec 6, 2022

Qard left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jasnell commented Dec 19, 2022

jasnell commented Mar 27, 2023

Faolain commented Jul 17, 2024

splitice left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

splitice Aug 1, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jasnell commented Aug 21, 2022 •

edited

Loading

jasnell commented Aug 21, 2022 •

edited

Loading

tniessen commented Sep 12, 2022 •

edited

Loading

jasnell commented Oct 16, 2022 •

edited

Loading

bricss commented Dec 5, 2022 •

edited

Loading

splitice Aug 1, 2024 •

edited

Loading