Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

src: fix Worker termination in inspector.waitForDebugger #52527

Merged
merged 9 commits into from
May 13, 2024

Conversation

daeyeon
Copy link
Member

@daeyeon daeyeon commented Apr 14, 2024

Fixes: #52467

When calling inspector.waitForDebugger() on a worker thread, it waits synchronously for debugger to connect using a conditional variable, incoming_message_cond_. In this state, the worker thread cannot be terminated by the main thread.

This patch adds a way to exit the wait state when node process exits.

Signed-off-by: Daeyeon Jeong [email protected]

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Apr 14, 2024
@daeyeon daeyeon added inspector Issues and PRs related to the V8 inspector protocol worker Issues and PRs related to Worker support. labels Apr 14, 2024
@daeyeon
Copy link
Member Author

daeyeon commented Apr 14, 2024

/cc @nodejs/cpp-reviewers @nodejs/inspector @nodejs/workers

Copy link
Member

@benjamingr benjamingr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice analysis!

src/inspector/main_thread_interface.cc Show resolved Hide resolved
src/env.cc Outdated Show resolved Hide resolved
@daeyeon daeyeon added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 15, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 15, 2024
@nodejs-github-bot

This comment was marked as outdated.

@daeyeon daeyeon added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 15, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 15, 2024
@nodejs-github-bot

This comment was marked as outdated.

await new Promise((r) => setTimeout(r, 2_000));

process.on('exit', (status) => assert.strictEqual(status, 0));
process.exit();
Copy link
Contributor

@eugeneo eugeneo Apr 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I suggest merging the test cases - start worker # 1, terminate it, start worker # 2, call process.exit
  2. What would happen if the main thread runs to completion? Would the worker just stay hanging? I don't remember details, would the WebSocket server shut down when the main thread completes and there's no ongoing sessions?

I believe the broader question is if inspector.waitForDebugger even makes sense for workers. The way I remember target domain is you either specify autoattach so existing session would attach to newly created workers, or workers just start and sessions are attached whenever.

In the first case (autoattach) worker should only wait if there are connected sessions that asked to be attached to workers.
In the second case worker do not need to wait.

I apologize if I am not making sense, haven't worked on this in years...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Feel free to dismiss my comments as I am not sure I am up to date with the current state of inspector)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the broader question is if inspector.waitForDebugger even makes sense for workers. The way I remember target domain is you either specify autoattach so existing session would attach to newly created workers, or workers just start and sessions are attached whenever.

As far as I know, inspector.waitForDebugger is the only working way to (programatically) debug Workers. #26609 (comment)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest merging the test cases

Applied.

What would happen if the main thread runs to completion?

If the main thread runs to completion (assuming it runs without process.exit() in this test), then the WS server doesn't shut down, leaving the worker hanging.

Users seem to be using inspector.waitForDebugger as a valid workaround for now. I did several tries, but autoattach doesn't seem to work.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack. Thank you for the explanations.

// inspector.waitForDebugger().
if (isMainThread) {
const worker = new Worker(fileURLToPath(import.meta.url));
await new Promise((r) => setTimeout(r, 2_000));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using timeout to wait until a worker is ready can be flaky on slow CI machines.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be more stable approach:

// Flags: --inspect=0
import * as common from '../common/index.mjs';
common.skipIfInspectorDisabled();

-import { isMainThread, Worker } from 'node:worker_threads';
+import { isMainThread, parentPort, Worker } from 'node:worker_threads';
import { fileURLToPath } from 'node:url';
import inspector from 'node:inspector';

// Ref: https://github.com/nodejs/node/issues/52467
// - worker.terminate() should terminate the worker and the pending
//   inspector.waitForDebugger().
if (isMainThread) {
  const worker = new Worker(fileURLToPath(import.meta.url));
- await new Promise((r) => setTimeout(r, 2_000));
+ await new Promise((r) => worker.on("message", r));

  worker.on('exit', common.mustCall());
  await worker.terminate();
} else {
  inspector.open();
+ parentPort.postMessage("inspector is open")
  inspector.waitForDebugger();
}

Copy link
Member Author

@daeyeon daeyeon Apr 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated using MessagePort. Thanks. PTAL.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using MessagePort doesn't seem to be enough. I tried using common.platformTimeout() to adjust the timeout to give the worker more time.

@daeyeon daeyeon added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 16, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 16, 2024
@nodejs-github-bot

This comment was marked as outdated.

@daeyeon daeyeon added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 17, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 17, 2024
@nodejs-github-bot

This comment was marked as outdated.

@daeyeon
Copy link
Member Author

daeyeon commented Apr 17, 2024

Changed the test to .js to fix the failed CI test, node-test-linux-linked-withoutssl.

common.skipIfInspectorDisabled() skips the test when the V8 inspector is disabled using either --without-inspector or --without-ssl. Since import statements are hoisted, importing node:inspector threw an error before executing common.skipIfInspectorDisabled() on the CI test.

@daeyeon daeyeon added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 17, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 17, 2024
@nodejs-github-bot

This comment was marked as outdated.

@daeyeon daeyeon added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 17, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 17, 2024
@nodejs-github-bot
Copy link
Collaborator

@daeyeon daeyeon added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. labels Apr 17, 2024
@targos targos removed the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Apr 24, 2024
@daeyeon daeyeon added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label May 12, 2024
@cola119 cola119 added the commit-queue Add this label to land a pull request using GitHub Actions. label May 13, 2024
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label May 13, 2024
@nodejs-github-bot nodejs-github-bot merged commit ac5f8f0 into nodejs:main May 13, 2024
67 checks passed
@nodejs-github-bot
Copy link
Collaborator

Landed in ac5f8f0

targos pushed a commit that referenced this pull request May 13, 2024
Signed-off-by: Daeyeon Jeong <[email protected]>
PR-URL: #52527
Fixes: #52467
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Kohei Ueno <[email protected]>
marco-ippolito pushed a commit that referenced this pull request Jun 17, 2024
Signed-off-by: Daeyeon Jeong <[email protected]>
PR-URL: #52527
Fixes: #52467
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Kohei Ueno <[email protected]>
marco-ippolito pushed a commit that referenced this pull request Jun 17, 2024
Signed-off-by: Daeyeon Jeong <[email protected]>
PR-URL: #52527
Fixes: #52467
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Kohei Ueno <[email protected]>
marco-ippolito pushed a commit that referenced this pull request Jun 17, 2024
Signed-off-by: Daeyeon Jeong <[email protected]>
PR-URL: #52527
Fixes: #52467
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Kohei Ueno <[email protected]>
EliphazBouye pushed a commit to EliphazBouye/node that referenced this pull request Jun 20, 2024
Signed-off-by: Daeyeon Jeong <[email protected]>
PR-URL: nodejs#52527
Fixes: nodejs#52467
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Kohei Ueno <[email protected]>
bmeck pushed a commit to bmeck/node that referenced this pull request Jun 22, 2024
Signed-off-by: Daeyeon Jeong <[email protected]>
PR-URL: nodejs#52527
Fixes: nodejs#52467
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Kohei Ueno <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. c++ Issues and PRs that require attention from people who are familiar with C++. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. inspector Issues and PRs related to the V8 inspector protocol lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. worker Issues and PRs related to Worker support.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Worker with inspector.waitForDebugger() cannot be terminated
10 participants