🐛 BUG: workerd inspector process doesn't die

[jjonte]

Which Cloudflare product(s) does this pertain to?

Workers Runtime

What version(s) of the tool(s) are you using?

3.21.0

What version of Node are you using?

18.17.1

What operating system are you using?

MacOS, Sonoma

Describe the Bug

When shutting down a running wrangler dev on a basic "hello world" Worker, an extra workerd process hangs around, occupies the port, and eats CPU cycles (1 core @ ~100%). The only solution is to Force Quit the process.

export default {
	async fetch(request: Request, env: unknown, ctx: ExecutionContext) {
		return new Response('hello world');
	},
};

Please provide a link to a minimal reproduction

No response

Please provide any relevant error logs

When running wrangler dev it looks like this:

After pressing x in the terminal window this happens:

[zhawtof]

I'm also experiencing the same issue. Same wrangler version and Mac specs.

[ezg27]

Experiencing the same with MacOS Ventura 13.5.1, Node v20.9.0.

It looks as though this may have started in Wrangler v3.20.0 with this PR to add Sentry crash reporting. Commenting out the await closeSentry() call here leads to Wrangler processes exiting successfully with both cmd+c and the [x] to exit shortcut. Processes also exit successfully when reverting to Wrangler v3.19.0 before the Sentry handling was added.

Haven't managed to figure out exactly what's causing things to hang yet though. The Node docs say the following re calls to process.disconnect():

...the process.disconnect() method will close the IPC channel to the parent process, allowing the child process to exit gracefully once there are no other connections keeping it alive.

So could be there's some connections from Sentry flushing events in Sentry.close() that are preventing things from exiting?

[alpertuna]

My setup: Arch Linux Kernel: 6.6.7-arch1-1, NodeJs: v21.4.0

I have a similar issue but it is not new for me, when I realized the hanging processes, the wrangler version was v3.17.

Now I use wrangler v3.21.0 but the issue is still there.

I use wrangler for local development. Whenever I stop the wrangler dev, there is always a workerd process left behind that keeps the default port open and, makes a single core 100% after some time.
I can't kill the process gracefully, so I need to kill it forcefully.

My issue may be different since I've been experiencing it with older wrangler versions too. If I collect enough information I may open another issue with the information

To automatically kill the zombie process, I use a script similar to the script mentioned here; #3378 (comment)

[admah]

@jjonte thanks for filing this bug. We are taking a look and will add updates as they are availaible.

[admah]

Possibly related to #4131

[peterboyer]

I am also experiencing this problem.

Downgrading to [email protected] fixes the zombie process for me.
Upgrading to [email protected] re-introduces the zombie process.
Problem still present as of [email protected].

Node v20.5.0

[peterboyer]

[EDIT: Whoops. I realise that should've read the thread more closely as the cause has already been discovered earlier in this thread by ezg27, and a PR to fix it has since been merged. But I'll leave this here anyway, as it describes the technique I used to independently debug the problem. 🤷]

---

I think I found the cause of the problem + a hacky/temporary workaround for local dev.

I just inspected the diff[1] between release tags [email protected] and [email protected].

[1] [email protected]@3.20.0

I found this newly added call[2] of await Sentry.close(); as part of wrangler's recent introduction of Sentry for error reporting.

[2] packages/wrangler/src/sentry/index.ts:130 (from diff)

---

If you install [email protected], open up .../node_modules/wrangler/wrangler-dist/cli.js, and comment out the corresponding minified await close(); call on line 154410 (found by searching for the nearby "Would you like to report this error to Cloudflare?" string) then the workerd process now properly quits without hanging.

We can upgrade to [email protected], open up .../node_modules/wrangler/wrangler-dist/cli.js, and comment out the same call on line 154209 (found by the same search string as above) and the workerd process now properly quits without hanging.

---

This await Sentry.close(); call is part of the closeSentry() function as part of the new Sentry integration.

workers-sdk/packages/wrangler/src/sentry/index.ts

Lines 128 to 132 in 35e8a5d

	export async function closeSentry() {
	if (typeof SENTRY_DSN !== "undefined") {
	await Sentry.close();
	}
	}

It seems that awaiting this function is not resolving when being called as part wrangler's main() function's finally block, which is then preventing process.disconnect() from being called, which I presume is in charge of letting the workerd process die.

workers-sdk/packages/wrangler/src/index.ts

Lines 767 to 775 in 35e8a5d

	} finally {
	await closeSentry();
	// In the bootstrapper script `bin/wrangler.js`, we open an IPC channel, so
	// IPC messages from this process are propagated through the bootstrapper.
	// Make sure this channel is closed once it's no longer needed, so we can
	// cleanly exit. Note, we don't want to disconnect if this file was imported
	// in Jest, as that would stop communication with the test runner.
	if (typeof jest === "undefined") process.disconnect?.();
	}

---

I hope this helps.

For now, if you'd like to hack around the problem for local development before this is patched upstream: you may upgrade to wrangler@latest and comment out this Sentry await close(); call to allow workerd to quit gracefully.