Exits with code 1 on SIGTERM without waiting for child

[adamdunkley]

Do you want to request a feature or report a bug?
This is a bug

What is the current behavior?
Currently yarn terminates with exit code 1 when it receives a SIGTERM without waiting for the child to exit and without passing its child's exit status up the chain.

The issue that this stemmed from was about handling SIGTERM events (#3424) and was introduced in pull request #3789.

If the current behavior is a bug, please provide the steps to reproduce.

Create a simple express script:

const app = require('./express');

const server = app.listen(config.port, () => {
    console.log(`server started`);
});
process.on('SIGTERM', () => {
    server.close(() => {
      setTimeout(() => {
        process.exit(0);
      }, 1000);
    });
});

Note that the script uses setTimeout to wait before exiting (as if it is waiting 1s for a client connection to end).

⇒  yarn start
yarn run v1.1.0
$ node src/index.js
server started

In another session run the following with the PID of that command:

kill $PID

Finally in that first session after it has exited:

⇒  echo $?
1

You can see that it exits immediately (not waiting 1s) and the error code is 1.

What is the expected behavior?

Expected behaviour is to do exactly as node would:

⇒  node src/index.js
server started
⇒  echo $?
0

This waits the 1s for the setTimeout (we kill using kill $PID in between the node and echo commands as above). You can also see we get the child's exit status (you can verify this by setting it to something else like process.exit(9) and seeing it come out).

I see some comments in the code where it is questioning what we should do about this:

yarn/src/util/signal-handler.js

Lines 6 to 8 in daa599d

	// We want to exit immediately here since `SIGTERM` means that
	// If we lose stdout messages due to abrupt exit, shoot the messenger?
	process.exit(1); // eslint-disable-line no-process-exit

I think it's the child's job to decide what to do if a SIGTERM event is received. The parent should hand the signal to the child and wait to see if it exits. The only thing that I think Yarn should consider is whether there is a time limit that we would like to wait for naughty child processes that do not exit on time. Docker sets this to 10 seconds, whatever we set this to (if we do at all!!) we should make it a) clear and b) configurable.

One other thing: currently not sure what we do with SIGINT. I see weird stuff happening with that too (and I presume other signals?). Probably worth investigating signal handling more holistically?

Please mention your node.js, yarn and operating system version.

• Node version: 7 & 8
• Yarn version: 1.10.0
• macOS 16.3.0 and alpine

[adamdunkley]

I am very willing to commit time/code to getting this fixed properly as I think it's in a bit of a state at the moment but would like to get some agreement on direction first :)

[thetimothyp]

Has there been any more discussion on this?

I agree that it might be worth taking a look at signal handling as whole; it doesn't look like SIGINT is handled at all, apart from where onDeath() handles it in mutex mode. I think it might have to be handled slightly differently, since SIGINT is sent to all the processes in the foreground group instead of needing to be forwarded through the parent.

I'd also be willing to devote some time to this to draft an RFC or something like that.

[ianstormtaylor]

Any maintainer input on this? It seems like it prevents Yarn from being used for real in any situations where you need yarn run to spawn a child in the background, which would be very useful in CI.

[zargold]

I am still getting this issue by running a process using yarn vs running it with npm run:
package.json:

{
"scripts": {
"test": "concurrently -k --success first \"npm run dev:test\" \"yarn jest\""
}

app.js

process.on('SIGTERM', async () => {
  await server.close()
  console.log('EXITING GRACEFULLY...')
  process.exit(0)
})

WHEN USING YARN:
"test": "concurrently -k --success first \"yarn dev:test\" \"yarn jest\""

yarn test
....
[1] Test Suites: 3 passed, 3 total
[1] Tests:       42 passed, 42 total
[1] Snapshots:   0 total
[1] Time:        15.137s
[1] Ran all test suites.
[1] yarn jest exited with code 0
--> Sending SIGTERM to other processes..
[0] EXITING GRACEFULLY...
[0] yarn dev:test exited with code 1
✨  Done in 16.58s.

WHEN USING NPM RUN:

yarn test
...
[1] Test Suites: 3 passed, 3 total
[1] Tests:       42 passed, 42 total
[1] Snapshots:   0 total
[1] Time:        15.138s
[1] Ran all test suites.
[1] yarn jest exited with code 0
--> Sending SIGTERM to other processes..
[0] EXITING GRACEFULLY...
[0] npm run dev:test exited with code 0
✨  Done in 16.54s.

So with npm run running my server I can remove the --success first and it still passes but with yarn it fails due to the exit code 1 on the server.

System Info:

>>> yarn -v
1.6.0
>>> npm -v
5.6.0
>>> node -v
v10.0.0

Also I am not using nodemon to run my server:
"dev:test": "NODE_ENV=test node -r dotenv/config ./app.js",
"jest": "NODE_ENV=test jest --no-cache --runInBand",

[presidenten]

Same problem.

$ yarn -v
1.12.3

$ node -v
v8.12.0

$ npm -v
6.4.1

$ node index.js
^C

Doing async cleaning
Done

$ npm start
^C

Doing async cleaning


Doing async cleaning
Done

$ yarn start
yarn run v1.12.3
^C

Doing async cleaning

$ Done # <--- Console.log('Done') ends up on new line in console, because yarn already quit.

To test:
--package.json--

{
  "scripts": {
    "start": "node index.js"
  }
}

--index.js--

process.stdin.resume();

process.on('SIGINT', () => {
  console.log('\n');
  console.log('Doing async cleaning');
  
  setTimeout(() => {
    console.log('Done');
    process.exit(0);
  }, 500);
});

[heisian]

Agreed, this makes it impossible to use yarn for running processes that manage child processes.

[heisian]

Here is what I ended up doing:

// The following captures keypress events and effectively blocks any of the
// CTRL+C/CTRL+Z/CTRL+D combinations from sending their respective signals.

readline.emitKeypressEvents(process.stdin)

if (process.stdin.isTTY) process.stdin.setRawMode(true)

process.stdin.on('keypress', (char, key) => {
  if (key.ctrl && key.name === 'c') {
    console.log("Caught CTRL+C and asking you to wait for this process to exit gracefully, mi'lord.")
    const exitCode = 0
    cleanupHandler(exitCode)
  }
})

[fyodorvi]

Same here, any ideas if this can be fixed?

[bodinsamuel]

Issue is still valid, is there any progress? ☺️

This can be reproduced with a very simple setup script

const wait = waitTime => new Promise(resolve => setTimeout(resolve, waitTime));
setInterval(() => {
  console.log("interval");
}, 1000);

process.on('SIGINT', async () => {
  console.log('caught');
  await wait(1000);
  console.log('killing');
  process.exit(0);
});

when doing node test.js it waits for proper killing, but not with yarn node test.js

[aleclarson]

I'm seeing this issue in yarn v1.22.4 but not in npm v6.12.0

It makes yarn unusable 😢

[adamdunkley]

@aleclarson my feeling is that attention now needs to moved to yarn v2 to get this fixed. They actually have a regression on issue #3424 (the fix for which created this bug). I have commented on the issue on v2 that talks about this regression asking whoever implements a solution for it to consider this documented behaviour in order to avoid these problems. (see yarnpkg/berry#991)

[adamdunkley]

After some careful review (read yarnpkg/berry#991 for all the juicy details) the new behaviour I mentioned in v2 is actually desirable and the nub of it is that this is no longer a problem in v2 (yarn's exit status is the exit status of the child process). I will leave this open in case someone wants to either backport how it works in v2 to here (which might cause more problems if people rely on v1's behaviour) or fix the issue for v1, but for anyone who wants a solution to this now I recommend upgrading to v2!

[morungos]

Whatever is happening with this issue, it's still an annoyance. I ended up throwing together a tiny wrapper script -- my use case is that I use yarn and run-p to test a hybrid PHP app, and I needed a way to serve PHP alongside a proxy for migration, and I was using php-fpm. Weirdly, if I SIGINT php-fpm on the command line, all was fine, but if I use a yarn script, the SIGINT never got passed to php-fpm, so all the worker processes were left orphaned. My tiny wrapper script simply executes a command with inherit on its pipes, but forwards SIGINT to the child php-fpm. It's literally 10 lines. But I could not find any other way to make yarn + run-p handle process interruption correctly. Of course, seeing as how this is php-fpm, this is one of those cases where leaving it to the child to decide won't work.

[danmelton]

Sample code for a run script with multiple arguments:

yarn run commands "command1" "command2" "etc"

// commands.js
const CHILD_PROCESSES = {};

function killProcessAfterChildrenDie() {
    if (Object.keys(CHILD_PROCESSES).length === 0) {
        process.exit(0)
    } else {
        setTimeout(() => {
            killProcessAfterChildrenDie()
        },             1000)
    }
}

readline.emitKeypressEvents(process.stdin)

if (process.stdin.isTTY) process.stdin.setRawMode(true)

process.stdin.on('keypress', (char, key) => {
    if (key.ctrl && key.name === 'c') {
        if (Object.keys(CHILD_PROCESSES).length) {
            Object.keys(CHILD_PROCESSES).forEach(key => {
                CHILD_PROCESSES[key].kill()
            })
        }
        killProcessAfterChildrenDie()
    }
})

const commands = process.argv.slice(2)

commands.forEach(command => {
        const cp = exec(command);
        cp.stdout.pipe(process.stdout);
        cp.stderr.pipe(process.stderr);
        cp.on("exit", code => {
            delete CHILD_PROCESSES[command]
            if (code) {
                console.error(`Non-Zero Exit(${code}): ${command}`);
                process.exit(1);
            }
        });
        cp.on("error", err => {
            console.error(`Error(${err.message}): ${command}`);
            process.exit(1);
        });

        CHILD_PROCESSES[command] = cp
    });

[JasonMan34]

Is this issue still unresolved?
I'm aware that this is solved in yarn v3, but my docker environment is using yarn 1 to spawn yarn 3, so it's still an issue

[dobesv]

This is a major issue for shutdown handling when using yarn berry or yarn v1. Node will not update the version of yarn in the docker image and yarn seemingly will not backport the fix to yarn v1, so you have to manually update the global yarn in your own docker images before using it to run apps with yarn inside docker.