Immediately kill workers to prevent straglers #4744
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #4737
Each Worker currently listens for the process "exit" event and attempts to kill it's child process to prevent leaving stragler processes. This was implemented by calling
fork.kill()
, which I assumed to work likeprocess.kill()
, but unfortunately it is async and doesn't sendSIGTERM
to the process in the same tick (a requirement of process "exit" listeners).This fixes the implementation by switching to
fork.process.exit()
, which synchronously sendsSIGTERM
.To replication steps from #4737:
delete your
optimize/bundles
directorycreate an empty file at
optimize/bundles/webpack.records
in a console tab, run this script to repeatedly log the iojs process list
in another console tab run the kibana server in dev mode
observe the server logging
optimizer crashed with status code 1
and the process list growing by one each time you do _#_4Finally
killall iojs