Add setting to spawn instead of fork a worker #16943
Conversation
Add a config setting per worker type to force the worker to be spawned instead of forked.
|
cc @jrafanie thinking of this for the DRb incompatible marshal file error |
|
Checked commit agrare@32adf61 with ruby 2.3.3, rubocop 0.52.0, haml-lint 0.20.0, and yamllint 1.10.0 |
| end | ||
|
|
||
| def spawn_worker? | ||
| ENV['MIQ_SPAWN_WORKERS'] || !Process.respond_to?(:fork) || worker_settings[:spawn_worker] |
There was a problem hiding this comment.
@agrare Why do we need a setting for this? Does the ENV variable not work? I'm not sure the "why" here
There was a problem hiding this comment.
I was hoping this would give us a way to control this on fine release, I wasn't aware of a way to change the env vars per worker type on that release but if it exists then great.
There was a problem hiding this comment.
The real "why" is to spawn not fork VMware provider workers on a fine appliance to see if it solves the drb errors connecting with the broker
There was a problem hiding this comment.
@agrare Pretty sure this SPAWN_WORKERS isn't available on fine:
https://github.com/ManageIQ/manageiq/blob/fine/app/models/miq_worker.rb
So you might need to do a bit of backporting if that was the end goal.
There was a problem hiding this comment.
Yes absolutely I was planning on needing to backport some of @jrafanie 's changes to make this an option.
There was a problem hiding this comment.
Mmmk, you will absolute need this then as well:
https://github.com/ManageIQ/manageiq/blob/master/lib/workers/bin/run_single_worker.rb
And there are a few PRs surrounding that as well that would need to be backported.
There was a problem hiding this comment.
Hmm, I have a hard time believing this would work without a significant amount of work to backport changes. The main problem is the forked workers inherit all of the booted code, $LOADED_FEATURES, so if we go spawn, we have to ensure anything that wasn't loaded in the spawned workers, we'd need to make it load as part of the boot of the spawned workers.
|
Closing in favor of #16953 |
Add a config setting per worker type to force the worker to be spawned
instead of forked.
This will allow us to force certain provider workers to be spawned
instead of forked to work around possible shared thread object ID
issues.