Always use EM_PYTHON_MULTIPROCESSING after win32 bugfix

[sbc100]

This removes the need for the alternative implementation now that the primary windows issue has been resolved.

[juj]

No!

We should not use Python multiprocessing anywhere in production. That is a hard no.

Rather if something, we should delete the EM_PYTHON_MULTIPROCESSING=1 path altogether.

[sbc100]

I wrote this patch 7 months ago, and I seem to remember researching the various bugs and finding them all fixed, but I wish now that I had written more a write up in the description.

I also can't quite remember why I decided to even create this patch :)

[sbc100]

BTW are all the issues you know about windows-specific, or are there reasons to avoid multi-processing on non-windows too?

[sbc100]

Sadly I don't remember specifically what I meant by "after win32 bugfix".

[juj]

BTW are all the issues you know about windows-specific, or are there reasons to avoid multi-processing on non-windows too?

I think the root cause of python multiprocessing being so bad is that it is based on the fork-join model, which Windows does not provide, and the concept of pickling, which they have left to be a half-developed abstraction. That paired with the issue that most Python developers do not actually care about Windows, has resulted in this mess.

While at Mozilla, Python process spawning + multiprocessing on Windows was the single biggest class of Python bugs that would repeatedly crop up, and at Unity, Python multiprocessing issues were by far the #1 most commonly reported bug to Unity's WebGL platform until I migrated Emscripten to the custom sub-process dispatch, after which these issues vanished to zero.

Currently there is still a bug that one cannot abort (Ctrl-Break) a Python process on Windows if a multiprocessing pool is alive e.g. running a parallel test suite for example. Other times when one is able to abort, the multiprocessing pool processes do not actually shut down, but stay hung as zombies in the background, which then results in having to run taskkill /f /im python.exe to kill all those python subprocesses.

On Linux and macOS I do not recall any multiprocessing issues.