Using Python's multiprocessing on Master

[ovanes]

Description of issue / feature request

I wonder if there any cautions to be taken if I plan to use Python's multiprocessing (e.g. https://docs.python.org/2/library/multiprocessing.html#using-a-pool-of-workers) library on locust master to process custom data. I am just afraid if underlying process forking might break smth. in locust Master. There were evaluations for that with slaves here: #12

Expected behavior

From the Locust's custom data event an item shall be placed in https://docs.python.org/2/library/multiprocessing.html#multiprocessing.JoinableQueue and using another Greelet mapped to a multiprocessing.Pool of workers.

Actual behavior

Currently the data slaves send fully utilizes a single thread used for greenlets and master lags pretty much behind.

Environment settings (for bug reports)

• OS: Linux (from PyPy docker container)
• Python version: 2.7.x & 3.6.x & PyPy
• Locust version: 0.8.1

[cgoldberg]

I wonder if there any cautions to be taken if I plan to use Python's multiprocessing

why don't you try it?

[ovanes]

why don't you try it?

Why spending time if someone did it already? Just wanted to rely on experience of others.