Skip to content

Commit

Permalink
Use spawn in process-pool executor fixture
Browse files Browse the repository at this point in the history
This raises a deprecation warning on POSIX where the default is fork()
which will change in 3.14. It is also unsafe to fork the test process
because it is multithreaded.

See python/cpython#100228 and
https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods
  • Loading branch information
gpauloski committed Sep 5, 2024
1 parent a137542 commit 2a6060c
Showing 1 changed file with 9 additions and 1 deletion.
10 changes: 9 additions & 1 deletion testing/fixtures.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from __future__ import annotations

import multiprocessing
import pathlib
from concurrent.futures import ProcessPoolExecutor
from concurrent.futures import ThreadPoolExecutor
Expand Down Expand Up @@ -47,7 +48,14 @@ def dask_process_executor() -> Generator[DaskDistributedExecutor, None, None]:

@pytest.fixture()
def process_executor() -> Generator[ProcessPoolExecutor, None, None]:
with ProcessPoolExecutor(4) as executor:
with ProcessPoolExecutor(
max_workers=4,
# Spawn is already the default on Windows and MacOS. Fork is
# the default on POSIX platforms but will change in 3.14 because
# forking a multithreaded process is not safe (and the test suite
# is multithreaded because the ThreadPoolExecutor).
mp_context=multiprocessing.get_context('spawn'),
) as executor:
yield executor


Expand Down

0 comments on commit 2a6060c

Please sign in to comment.