Skip to content

Commit

Permalink
Merge pull request #259 from pyiron/update_docstrings
Browse files Browse the repository at this point in the history 
Update hostname_localhost Docstring
  • Loading branch information
@jan-janssen
jan-janssen authored Jan 31, 2024
2 parents eca7c31 + 431a9d9 commit c841760
Show file tree
Hide file tree
Showing 6 changed files with 82 additions and 7 deletions.
15 changes: 15 additions & 0 deletions pympipool/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,13 @@ class Executor:
init_function (None): optional function to preset arguments for functions which are submitted later
cwd (str/None): current working directory where the parallel python task is executed
sleep_interval (float): synchronization interval - default 0.1
hostname_localhost (boolean): use localhost instead of the hostname to establish the zmq connection. In the
context of an HPC cluster this essential to be able to communicate to an
Executor running on a different compute node within the same allocation. And
in principle any computer should be able to resolve that their own hostname
points to the same address as localhost. Still MacOS >= 12 seems to disable
this look up for security reasons. So on MacOS it is required to set this
option to true
Examples:
```
Expand Down Expand Up @@ -107,6 +114,14 @@ def __new__(
init_function (None): optional function to preset arguments for functions which are submitted later
cwd (str/None): current working directory where the parallel python task is executed
sleep_interval (float): synchronization interval - default 0.1
hostname_localhost (boolean): use localhost instead of the hostname to establish the zmq connection. In the
context of an HPC cluster this essential to be able to communicate to an
Executor running on a different compute node within the same allocation. And
in principle any computer should be able to resolve that their own hostname
points to the same address as localhost. Still MacOS >= 12 seems to disable
this look up for security reasons. So on MacOS it is required to set this
option to true
"""
if flux_installed:
if oversubscribe:
Expand Down
16 changes: 14 additions & 2 deletions pympipool/flux/executor.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,13 @@ class PyFluxExecutor(ExecutorBase):
cwd (str/None): current working directory where the parallel python task is executed
sleep_interval (float): synchronization interval - default 0.1
executor (flux.job.FluxExecutor): Flux Python interface to submit the workers to flux
hostname_localhost (boolean): use localhost as hostname to establish the zmq connection
hostname_localhost (boolean): use localhost instead of the hostname to establish the zmq connection. In the
context of an HPC cluster this essential to be able to communicate to an
Executor running on a different compute node within the same allocation. And
in principle any computer should be able to resolve that their own hostname
points to the same address as localhost. Still MacOS >= 12 seems to disable
this look up for security reasons. So on MacOS it is required to set this
option to true
Examples:
Expand Down Expand Up @@ -96,7 +102,13 @@ class PyFluxSingleTaskExecutor(ExecutorBase):
init_function (None): optional function to preset arguments for functions which are submitted later
cwd (str/None): current working directory where the parallel python task is executed
executor (flux.job.FluxExecutor): Flux Python interface to submit the workers to flux
hostname_localhost (boolean): use localhost as hostname to establish the zmq connection
hostname_localhost (boolean): use localhost instead of the hostname to establish the zmq connection. In the
context of an HPC cluster this essential to be able to communicate to an
Executor running on a different compute node within the same allocation. And
in principle any computer should be able to resolve that their own hostname
points to the same address as localhost. Still MacOS >= 12 seems to disable
this look up for security reasons. So on MacOS it is required to set this
option to true
"""

Expand Down
16 changes: 14 additions & 2 deletions pympipool/mpi/executor.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,13 @@ class PyMPIExecutor(ExecutorBase):
init_function (None): optional function to preset arguments for functions which are submitted later
cwd (str/None): current working directory where the parallel python task is executed
sleep_interval (float): synchronization interval - default 0.1
hostname_localhost (boolean): use localhost as hostname to establish the zmq connection
hostname_localhost (boolean): use localhost instead of the hostname to establish the zmq connection. In the
context of an HPC cluster this essential to be able to communicate to an
Executor running on a different compute node within the same allocation. And
in principle any computer should be able to resolve that their own hostname
points to the same address as localhost. Still MacOS >= 12 seems to disable
this look up for security reasons. So on MacOS it is required to set this
option to true
Examples:
Expand Down Expand Up @@ -85,7 +91,13 @@ class PyMPISingleTaskExecutor(ExecutorBase):
oversubscribe (bool): adds the `--oversubscribe` command line flag (OpenMPI only) - default False
init_function (None): optional function to preset arguments for functions which are submitted later
cwd (str/None): current working directory where the parallel python task is executed
hostname_localhost (boolean): use localhost as hostname to establish the zmq connection
hostname_localhost (boolean): use localhost instead of the hostname to establish the zmq connection. In the
context of an HPC cluster this essential to be able to communicate to an
Executor running on a different compute node within the same allocation. And
in principle any computer should be able to resolve that their own hostname
points to the same address as localhost. Still MacOS >= 12 seems to disable
this look up for security reasons. So on MacOS it is required to set this
option to true
"""

Expand Down
18 changes: 18 additions & 0 deletions pympipool/shared/communication.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,24 @@ def interface_bootup(
connections,
hostname_localhost=False,
):
"""
Start interface for ZMQ communication
Args:
command_lst (list): List of commands as strings
connections (pympipool.shared.interface.BaseInterface): Interface to start parallel process, like MPI, SLURM or
Flux
hostname_localhost (boolean): use localhost instead of the hostname to establish the zmq connection. In the
context of an HPC cluster this essential to be able to communicate to an
Executor running on a different compute node within the same allocation. And
in principle any computer should be able to resolve that their own hostname
points to the same address as localhost. Still MacOS >= 12 seems to disable
this look up for security reasons. So on MacOS it is required to set this
option to true
Returns:
pympipool.shared.communication.SocketInterface: socket interface for zmq communication
"""
if not hostname_localhost:
command_lst += [
"--host",
Expand Down
8 changes: 7 additions & 1 deletion pympipool/shared/executorbase.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,13 @@ def execute_parallel_tasks(
future_queue (queue.Queue): task queue of dictionary objects which are submitted to the parallel process
cores (int): defines the total number of MPI ranks to use
interface_class:
hostname_localhost (boolean): use localhost as hostname to establish the zmq connection
hostname_localhost (boolean): use localhost instead of the hostname to establish the zmq connection. In the
context of an HPC cluster this essential to be able to communicate to an
Executor running on a different compute node within the same allocation. And
in principle any computer should be able to resolve that their own hostname
points to the same address as localhost. Still MacOS >= 12 seems to disable
this look up for security reasons. So on MacOS it is required to set this
option to true
"""
execute_parallel_tasks_loop(
interface=interface_bootup(
Expand Down
16 changes: 14 additions & 2 deletions pympipool/slurm/executor.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,13 @@ class PySlurmExecutor(ExecutorBase):
init_function (None): optional function to preset arguments for functions which are submitted later
cwd (str/None): current working directory where the parallel python task is executed
sleep_interval (float): synchronization interval - default 0.1
hostname_localhost (boolean): use localhost as hostname to establish the zmq connection
hostname_localhost (boolean): use localhost instead of the hostname to establish the zmq connection. In the
context of an HPC cluster this essential to be able to communicate to an
Executor running on a different compute node within the same allocation. And
in principle any computer should be able to resolve that their own hostname
points to the same address as localhost. Still MacOS >= 12 seems to disable
this look up for security reasons. So on MacOS it is required to set this
option to true
Examples:
Expand Down Expand Up @@ -92,7 +98,13 @@ class PySlurmSingleTaskExecutor(ExecutorBase):
oversubscribe (bool): adds the `--oversubscribe` command line flag (OpenMPI only) - default False
init_function (None): optional function to preset arguments for functions which are submitted later
cwd (str/None): current working directory where the parallel python task is executed
hostname_localhost (boolean): use localhost as hostname to establish the zmq connection
hostname_localhost (boolean): use localhost instead of the hostname to establish the zmq connection. In the
context of an HPC cluster this essential to be able to communicate to an
Executor running on a different compute node within the same allocation. And
in principle any computer should be able to resolve that their own hostname
points to the same address as localhost. Still MacOS >= 12 seems to disable
this look up for security reasons. So on MacOS it is required to set this
option to true
"""

Expand Down

0 comments on commit c841760

Please sign in to comment.