Skip to content

Commit

Permalink
Support cpu and mem specification via env (#1207)
Browse files Browse the repository at this point in the history
* Support cpu and mem specification via env

This commit adds support for respecting specification of how much CPU
and memory should be used by workers via the PARSL_CORES and
PARSL_GB_MEMORY environmental variables (as opposed to inferring it
to be the full node, once a job has started on the node.) After this commit, the config
below will result in 4 workers being started, using a total of 4 cores
and 12 GB memory.

This is a pre-requisite to a full solution to #942.

```
from parsl.config import Config
from parsl.providers import SlurmProvider
from parsl.addresses import address_by_hostname
from parsl.executors import HighThroughputExecutor

config = Config(
    executors=[
        HighThroughputExecutor(
            cores_per_worker=1,
            mem_per_worker=3,
            address=address_by_hostname(),
            provider=SlurmProvider(
                'broadwl',
                cmd_timeout=60,
                nodes_per_block=1,
                init_blocks=1,
                min_blocks=1,
                max_blocks=1,
                scheduler_options='#SBATCH --cpus-per-task=4 --mem-per-cpu=3g,
                worker_init='worker_init='export PARSL_MEMORY_GB=$(expr $SLURM_CPUS_ON_NODE \* $SLURM_MEM_PER_CPU); export PARSL_CORES=$SLURM_CPUS_ON_NODE',
                exclusive=False
            ),
        )
    ],
)
```

* Change PARSL_GB_MEMORY -> PARSL_MEMORY_GB

* Revert changes to launchers

This reversts the launcher changes introduced in
18061d2.
  • Loading branch information
annawoodard authored Aug 23, 2019
1 parent e0f3269 commit df71e4c
Showing 1 changed file with 9 additions and 2 deletions.
11 changes: 9 additions & 2 deletions parsl/executors/high_throughput/process_worker_pool.py
Original file line number Diff line number Diff line change
Expand Up @@ -123,8 +123,15 @@ def __init__(self,
self.uid = uid
self.block_id = block_id

cores_on_node = multiprocessing.cpu_count()
available_mem_on_node = round(psutil.virtual_memory().available / (2**30), 1)
if os.environ.get('PARSL_CORES'):
cores_on_node = int(os.environ['PARSL_CORES'])
else:
cores_on_node = multiprocessing.cpu_count()

if os.environ.get('PARSL_MEMORY_GB'):
available_mem_on_node = float(os.environ['PARSL_MEMORY_GB'])
else:
available_mem_on_node = round(psutil.virtual_memory().available / (2**30), 1)

self.max_workers = max_workers
self.prefetch_capacity = prefetch_capacity
Expand Down

0 comments on commit df71e4c

Please sign in to comment.