Update documentation

docs/developers_guide/api.rst

@@ -148,7 +148,10 @@ step
    :toctree: generated/
 
    Step
+   Step.set_resources
+   Step.constrain_resources
    Step.setup
+   Step.runtime_setup
    Step.run
    Step.add_input_file
    Step.add_output_file
@@ -222,6 +225,9 @@ parallel
    :toctree: generated/
 
    get_available_cores_and_nodes
+   check_parallel_system
+   set_cores_per_node
+   run_command
 
 provenance
 ^^^^^^^^^^

docs/developers_guide/framework.rst

@@ -3,8 +3,8 @@
 Framework
 =========
 
-All of the :ref:`dev_packages` that are not in the two cores (``landice`` and
-``ocean``) belong to the ``compass`` framework.  Some of these
+All of the :ref:`dev_packages` that are not in the two MPAS cores (``landice``
+and ``ocean``) belong to the ``compass`` framework.  Some of these
 modules and packages are used by the :ref:`dev_command_line`, while others are
 meant to be called within test cases and steps to simplify tasks like adding
 input and output files, downloading data sets, building up config files,
@@ -62,13 +62,15 @@ work directory.  Setting up a test suite includes setting up the test cases
 (see :ref:`dev_setup`), writing out a :ref:`dev_provenance` file, and saving
 a pickle file containing a python dictionary that defines the test suite for
 later use by ``compass run``.  The "target" and "minimum" number of cores
-required for running the test suite are displayed.  The "target" is the maximum
-of the ``cores`` attribute of all steps in the test suite.  This is the number
-of cores to run on to complete the test suite as quickly as possible, with the
+required for running the test suite are displayed.  The "target" is determined
+based on the maximum product of the ``ntasks`` and ``cpus_per_task``
+attributes of each step in the test suite.  This is the number of cores to run
+on to complete the test suite as quickly as possible, with the
 caveat that many cores may sit idle for some fraction of the runtime.  The
-"minimum" number of cores is the maximum of the ``min_cores`` attribute for
-all steps int he suite, indicating the fewest cores that the test may be run
-with before at least some steps in the suite will fail.
+"minimum" number of cores is the maximum of the product of the ``min_tasks``
+and ``min_cpus_per_task``` attribute for all steps in the suite, indicating the
+fewest cores that the test may be run with before at least some steps in the
+suite will fail.
 
 .. _dev_run:
 
@@ -347,9 +349,9 @@ function first updates the namelist options associated with the
 `PIO library <https://ncar.github.io/ParallelIO/>`_ and partitions the mesh
 across MPI tasks, as we will discuss in a moment, before running the model.
 You can provide non-default names for the graph, namelist and streams files.
-The number of cores and threads is determined from the ``cores``, ``min_cores``
-and ``threads`` attributes of the step object, set in its
-constructor or :ref:`dev_step_setup` method (i.e. before calling
+The number of cores and threads is determined from the ``ntasks``,
+``min_tasks``, ``cpus_per_task``, `min_cpus_per_task`` and ``openmp_threads``
+attributes of the step object, set in its constructor or :ref:`dev_step_setup` method (i.e. before calling
 :ref:`dev_step_run`) so that the ``compass`` framework can ensure that the
 required resources are available.
 
@@ -358,7 +360,7 @@ Partitioning the mesh
 
 The function :py:func:`compass.model.partition()` calls the graph partitioning
 executable (`gpmetis <https://arc.vt.edu/userguide/metis/>`_ by default) to
-divide up the MPAS mesh across cores.  If you call
+divide up the MPAS mesh across MPI tasks.  If you call
 :py:func:`compass.model.run_model()` with `partition_graph=True` (the default),
 this function is called automatically.
 

docs/developers_guide/ocean/test_groups/baroclinic_channel.rst

@@ -118,14 +118,14 @@ a python dictionary:
 
 .. code-block:: python
 
-    res_params = {'1km': {'cores': 144, 'min_cores': 36},
-                  '4km': {'cores': 36, 'min_cores': 8},
-                  '10km': {'cores': 8, 'min_cores': 4}}
+    res_params = {'1km': {'ntasks': 144, 'min_tasks': 36},
+                  '4km': {'ntasks': 36, 'min_tasks': 8},
+                  '10km': {'ntasks': 8, 'min_tasks': 4}}
 
     if resolution not in res_params:
         raise ValueError(
-            'Unsupported resolution {}. Supported values are: '
-            '{}'.format(resolution, list(res_params)))
+            f'Unsupported resolution {resolution}. Supported values are: '
+            f'{list(res_params)}')
 
     params = res_params[resolution]
 
@@ -134,12 +134,12 @@ constructors when they are added to the test case:
 
 .. code-block:: python
 
-            step = Forward(
-                test_case=self, name=name, subdir=name, cores=params['cores'],
-                min_cores=params['min_cores'], resolution=resolution,
-                nu=float(nu))
-            ...
-            self.add_step(step)
+        step = Forward(
+            test_case=self, name=name, subdir=name,
+            ntasks=params['ntasks'], min_tasks=params['min_tasks'],
+            resolution=resolution, nu=float(nu))
+        ...
+        self.add_step(step)
 
 The ``analysis`` step defined by
 :py:class:`compass.ocean.tests.baroclinic_channel.rpe_test.analysis.Analysis`

docs/developers_guide/ocean/test_groups/global_ocean.rst

@@ -222,9 +222,9 @@ time integrator (``split-explicit`` or ``RK4``).  Here is an example from the
 As in the example above, these are typically passed along from the arguments
 to the the test case's own constructor.
 
-Performance-related parameters---``cores``, ``min_cores``, and
-``threads``---can be passed as optional arguments, but they are more typically
-read from the corresponding ``forward_<param>`` config options in the
+Performance-related parameters---``ntasks``, ``min_tasks``, and
+``openmp_threads``---can be passed as optional arguments, but they are more
+typically read from the corresponding ``forward_<param>`` config options in the
 ``global_ocean`` section of the config file.  This lets users update these
 values as appropriate if the machine and/or mesh defaults aren't quite right
 for them.
@@ -236,11 +236,12 @@ namelist replacements and streams files can be added in the test case
 before adding the step, as in the example above.
 
 The MPAS model is linked in as in input to the step in the ``setup()`` method,
-which also updates the ``self.cores``, ``self.min_cores`` and ``self.threads``
-attributes from config options if they have not been set explicitly in the
-constructor.  Then, in the ``run()`` method, it runs MPAS-Ocean (including
-updating PIO namelist options and generating a graph partition), then
-:ref:`global_ocean_metadata` is added to the output NetCDF files.
+which also updates the ``self.ntasks``, ``self.min_tasks`` and
+``self.openmp_threads`` attributes from config options if they have not been
+set explicitly in the constructor.  Then, in the ``run()`` method, it runs
+MPAS-Ocean (including updating PIO namelist options and generating a graph
+partition), then :ref:`global_ocean_metadata` is added to the output NetCDF
+files.
 
 .. _dev_ocean_global_ocean_testcases:
 

docs/developers_guide/ocean/test_groups/internal_wave.rst

@@ -68,8 +68,8 @@ constructors when they are added to the test case:
 .. code-block:: python
 
             step = Forward(
-                test_case=self, name=name, subdir=name, cores=cores,
-                min_cores=min_cores, nu=float(nu))
+                test_case=self, name=name, subdir=name, ntasks=4,
+                openmp_threads=1, nu=float(nu))
             ...
             self.add_step(step)
 

docs/developers_guide/ocean/test_groups/ziso.rst

@@ -58,31 +58,31 @@ and the time steps for each resolution:
 .. code-block:: python
 
     res_params = {'20km': {'cores': 20,
-                           'min_cores': 2,
+                           'min_tasks': 2,
                            'cores_with_particles': 32,
                            'min_cores_with_particles': 12,
                            'dt': "'00:12:00'",
                            'btr_dt': "'00:00:36'",
                            'mom_del4': "5.0e10",
                            'run_duration': "'0000_00:36:00'"},
                   '10km': {'cores': 80,
-                           'min_cores': 8,
+                           'min_tasks': 8,
                            'cores_with_particles': 130,
                            'min_cores_with_particles': 50,
                            'dt': "'00:06:00'",
                            'btr_dt': "'00:00:18'",
                            'mom_del4': "6.25e9",
                            'run_duration': "'0000_00:18:00'"},
                   '5km': {'cores': 300,
-                          'min_cores': 30,
+                          'min_tasks': 30,
                           'cores_with_particles': 500,
                           'min_cores_with_particles': 200,
                           'dt': "'00:03:00'",
                           'btr_dt': "'00:00:09'",
                           'mom_del4': "7.8e8",
                           'run_duration': "'0000_00:09:00'"},
                   '2.5km': {'cores': 1200,
-                            'min_cores': 120,
+                            'min_tasks': 120,
                             'cores_with_particles': 2100,
                             'min_cores_with_particles': 900,
                             'dt': "'00:01:30'",