Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stack of updates to docs in preparation for 1.0 #1719

Merged
merged 18 commits into from
May 22, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 11 additions & 7 deletions docs/quickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ $ gcc --version

Parsl has been tested on Linux and MacOS.

.. note:: As of Parsl v0.7.2 we are switching to an opt-in model for anonymous usage tracking. To help support the
.. note:: Parsl uses an opt-in model for anonymous usage tracking. To help support the
Parsl project we request that users opt-in where possible by setting ``PARSL_TRACKING=true`` in their environment
or by setting ``usage_tracking=True`` in the configuration object (`parsl.config.Config`). To read more about
what information is collected and how it is used see :ref:`label-usage-tracking`.
Expand All @@ -26,7 +26,7 @@ Installation using Pip
^^^^^^^^^^^^^^^^^^^^^^

While ``pip`` and ``pip3`` can be used to install Parsl we suggest the following approach
for reliable installation when many Python environments are avaialble.
for reliable installation when many Python environments are available.

1. Install Parsl::

Expand All @@ -53,9 +53,9 @@ Installation using Conda
2. Install Parsl::

$ python3 -m pip install parsl

or

$ conda config --add channels conda-forge
$ conda install parsl

Expand All @@ -71,11 +71,15 @@ can be installed easily via `pip` using a pip extras option.
Here's a list of the components and their extras option:

* Amazon Web Services (Cloud) : `aws`
* OAuth based SSH : `oauth_ssh`
* Logging monitoring data to a database: `monitoring`
* Extreme Scale Executor (Supercomputing) : `extreme_scale`
* Google Cloud : `google_cloud`
* Python GSSAPI for SSH : `gssapi`
* Azure (Cloud) : `azure`
* Kubernetes : `kubernetes`
* Extreme Scale Executor (Supercomputing) : `extreme_scale`
* Logging monitoring data to a database: `monitoring`
yadudoc marked this conversation as resolved.
Show resolved Hide resolved
* Jetstream (NSF Cloud) : `jetstream`
* Work Queue execution framework : `workqueue`

yadudoc marked this conversation as resolved.
Show resolved Hide resolved

Optional extras can be installed using the following syntax::

Expand Down
1 change: 1 addition & 0 deletions docs/userguide/app_caching.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ two invocations are the same. This can be useful when generating log file
names automatically based on time or run information. The names of keyword
arguments to ignore can be specified as an ``ignore_for_cache``
parameter to the decorator:

.. code-block:: python
@bash_app(cache=True, ignore_for_cache=['stdout'])
Expand Down
309 changes: 155 additions & 154 deletions docs/userguide/configuring.rst

Large diffs are not rendered by default.

19 changes: 15 additions & 4 deletions docs/userguide/examples/config.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,18 @@
import parsl
from parsl.config import Config
from parsl.executors.threads import ThreadPoolExecutor
from parsl.channels import LocalChannel
from parsl.executors import HighThroughputExecutor
from parsl.providers import LocalProvider

local_threads = Config(
executors=[ThreadPoolExecutor(max_threads=4)]
htex_config = Config(
executors=[
HighThroughputExecutor(
label="htex_local",
cores_per_worker=1,
provider=LocalProvider(
channel=LocalChannel(),
init_blocks=1,
max_blocks=1,
),
)
],
)
1 change: 1 addition & 0 deletions docs/userguide/examples/library.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from parsl import python_app


@python_app
def increment(x):
return x + 1
5 changes: 2 additions & 3 deletions docs/userguide/examples/run_increment.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
import parsl
from config import local_threads
from config import htex_config
from library import increment

parsl.load(local_threads)
parsl.load(htex_config)

for i in range(5):
print('{} + 1 = {}'.format(i, increment(i).result()))

13 changes: 5 additions & 8 deletions docs/userguide/exceptions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Failures might occur for various reasons:

Since Parsl tasks are executed asynchronously, it can be difficult to determine
where to place exception handling code in the workflow.
In Parsl all exceptions are associated with the task futures.
In Parsl all exceptions are associated with the task futures.
These exceptions are raised only when a result is called on the future
of a failed task. For example:

Expand Down Expand Up @@ -64,12 +64,11 @@ The following example shows how the number of retries can be set to 2:

.. code-block:: python
from parsl import load
from parsl.tests.configs.local_threads import config
import parsl
from parsl.configs.htex_local import config
config.retries = 2
load(config)
parsl.load(config)
Expand All @@ -79,7 +78,7 @@ Lazy fail
Parsl implements a lazy failure model through which a workload will continue
to execute in the case that some tasks fail. That is, the workflow
does not halt as soon as it encounters a failure, but continues execution of every
app that is unaffected.
app that is unaffected.

For example:

Expand All @@ -100,5 +99,3 @@ For example:
(F) (F) (!F)
time ----->
5 changes: 0 additions & 5 deletions docs/userguide/modularizing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,6 @@ benefits to be obtained by modularizing the workflow, including:
3. Ease of reuse of components


.. note::
Support for isolating configuration loading and app definition is available since 0.6.0.
Refer: `Issue#50 <https://github.com/Parsl/parsl/issues/50>`_


The following example illustrates how a Parsl project can be organized into modules.

The configuration(s) can be defined in a module or file (e.g., ``config.py``)
Expand Down