Fix and improve docs (#72)

* Add version command to the CLI and fix a minor issue with the docstrings of wait function in API. * Improve the main readme, render notebooks in readthedocs and add a whole new CLI quickstart to the documentation. * Add a requirements file just for sphinx and readthedocs build. * Update README.rst * Fix a style issue with the main readme file. * Update README.rst * Fix another style issue with the main readme file.
broadinstitute · May 13, 2019 · e688df5 · e688df5
1 parent 75f255d
commit e688df5
Show file tree

Hide file tree

Showing 17 changed files with 702 additions and 43 deletions.
diff --git a/README.rst b/README.rst
@@ -39,7 +39,7 @@ The cromwell_tools Python package is designed to be a Python API and Command Lin
     - Consistency in authentication to work with Cromwell.
     - Consistency between API and CLI interfaces.
     - Sufficient test cases.
-    - User-friendly documentation on `Read The Docs <https://cromwell-tools.readthedocs.io/en/latest/>`_.
+    - Documentation on `Read The Docs <https://cromwell-tools.readthedocs.io/en/latest/>`_.
 
 The accessory scripts and IPython notebooks are useful to:
     - Monitor the resource usages of workflows running in Cromwell.
@@ -48,17 +48,22 @@ The accessory scripts and IPython notebooks are useful to:
 
 Installation
 ============
-From version ``v1.0.0``, you can install cromwell-tools from `PyPI <https://pypi.org/>`_:
+
+1. (optional and highly recommended) Create a Python 3 `virtual environment <https://virtualenv.pypa.io/en/latest/userguide/#usage>`_
+locally and activate it: e.g. ``virtualenv -p python3 myenv && source myenv/bin/activate``
+
+2. Install (or upgrade) Cromwell-tools from `PyPI <https://pypi.org/>`_:
 
 .. code:: bash
 
-    pip install cromwell-tools
+    pip install -U cromwell-tools
 
-To install the testing requirements as extras of the library, use:
+3. You can verify the installation by:
 
 .. code:: bash
 
-    pip install "cromwell-tools[test]"
+    cromwell-tools --version
+
 
 Usage
 =====
@@ -72,7 +77,7 @@ In Python, you can import the package with:
     import cromwell_tools.api as cwt
     cwt.submit(*args)
 
-assuming args is a list of arguments needed. For more details, please check the example `Jupyter Notebooks <https://github.com/broadinstitute/cromwell-tools/tree/master/notebooks/Quickstart>`_.
+assuming args is a list of arguments needed. For more details, please check the example `Quickstart Jupyter Notebook <https://github.com/broadinstitute/cromwell-tools/tree/master/notebooks/Tutorials/Quickstart/quickstart.ipynb>`_.
 
 Commandline Interface
 ---------------------
@@ -87,7 +92,7 @@ This package also installs a command line interface that mirrors the API and is
                       ...
 
     positional arguments:
-      {submit,wait,status,abort,release_hold,query,health,validate}
+      {submit,wait,status,abort,release_hold,query,health}
                             sub-command help
         submit              submit help
         wait                wait help
@@ -99,10 +104,12 @@ This package also installs a command line interface that mirrors the API and is
 
     optional arguments:
       -h, --help            show this help message and exit
+      -V, --version         show program's version number and exit
+
 
+A set of sub-commands to submit, query, abort, release on-hold workflows, wait for workflow completion and determining
+status of jobs are exposed by this CLI.
 
-A set of sub-commands to submit, query, abort, release on-hold workflows, wait for workflow completion, determining
-status of jobs and validate workflow files are exposed by this CLI.
 
 Testing
 =======
@@ -164,3 +171,31 @@ If you really want to manually trigger the linters and formatters on your code,
 Dependencies
 ------------
 When upgrading the dependencies of cromwell-tools, please make sure ``requirements.txt``, ``requirements-test.txt`` and ``setup.py`` are consistent!
+
+Documentation
+-------------
+To edit the docmentation and rebuild it locally, make sure you have `Sphinx <http://www.sphinx-doc.org/en/master/>`_ installed. You might
+also want to install the dependencies for building the docs: ``pip install requirements-docs.txt``.
+Finally from within the root directory, run:
+
+.. code::
+
+    sphinx-build -b html docs/ docs/_build/
+
+and then you could preview the built documentation by opening ``docs/_build/index.html`` in your web browser.
+
+
+Publish on PyPI
+---------------
+To publish a new version of Cromwell-tools on PyPI:
+
+1. Make sure you have an empty ``dist`` folder locally.
+2. Make sure you have ``twine`` installed: ``pip install twine``.
+3. Build the package: ``python setup.py sdist bdist_wheel``
+4. Upload and publish on PyPI: ``twine upload dist/* --verbose``, note you will need the username and password of the development account to finish this step.
+
+
+Contribute
+==========
+
+Coming soon... For now, feel free to submit issues and open a PR, we will try our best to address them.
diff --git a/cromwell_tools/cli.py b/cromwell_tools/cli.py
@@ -2,13 +2,19 @@
 import requests
 from cromwell_tools.cromwell_api import CromwellAPI
 from cromwell_tools.cromwell_auth import CromwellAuth
+from cromwell_tools import __version__
 
 
 def parser(arguments=None):
     # TODO: dynamically walk through the commands and automatcally create parsers here
 
     main_parser = argparse.ArgumentParser()
 
+    # Check the installed version of Cromwell-tools
+    main_parser.add_argument(
+        '-V', '--version', action='version', version=f'%(prog)s {__version__}'
+    )
+
     subparsers = main_parser.add_subparsers(help='sub-command help', dest='command')
 
     # sub-commands of cromwell-tools

diff --git a/cromwell_tools/cromwell_api.py b/cromwell_tools/cromwell_api.py
@@ -238,14 +238,14 @@ def wait(
         one of the workflows fails or is aborted.
 
         Args:
-        workflow_ids (List[str]): A list of workflow ids to wait for terminal status.
-        timeout_minutes (int): Maximum number of minutes to wait. (default 120)
-        auth (cromwell_tools._cromwell_auth.CromwellAuth): Authentication class holding headers
-            or auth information to a Cromwell server.
-        poll_interval_seconds (Optional[int]): Number of seconds between checks for workflow
-            completion. (default 30)
-        verbose (Optional[bool]): If True, report to stdout when all workflows succeed.
-            (default True)
+            workflow_ids (List[str]): A list of workflow ids to wait for terminal status.
+            timeout_minutes (int): Maximum number of minutes to wait. (default 120)
+            auth (cromwell_tools._cromwell_auth.CromwellAuth): Authentication class holding headers
+                or auth information to a Cromwell server.
+            poll_interval_seconds (Optional[int]): Number of seconds between checks for workflow
+                completion. (default 30)
+            verbose (Optional[bool]): If True, report to stdout when all workflows succeed.
+                (default True)
         """
         start = datetime.now()
         timeout = timedelta(minutes=int(timeout_minutes))

diff --git a/docs/API/index.rst b/docs/API/index.rst
@@ -0,0 +1,34 @@
+.. _API_Documentation:
+
+API Documentation
+=================
+
+Comprehensive reference material for most of the public API exposed by Cromwell-tools:
+
+cromwell_tools.cromwell_api
+---------------------------
+
+.. automodule:: cromwell_tools.cromwell_api
+   :members:
+   :inherited-members:
+
+cromwell_tools.cromwell_auth
+----------------------------
+
+.. automodule:: cromwell_tools.cromwell_auth
+   :members:
+   :inherited-members:
+
+cromwell_tools.utilities
+------------------------
+
+.. automodule:: cromwell_tools.utilities
+   :members:
+   :inherited-members:
+
+cromwell_tools.cli
+------------------
+
+.. automodule:: cromwell_tools.cli
+   :members:
+   :inherited-members:
diff --git a/docs/README_SYMLINK.rst b/docs/README_SYMLINK.rst
@@ -0,0 +1 @@
+.. include:: ../README.rst
diff --git a/...books/Quickstart/Examples/hello_world.wdl → ...rials/Quickstart/Examples/hello_world.wdl b/...books/Quickstart/Examples/hello_world.wdl → ...rials/Quickstart/Examples/hello_world.wdl
diff --git a/notebooks/Quickstart/Examples/helloworld.wdl → ...orials/Quickstart/Examples/helloworld.wdl b/notebooks/Quickstart/Examples/helloworld.wdl → ...orials/Quickstart/Examples/helloworld.wdl
diff --git a/notebooks/Quickstart/Examples/inputs.json → ...Tutorials/Quickstart/Examples/inputs.json b/notebooks/Quickstart/Examples/inputs.json → ...Tutorials/Quickstart/Examples/inputs.json
diff --git a/docs/Tutorials/Quickstart/Examples/labels.json b/docs/Tutorials/Quickstart/Examples/labels.json
@@ -0,0 +1,4 @@
+{
+  "project_shortname": "testing workflow",
+  "comment": "This is just an example labels file!"
+}
diff --git a/docs/Tutorials/Quickstart/Examples/options.json b/docs/Tutorials/Quickstart/Examples/options.json
@@ -0,0 +1,5 @@
+{
+  "read_from_cache": true,
+  "write_to_cache": true,
+  "backend": "PAPIv2"
+}
diff --git a/notebooks/Quickstart/quickstart.ipynb → ...Tutorials/Quickstart/api_quickstart.ipynb b/notebooks/Quickstart/quickstart.ipynb → ...Tutorials/Quickstart/api_quickstart.ipynb
@@ -4,7 +4,11 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Quickstart"
+    "# Cromwell-tools Python API Quickstart\n",
+    "\n",
+    "This notebook will help you get familiar with the Cromwell-tools' Python API client. After walking through this notebook \n",
+    "you should be able to import cromwell-tools in your Python application and talk to Cromwell engine using various \n",
+    "authentication methods.\n"
    ]
   },
   {
@@ -306,7 +310,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Submit the job"
+    "### Submit the job"
    ]
   },
   {
@@ -356,7 +360,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Check the status of the job"
+    "### Check the status of the job"
    ]
   },
   {
@@ -385,7 +389,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Release the job"
+    "### Release the job"
    ]
   },
   {
@@ -428,8 +432,17 @@
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
    "version": "3.6.4"
+  },
+  "pycharm": {
+   "stem_cell": {
+    "cell_type": "raw",
+    "source": [],
+    "metadata": {
+     "collapsed": false
+    }
+   }
   }
  },
  "nbformat": 4,
  "nbformat_minor": 2
-}
+}