Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update instructions: nuke VDI and OOD; add ARE #311

Closed
wants to merge 12 commits into from
61 changes: 41 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,44 +5,65 @@
<img alt="latest docs" src="https://img.shields.io/badge/docs-latest-blue.svg">
</a>

<a href="https://anaconda.org/coecms/cosima-cookbook">
<img src="https://anaconda.org/coecms/cosima-cookbook/badges/version.svg" />
</a>

# cosima-cookbook

The COSIMA Cookbook is a framework for analysing output from ocean-sea ice models. The focus is on the ACCESS-OM2 suite of models being developed and run by members of [COSIMA: Consortium for Ocean-Sea Ice Modelling in Australia](http://cosima.org.au). But this framework is suited to analysing any MOM5/MOM6 output, as well as output from other models.
The COSIMA Cookbook is a framework for analysing output from ocean-sea ice models. The focus is on
the ACCESS-OM2 suite of models being developed and run by members of [COSIMA: Consortium for Ocean-Sea
Ice Modelling in Australia](http://cosima.org.au). But this framework is suited to analysing any MOM5/MOM6
output, as well as output from other models.

The cookbook is structured as follows:
* This repository includes boiler-plate code and scripts that underpin the cookbook.
* The [`cosima-recipes`](https://github.com/COSIMA/cosima-recipes) repository includes example notebooks on which you can base your analyses.
* The [`cosima-recipes` template](https://github.com/COSIMA/cosima-recipes/blob/master/Tutorials/Template_For_Notebooks.ipynb) provides you with a template if you want to contribute your own scripts to the analysis.
* The [`cosima-recipes`](https://github.com/COSIMA/cosima-recipes) repository includes example
notebooks on which you can base your analyses.
* The [`cosima-recipes` template](https://github.com/COSIMA/cosima-recipes/blob/master/Tutorials/Template_For_Notebooks.ipynb)
provides you with a template if you want to contribute your own scripts to the analysis.


## Getting Started

The easiest way to use the COSIMA Cookbook is through NCI's HPC systems (either VDI or Gadi). The cookbook is preinstalled in the latest `conda/analysis3` environment.
The easiest way to use the COSIMA Cookbook is through NCI's HPC systems (either via the Australian
Research Environment (ARE), Open OnDemand (OOD), or on Gadi). The cookbook is preinstalled in the
navidcy marked this conversation as resolved.
Show resolved Hide resolved
latest `conda/analysis3` environment.

Once you have an NCI account you can connect via

Once you have an account on the VDI, you should:
1. Clone the [`cosima-recipes`](https://github.com/COSIMA/cosima-recipes) repository to your local file space.
2. Start a jupyter notebook session using the following commands:
```
>> module use /g/data/hh5/public/modules/
>> module load conda/analysis3-unstable
>> jupyter notebook
```
3. Navigate to one of the COSIMA recipes and run the analysis.

Alternatively, you might prefer to download `vdi_jupyter` or the `gadi_jupyter` scripts hosted in the CLEx CMS Github Repository [coecms/nci_scripts](https://github.com/coecms/nci_scripts). These scripts will allow you to open a Jupyter notebook in your local browser window.
- the Australian Research Environment (ARE) at https://are.nci.org.au

Alternatively, you might prefer to download the `gadi_jupyter` scripts hosted in the CLEx CMS Github
Repository [coecms/nci_scripts](https://github.com/coecms/nci_scripts). Running the script will open
a Jupyter notebook in your local browser window.

**Note**: Access to COSIMA ocean-sea ice model output requires you are a member of NCI projects `hh5`
and `ik11`, and potentially also `cj50` and `jk72`.


## Using the Cookbook

The COSIMA Cookbook relies on several components:
1. There needs to be a database of simulations -- on the NCI system, model output that is stored in the COSIMA space on the `/g/data/ik11/` directory.
2. Once you have access to data, the best place to start is the [`cosima-recipes`](https://github.com/COSIMA/cosima-recipes) repository which includes a series of jupyter notebooks containing examples that guide you through to use the cookbook to load model output and then proceed doing simple (or elaborate) computations. The best starting point of exploring the [`cosima-recipes`](https://github.com/COSIMA/cosima-recipes) is the [Documented Examples](https://cosima-recipes.readthedocs.io/en/latest/documented_examples.html). A collection of useful examples leveraging the `cosima-cookbook` is also found [here](https://github.com/COSIMA/ACCESS-OM2-1-025-010deg-report/tree/master/figures).

1. There needs to be a database of simulations -- on the NCI system, model output that is stored in the COSIMA
space on the `/g/data/ik11/` directory.

2. Once you have access to data, the best place to start is the [`cosima-recipes`](https://github.com/COSIMA/cosima-recipes)
repository which includes a series of jupyter notebooks containing examples that guide you through to use the cookbook to
load model output and then proceed doing simple (or elaborate) computations. The best starting point of exploring the
[`cosima-recipes`](https://github.com/COSIMA/cosima-recipes) is the [Tutorials](https://cosima-recipes.readthedocs.io/en/latest/tutorials/index.html)
and the [Documented Examples](https://cosima-recipes.readthedocs.io/en/latest/documented_examples/index.html). A collection
of useful examples leveraging the `cosima-cookbook` that were used in the [ACCESS-OM2 model announcement paper](https://doi.org/10.5194/gmd-13-401-2020)
can be found at [COSIMA/ACCESS-OM2-1-025-010deg-report](https://github.com/COSIMA/ACCESS-OM2-1-025-010deg-report/tree/master/figures)
repository.

## Contributing to the Cookbook

If you like the cookbook, you may like to interact more closely with us:
* Contributions of new notebooks or analysis scripts are always welcome. Please check out the [`cosima-recipes`](https://github.com/COSIMA/cosima-recipes) repository.
* Contributions of new notebooks or analysis scripts are always welcome. Please check out the
[`cosima-recipes`](https://github.com/COSIMA/cosima-recipes) repository.
* If you find a problem, or have a suggestion for improvement, please log an issue.
* All code submitted as part of the `cosima-cookbook` itself must be formatted with [black](https://github.com/psf/black)

Expand All @@ -52,7 +73,7 @@ We request that users of ACCESS-OM2 model [code](https://github.com/COSIMA/acces
1. consider citing Kiss et al. (2020) ([http://doi.org/10.5194/gmd-13-401-2020](http://doi.org/10.5194/gmd-13-401-2020))
2. include an acknowledgement such as the following:

*The authors thank the Consortium for Ocean-Sea Ice Modelling in Australia (COSIMA; [http://www.cosima.org.au](http://www.cosima.org.au)) for making the ACCESS-OM2 suite of models available at [https://github.com/COSIMA/access-om2](https://github.com/COSIMA/access-om2).*
3. let us know of any publications which use these models or data so we can add them to [our list](https://scholar.google.com/citations?hl=en&user=inVqu_4AAAAJ).
> The authors thank the Consortium for Ocean-Sea Ice Modelling in Australia (COSIMA; [www.cosima.org.au](https://cosima.org.au)) for making the ACCESS-OM2 suite of models available at [github.com/COSIMA/access-om2](https://github.com/COSIMA/access-om2).

[![Documentation Status](https://readthedocs.org/projects/cosima-cookbook/badge/?version=latest)](https://cosima-cookbook.readthedocs.org/en/latest)
3. let us know of any publications which use these models or data so we can add them to
[our list](https://scholar.google.com/citations?hl=en&user=inVqu_4AAAAJ).
30 changes: 9 additions & 21 deletions docs/source/getting_started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,53 +2,41 @@
Getting Started
===============

The cookbook consists of a Python 3 package that contains infrastructure
for indexing COSIMA model output and convenient methods for searching for
The `cosima_cookbook` is a Python 3 package that contains infrastructure
for indexing ocean-sea ice model output and convenient methods for searching for
and loading the data into `xarray <http://xarray.pydata.org/>`_ datastructures.

Some users may find it sufficient to browse through the examples and tutorials
in the `COSIMA recipes <http://cosima-recipes.readthedocs.io/>`_ repository.
The Jupyter notebooks that can be downloaded from COSIMA recipes need this package
(called cosima_cookbook) to be installed.
(called `cosima_cookbook`) to be installed.

Choosing your platform
======================

COSIMA ocean and ice models are typically run on `NCI <nci.org.au>`_, a HPC
computing centre in Australia. The output data is very large and it is
computing centre in Australia. The output data is very large and it is
assumed that this data resides on a NCI storage system.

The cookbook is supported on two NCI systems
The cookbook is supported on NCI systems:

#. `Virtual Desktop Infrastructure (VDI) <http://nci.org.au/services/vdi/>`_
#. `Australian Research Environment (ARE) <https://are.nci.org.au/>`_
#. `gadi (gadi.nci.org.au) <http://nci.org.au/systems-services/peak-system/gadi/>`_
navidcy marked this conversation as resolved.
Show resolved Hide resolved

Connecting
==========

For both VDI and gadi scripts are used to start a `jupyter notebook <http://jupyter-notebook.readthedocs.io>`_
For gadi, scripts used to start a `jupyter notebook <http://jupyter-notebook.readthedocs.io>`_
or `jupyter lab <http://jupyterlab.readthedocs.io>`_ session on the chosen system
navidcy marked this conversation as resolved.
Show resolved Hide resolved
and automatically create an `ssh tunnel <https://www.ssh.com/ssh/tunneling/>`_
such that the jupyter session can be opened in your local browser using a url
like <http://localhost:8888> that appears to be on your own local machine.

Scripts for this purpose are provided by the CLEX CMS team in this repository

like <http://localhost:8888> that appears to be on your own local machine. Scripts for
this purpose are provided by the CLEX CMS team in at the repository
navidcy marked this conversation as resolved.
Show resolved Hide resolved
https://github.com/coecms/nci_scripts

Clone the repository to your local computer. There are instructions in the repository
on the requirements for each script and how to use them.

Alternatively if you are using the VDI Strudel environment and accessing the VDI
through a virtual desktop you can load the same python conda environment that is
used in the scripts above and start a jupyter notebook session like so:
::

module use /g/data3/hh5/public/modules
module load conda/analysis3

jupyter notebook

Finding data
============

Expand Down
Loading