diff --git a/README.md b/README.md index a5a243c..12d3f50 100644 --- a/README.md +++ b/README.md @@ -215,8 +215,8 @@ Output will be stored in `$ACCESS_OM_DIR/control/1deg_jra55_ryf/archive` if the If the run crashes you can find diagnostics in `$ACCESS_OM_DIR/control/1deg_jra55_ryf/access-om2.err` and any output from the run will be in `$ACCESS_OM_DIR/control/1deg_jra55_ryf/work`. To rerun after a crash, do `payu sweep` before `payu run` to clear out all the debris from the crash. -## NCI users only: Integration with cosima cookbook -Here's how you can automatically copy your model output to `/g/data3/` to make it available for analysis via the [cosima cookbook](http://cosima-cookbook.readthedocs.io/en/). +## NCI users only: Integration with COSIMA Cookbook +Here's how you can automatically copy your model output to `/g/data3/` to make it available for analysis via the [COSIMA Cookbook](http://cosima-cookbook.readthedocs.io/en/latest). You will need write access to `/g/data3/hh5/tmp/cosima/`. @@ -227,36 +227,29 @@ First check the existing directories in `/g/data3/hh5/tmp/cosima/access-om2/` an mkdir /g/data3/hh5/tmp/cosima/access-om2/1deg_jra55_ryf_spinupN/ ``` -Then copy ``sync_to_gdata.sh`` to your control directory, e.g. -``` -cp /short/v45/amh157/access-om2/control/1deg_jra55_ryf/sync_to_gdata.sh $ACCESS_OM_DIR/control/1deg_jra55_ryf/sync_to_gdata.sh -``` -*TODO: source `sync_to_gdata.sh` from somewhere else?* - -Now edit ``$ACCESS_OM_DIR/control/1deg_jra55_ryf/sync_to_gdata.sh`` to set ``GDATADIR`` to the directory in `g/data3` you created above, e.g. +Now edit ``$ACCESS_OM_DIR/control/1deg_jra55_ryf/sync_output_to_gdata.sh`` to set ``GDATADIR`` to the directory in `g/data3` you created above, e.g. ``` GDATADIR=/g/data3/hh5/tmp/cosima/access-om2/1deg_jra55_ryf_spinupN/ ``` -Finally, edit ``$ACCESS_OM_DIR/control/1deg_jra55_ryf/config.yaml`` to add +Finally, edit `$ACCESS_OM_DIR/control/1deg_jra55_ryf/config.yaml` to uncomment the line ``` -postscript: sync_to_gdata.sh +postscript: sync_output_to_gdata.sh ``` -as the final line. -This will run ``sync_to_gdata.sh`` after each run, automatically rsynching collated output from all previous runs to ``/g/data3/hh5/tmp/cosima/access-om2/1deg_jra55_ryf_spinupN/``, where cosima cookbook can find it and add it to the cookbook database. +This will run `sync_output_to_gdata.sh` after each run, automatically rsynching collated output from all previous runs to `/g/data3/hh5/tmp/cosima/access-om2/1deg_jra55_ryf_spinupN/`, where COSIMA Cookbook can find it and add it to the cookbook database. -In python, you need to use `build_index` to update the cookbook index to see your new runs: +In python, you then need to use `build_index` to update the cookbook index to see your new runs: ```{python} import cosima_cookbook as cc cc.build_index() ``` -The `/g/data3` directory you created above (e.g. `1deg_jra55_ryf_spinupN`) will be the experiment's name in cosima cookbook, i.e. in the list returned by `cc.get_experiments(configuration)`, where `configuration` is the parent directory name (e.g. `access-om2`): +The `/g/data3` directory you created above (e.g. `1deg_jra55_ryf_spinupN`) will be the experiment's name in the COSIMA Cookbook, i.e. in the list returned by `cc.get_experiments(configuration)`, where `configuration` is the parent directory name (e.g. `access-om2`): ```{python} configuration = 'access-om2' expts = cc.get_experiments(configuration) ``` -For further cosima cookbook usage instructions and examples see . +For further COSIMA Cookbook usage instructions and examples see . ## Testing diff --git a/control/01deg_jra55_ryf/config.yaml b/control/01deg_jra55_ryf/config.yaml index 3304823..da80c8b 100644 --- a/control/01deg_jra55_ryf/config.yaml +++ b/control/01deg_jra55_ryf/config.yaml @@ -58,5 +58,6 @@ collate_flags: -n4 -z -m -r runlog: False stacksize: unlimited mpirun: --mca orte_base_help_aggregate 0 --mca btl_openib_eager_limit 4096 --mca btl_openib_max_send_size 8192 --mca mtl mxm -qsub_flags: -lother=hyperthread ipm: 2.0.5 +qsub_flags: -lother=hyperthread -W umask=027 +# postscript: sync_output_to_gdata.sh diff --git a/control/01deg_jra55_ryf/sync_output_to_gdata.sh b/control/01deg_jra55_ryf/sync_output_to_gdata.sh new file mode 100755 index 0000000..4a54b6e --- /dev/null +++ b/control/01deg_jra55_ryf/sync_output_to_gdata.sh @@ -0,0 +1,15 @@ +#!/bin/bash +#PBS -q copyq +#PBS -l ncpus=1 +#PBS -l wd +#PBS -l walltime=1:00:00,mem=2GB +#PBS -P v45 +#PBS -N sync_to_gdata + +# Set this directory to something in /g/data3/hh5/tmp/cosima/ +# (make a unique path for your set of runs) +GDATADIR=/ERROR/SET/GDATADIR/IN/sync_output_to_gdata.sh + +mkdir -p ${GDATADIR} +cd archive +rsync --exclude "*.nc.*" -av --safe-links --no-g output* ${GDATADIR} diff --git a/control/01deg_jra55_ryf/sync_restarts_to_gdata.sh b/control/01deg_jra55_ryf/sync_restarts_to_gdata.sh new file mode 100755 index 0000000..0be20bc --- /dev/null +++ b/control/01deg_jra55_ryf/sync_restarts_to_gdata.sh @@ -0,0 +1,11 @@ +#!/bin/bash +#PBS -q copyq +#PBS -l ncpus=1 +#PBS -l wd +#PBS -l walltime=2:00:00,mem=2GB +#PBS -P v45 +#PBS -N sync_restarts_to_gdata + +source sync_output_to_gdata.sh # to define GDATADIR and cd archive + +rsync -av --safe-links --no-g restart??[05]* ${GDATADIR} diff --git a/control/025deg_jra55_ryf/config.yaml b/control/025deg_jra55_ryf/config.yaml index d570670..271ef75 100644 --- a/control/025deg_jra55_ryf/config.yaml +++ b/control/025deg_jra55_ryf/config.yaml @@ -56,3 +56,4 @@ collate_flags: -n4 -z -m -r collate_mem: 30GB mpirun: --mca orte_base_help_aggregate 0 --mca btl_openib_eager_limit 4096 --mca btl_openib_max_send_size 8192 --mca mtl mxm qsub_flags: -lother=hyperthread -W umask=027 +# postscript: sync_output_to_gdata.sh diff --git a/control/025deg_jra55_ryf/sync_output_to_gdata.sh b/control/025deg_jra55_ryf/sync_output_to_gdata.sh new file mode 100755 index 0000000..4a54b6e --- /dev/null +++ b/control/025deg_jra55_ryf/sync_output_to_gdata.sh @@ -0,0 +1,15 @@ +#!/bin/bash +#PBS -q copyq +#PBS -l ncpus=1 +#PBS -l wd +#PBS -l walltime=1:00:00,mem=2GB +#PBS -P v45 +#PBS -N sync_to_gdata + +# Set this directory to something in /g/data3/hh5/tmp/cosima/ +# (make a unique path for your set of runs) +GDATADIR=/ERROR/SET/GDATADIR/IN/sync_output_to_gdata.sh + +mkdir -p ${GDATADIR} +cd archive +rsync --exclude "*.nc.*" -av --safe-links --no-g output* ${GDATADIR} diff --git a/control/025deg_jra55_ryf/sync_restarts_to_gdata.sh b/control/025deg_jra55_ryf/sync_restarts_to_gdata.sh new file mode 100755 index 0000000..0be20bc --- /dev/null +++ b/control/025deg_jra55_ryf/sync_restarts_to_gdata.sh @@ -0,0 +1,11 @@ +#!/bin/bash +#PBS -q copyq +#PBS -l ncpus=1 +#PBS -l wd +#PBS -l walltime=2:00:00,mem=2GB +#PBS -P v45 +#PBS -N sync_restarts_to_gdata + +source sync_output_to_gdata.sh # to define GDATADIR and cd archive + +rsync -av --safe-links --no-g restart??[05]* ${GDATADIR} diff --git a/control/1deg_core_nyf/config.yaml b/control/1deg_core_nyf/config.yaml index 0dc0905..c9f19dc 100644 --- a/control/1deg_core_nyf/config.yaml +++ b/control/1deg_core_nyf/config.yaml @@ -59,4 +59,5 @@ collate_queue: express collate_ncpus: 1 collate_flags: -n4 -z -m -r mpirun: --mca orte_base_help_aggregate 0 -qsub_flags: -lother=hyperthread +qsub_flags: -lother=hyperthread -W umask=027 +# postscript: sync_output_to_gdata.sh diff --git a/control/1deg_core_nyf/sync_output_to_gdata.sh b/control/1deg_core_nyf/sync_output_to_gdata.sh new file mode 100755 index 0000000..4a54b6e --- /dev/null +++ b/control/1deg_core_nyf/sync_output_to_gdata.sh @@ -0,0 +1,15 @@ +#!/bin/bash +#PBS -q copyq +#PBS -l ncpus=1 +#PBS -l wd +#PBS -l walltime=1:00:00,mem=2GB +#PBS -P v45 +#PBS -N sync_to_gdata + +# Set this directory to something in /g/data3/hh5/tmp/cosima/ +# (make a unique path for your set of runs) +GDATADIR=/ERROR/SET/GDATADIR/IN/sync_output_to_gdata.sh + +mkdir -p ${GDATADIR} +cd archive +rsync --exclude "*.nc.*" -av --safe-links --no-g output* ${GDATADIR} diff --git a/control/1deg_core_nyf/sync_restarts_to_gdata.sh b/control/1deg_core_nyf/sync_restarts_to_gdata.sh new file mode 100755 index 0000000..0be20bc --- /dev/null +++ b/control/1deg_core_nyf/sync_restarts_to_gdata.sh @@ -0,0 +1,11 @@ +#!/bin/bash +#PBS -q copyq +#PBS -l ncpus=1 +#PBS -l wd +#PBS -l walltime=2:00:00,mem=2GB +#PBS -P v45 +#PBS -N sync_restarts_to_gdata + +source sync_output_to_gdata.sh # to define GDATADIR and cd archive + +rsync -av --safe-links --no-g restart??[05]* ${GDATADIR} diff --git a/control/1deg_jra55_ryf/config.yaml b/control/1deg_jra55_ryf/config.yaml index 23f520e..d6e5cf1 100644 --- a/control/1deg_jra55_ryf/config.yaml +++ b/control/1deg_jra55_ryf/config.yaml @@ -59,4 +59,5 @@ collate_queue: express collate_ncpus: 1 collate_flags: -n4 -z -m -r mpirun: --mca orte_base_help_aggregate 0 -qsub_flags: -lother=hyperthread +qsub_flags: -lother=hyperthread -W umask=027 +# postscript: sync_output_to_gdata.sh diff --git a/control/1deg_jra55_ryf/sync_output_to_gdata.sh b/control/1deg_jra55_ryf/sync_output_to_gdata.sh new file mode 100755 index 0000000..4a54b6e --- /dev/null +++ b/control/1deg_jra55_ryf/sync_output_to_gdata.sh @@ -0,0 +1,15 @@ +#!/bin/bash +#PBS -q copyq +#PBS -l ncpus=1 +#PBS -l wd +#PBS -l walltime=1:00:00,mem=2GB +#PBS -P v45 +#PBS -N sync_to_gdata + +# Set this directory to something in /g/data3/hh5/tmp/cosima/ +# (make a unique path for your set of runs) +GDATADIR=/ERROR/SET/GDATADIR/IN/sync_output_to_gdata.sh + +mkdir -p ${GDATADIR} +cd archive +rsync --exclude "*.nc.*" -av --safe-links --no-g output* ${GDATADIR} diff --git a/control/1deg_jra55_ryf/sync_restarts_to_gdata.sh b/control/1deg_jra55_ryf/sync_restarts_to_gdata.sh new file mode 100755 index 0000000..0be20bc --- /dev/null +++ b/control/1deg_jra55_ryf/sync_restarts_to_gdata.sh @@ -0,0 +1,11 @@ +#!/bin/bash +#PBS -q copyq +#PBS -l ncpus=1 +#PBS -l wd +#PBS -l walltime=2:00:00,mem=2GB +#PBS -P v45 +#PBS -N sync_restarts_to_gdata + +source sync_output_to_gdata.sh # to define GDATADIR and cd archive + +rsync -av --safe-links --no-g restart??[05]* ${GDATADIR}