Getting Started

This page will help you to set up and run a basic test case using the ufs-weather-model, which is used in multiple UFS applications.

If you would like to engage in development or run a full set of regression tests, you will need to refer to the UFS Weather Model (WM) User's Guide.

The UFS WM repository does not include a workflow, pre-processing (preparation of inputs), or post-processing. Such workflows are available through the application-level repositories in the ufs-community organization, such as the UFS Short-Range Weather Application. Inclusion of workflows makes the model configuration much easier to modify. If you would like to change the resolution, physics package, initial conditions, output frequency, or other aspects of the model, it is recommended that you use an application. The UFS Short-Range Weather Application v2.2.0, which was released in October 2023, is a good starting place.

If you encounter problems, please check the ufs-weather-model issues page. You can also submit a question to the UFS WM Discussions Q&A.

1. Download Prerequisite Libraries

The UFS Weather Model requires a number of other libraries for it to compile. If you are working on a preconfigured/Tier-1 platform, the prerequisite libraries have been built and are available there (see list of WM Tier-1 platforms). If you are NOT working on a pre-configured platform, then you will need to build the necessary libraries.

The starting point for downloading and building prerequisite libraries is the spack-stack documentation. Users can build all prerequisite libraries from NCEPLIBS and NCEPLIBS-external as a bundle using spack-stack.

2. Download the Model Source Code

Check Git Configuration

The model source code is hosted on the ufs-weather-model repository on GitHub. Check that git is installed on your machine and that you can reach GitHub. See Set Up Git and Getting Started with GitHub for more details.

Where to download/install

All files need to be visible to the compute nodes on your machine. Please go to the appropriate working directory for your system and project.

% cd <working_directory>

Clone the ufs-weather-model Repository

You can now clone the ufs-weather-model repository:

git clone --recursive https://github.com/ufs-community/ufs-weather-model.git
cd ufs-weather-model

The above commands do not require a GitHub account, but it is strongly recommended that you have one so you can save your own modifications. A GitHub account is required if you wish to contribute code back to the ufs-weather-model repository.

3. Download Data

The UFS Weather Model maintains regression test (RT) data on all Tier-1 systems and in a publicly available data bucket. Users on Tier-1 systems do not need to download data for existing RTs. Users on non-Tier-1 systems can find and download data for the most recent version of the RTs in the data bucket. For example, to run the control_c48 test on a system with Intel compilers, users could run:

mkdir data
cd data
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/atmf000.nc
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/atmf024.nc
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/sfcf000.nc
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/sfcf024.nc

mkdir RESTART
cd RESTART
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/RESTART/20210323.060000.coupler.res
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/RESTART/20210323.060000.fv_core.res.nc
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/RESTART/20210323.060000.fv_core.res.tile#.nc
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/RESTART/20210323.060000.fv_srf_wnd.res.tile#.nc
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/RESTART/20210323.060000.fv_tracer.res.tile#.nc
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/RESTART/20210323.060000.phy_data.tile#.nc
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/RESTART/20210323.060000.sfc_data.tile#.nc

Note that users will need to replace develop-20240301 in the path with the date for the data they wish to download, and the wget commands ending in tile#.nc will each need to be run 6 times, replacing tile#.nc with tile1.nc, tile2.nc, etc., through tile6.nc.

This test case will run a global C48 grid configuration for a 24 hour forecast period.

4. Run the Test Case

Configure the Test(s)

First, navigate to the tests directory and create a file (e.g., my_rt.conf) to add the test(s) you want to run:

cd tests/
touch my_rt.conf

Then, edit that file by adding information on the test(s) you want to run. For example:

vim my_rt.conf

Then press the i key (for insert) and paste compile and run information into the file. This example uses the control_c48 test:

### ATM tests ###
COMPILE | atm_dyn32 | intel | -DAPP=ATM -DCCPP_SUITES=FV3_GFS_v16 -D32BIT=ON | | fv3 |
RUN | control_c48  | + <machine>   | baseline |

where <machine> refers to your machine name. Valid machine names are: Acorn, WCOSS2, Gaea, Hera, Jet, S4, Orion, Hercules, Derecho, Stampede, Expanse, Discover, NOAA Cloud. Press the esc key, and then type :wq to save and quit the file editor.

In rt.sh, users may also need to modify dprefix so that the script creates experiment directories in the correct location.

Users on other systems will need to make additional adjustments to several files in the tests directory before running rt.sh, including: rt.sh, run_test.sh, detect_machine.sh, default_vars.sh, fv3_conf/fv3_slurm.IN_*, fv3_conf/compile_slurm.IN_*, compile.sh, and module-setup.sh. In any file that has machine-specific logic, the user will need to add an if block to rt.sh with logic for what to do on their system. Additionally, users will need to adjust DISKNM in rt.sh to set the path to the RT data on their system.

Run the Test(s)

Simply run:

./rt.sh -a <account> -l my_rt.conf

where <account> is the account name you use to submit jobs on your system.

Check the Output

Coming Soon!

Resources

If your computing system includes a job scheduler, such as Portable Batch System (PBS), Slurm Workload Manager (Slurm), or Platform Load Sharing Facility (LSF) then follow instructions for Systems with Job Schedulers. If your system does not include a job scheduler then follow instructions for Systems without Job Schedulers.

Systems with Job Schedulers

If your system includes a job scheduler, run the global C48 test on 8 mpi tasks via submission to the job scheduler.

If you are not already set up for this, you can download an appropriate template file based on your job scheduler.

Job Scheduler	Template
PBS	% wget https://raw.githubusercontent.com/wiki/ufs-community/ufs-weather-model/tools/ufs_simple_test_case.qsub.template
Slurm	% wget https://raw.githubusercontent.com/wiki/ufs-community/ufs-weather-model/tools/ufs_simple_test_case.sbatch.template
LSF	% wget https://raw.githubusercontent.com/wiki/ufs-community/ufs-weather-model/tools/ufs_simple_test_case.bsub.template

You will need to edit placeholders in the template file.

Placeholder	Description
__ACCOUNT__	Project or group identifier
__QUEUE_OR_QOS__	Queue or Quality of Service (e.g. regular, debug)
__PATH_TO_MODULEFILE__	Path to modulefile
__MODULEFILE__	Modulefile used to setup run (often the modulefile used when compiling )
__MPIEXEC__	Program executing MPI processes (e.g. mpiexec, mpiexec_mpt, mpirun, aprun)

Once this is done you can submit the job. The instructions for each type of job scheduler are below.

Job Scheduler	Template
PBS	% qsub ufs_simple_test_case.qsub.template
Slurm	% sbatch ufs_simple_test_case.sbatch.template
LSF	% bsub ufs_simple_test_case.bsub.template

Standard output and standard error will be directed to a file name 'out'.

Systems without Job Schedulers

If your system does not include a job scheduler, run the C48 test configuration on 8 processors using the appropriate MPI program (e.g., mpiexec, mpiexec_mpt, mpirun, aprun).

% mpiexec -np 8 ./ufs_weather_model

Where to Find Outputs

If the run is successful you'll find output files named dynf***.nc (dynamics variables) and phyf***.nc (physics variables) through 024h in the run directory.