Getting Started

This page will help users set up and run a basic test case using the ufs-weather-model, which is used in multiple UFS applications.

Users who would like to engage in development or run a full set of regression tests will need to refer to the UFS Weather Model (WM) User's Guide.

The UFS WM repository does not include a workflow, pre-processing (preparation of inputs), or post-processing. Such workflows are available through the application-level repositories in the ufs-community organization, such as the UFS Short-Range Weather (SRW) Application. Inclusion of workflows makes the model configuration much easier to modify. It is recommended that users who wish to change the resolution, physics package, initial conditions, output frequency, or other aspects of the model use an application rather than the standalone UFS WM. The UFS SRW Application v2.2.0, which was released in October 2023, is a good starting place.

Users can submit questions to the UFS WM Discussions Q&A. Problems (bugs) requiring a code change can be viewed/reported on the ufs-weather-model Issues page.

1. Download Prerequisites

Note that preconfigured/Tier-1 platforms already include these prerequisites.

Basic System Prerequisites

• POSIX-compliant UNIX-style operating system
• ~27 GB disk space
  • ~23 GB for full spack-stack installation
  • ~1.5 GB for WM clone and build
  • ~0.5 GB input data for a short 24-hour test forecast using the C48 (~200 km) resolution
  • ~2 GB to run the experiment
• Fortran compiler released since 2018
• C and C++ compilers compatible with the Fortran compiler
• Git
• Lmod
• CMake v3.20+ and MPI (e.g., MPICH, OpenMPI)
  • Note that spack-stack can be configured to build these if they are not already on the system.

Prerequisite Software Stack

The UFS Weather Model requires a number of other libraries for it to compile. The prerequisite libraries have been built and are available on preconfigured/Tier-1 platforms (see the list of WM Tier-1 platforms). Users who are NOT working on a pre-configured platform will need to build the necessary libraries.

The starting point for downloading and building prerequisite libraries is the spack-stack documentation. Users can build all prerequisite libraries from NCEPLIBS and NCEPLIBS-external as a bundle using spack-stack.

2. Download the Model Source Code

Check Git Configuration

The model source code is hosted on the ufs-weather-model repository on GitHub. Check that git is installed on your machine and that you can reach GitHub. See Set Up Git and Getting Started with GitHub for more details.

Where to download/install

All files need to be visible to the compute nodes on your machine. Please go to the appropriate working directory for your system and project.

cd <working_directory>

Clone the ufs-weather-model Repository

You can now clone the ufs-weather-model repository:

git clone --recursive https://github.com/ufs-community/ufs-weather-model.git
cd ufs-weather-model

The above commands do not require a GitHub account, but it is strongly recommended that users and developers create one so that they can save any modifications. A GitHub account is required if you wish to contribute code back to the ufs-weather-model repository.

3. Download Data

The UFS Weather Model maintains regression test (RT) data on all Tier-1 systems and in a publicly available data bucket. Users on Tier-1 systems do not need to download data for existing RTs. Users on non-Tier-1 systems can find and download data for the most recent version of the RTs in the data bucket. For example, to run the control_c48 test on a system with Intel compilers, users could run:

mkdir data
cd data
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/atmf000.nc
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/atmf024.nc
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/sfcf000.nc
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/sfcf024.nc

mkdir RESTART
cd RESTART
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/RESTART/20210323.060000.coupler.res
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/RESTART/20210323.060000.fv_core.res.nc
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/RESTART/20210323.060000.fv_core.res.tile#.nc
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/RESTART/20210323.060000.fv_srf_wnd.res.tile#.nc
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/RESTART/20210323.060000.fv_tracer.res.tile#.nc
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/RESTART/20210323.060000.phy_data.tile#.nc
wget https://noaa-ufs-regtests-pds.s3.amazonaws.com/develop-20240301/control_c48_intel/RESTART/20210323.060000.sfc_data.tile#.nc

Note that users will need to replace develop-20240301 in the path with the date for the data they wish to download, and the wget commands ending in tile#.nc will each need to be run 6 times, replacing tile#.nc with tile1.nc, tile2.nc, etc., through tile6.nc. Users can check the RT script they want to run for the list of data files they will need (see, e.g., list of files in the control_c48 LIST_FILES variable).

This test case will run a global C48 grid configuration for a 24-hour forecast period.

4. Run the Test Case

Configure the Test(s)

First, navigate to the tests directory and create a file (e.g., my_rt.conf) to add the test(s) you want to run:

cd tests/
touch my_rt.conf

Then, edit that file by adding information on the test(s) you want to run. For example, run:

vim my_rt.conf

Then press the i key (for insert) and paste compile and run information into the file. This example uses the control_c48 test:

### ATM tests ###
COMPILE | atm_dyn32 | intel | -DAPP=ATM -DCCPP_SUITES=FV3_GFS_v16 -D32BIT=ON | | fv3 |
RUN | control_c48  | + <machine>   | baseline |

where <machine> refers to your machine name. Valid machine names are: Acorn, WCOSS2, Gaea, Hera, Jet, S4, Orion, Hercules, Derecho, Stampede, Expanse, Discover, NOAA Cloud. Press the esc key, and then type :wq to save and quit the file editor.

In rt.sh, users may also need to modify dprefix so that the script creates experiment directories in the correct location.

Users on other systems will need to make additional adjustments to several files in the tests directory before running rt.sh, including: rt.sh, run_test.sh, detect_machine.sh, default_vars.sh, fv3_conf/fv3_slurm.IN_*, fv3_conf/compile_slurm.IN_*, compile.sh, and module-setup.sh. In any file that has machine-specific logic, users will need to add an if block to rt.sh with logic for what to do on their system (the UFS WM User's Guide provides some additional guidance). Additionally, users will need to adjust DISKNM in rt.sh to set the path to the RT data on their system.

Run the Test(s)

Simply run:

./rt.sh -a <account> -k -l my_rt.conf

where <account> is the account name you use to submit jobs on your system.

Check the Output

The rt.sh script will produce output indicating success if the test case ran successfully. For example, the end of the output should look similar to this:

+ echo 'REGRESSION TEST RESULT: SUCCESS'
REGRESSION TEST RESULT: SUCCESS
+ echo 'rt.sh finished'
rt.sh finished
+ cleanup
++ awk '{print $2}'
+ [[ 2447959 == 2447959 ]]
+ rm -rf /Users/username/ufs-weather-model/tests/lock
+ [[ false == true ]]
+ trap 0
+ exit

If there is no success message, the output should instead indicate what the problem was. Users can use this information to troubleshoot and adjust values in rt.sh (or other scripts) that may be causing problems.

Where to Find Outputs

If the run is successful, you'll find output files named dynf***.nc (dynamics variables) and phyf***.nc (physics variables) through 024h in the run directory.