Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add verification tasks to documentation #243

Merged
merged 135 commits into from
May 5, 2022
Merged
Show file tree
Hide file tree
Changes from 133 commits
Commits
Show all changes
135 commits
Select commit Hold shift + click to select a range
370bb6c
updated docs
gspetro Feb 11, 2022
a907487
added git submodule
gspetro Feb 11, 2022
b628dd6
fix formatting
gspetro Feb 11, 2022
467071f
added new submodule commits
gspetro Feb 11, 2022
de00e4c
fixed ref links
gspetro Feb 11, 2022
fb34100
finished Intro
gspetro Feb 11, 2022
701f9e9
finish Components & Intro edits
gspetro Feb 14, 2022
cab1c6f
edited Rocoto workflow section of Quickstart
gspetro Feb 14, 2022
290364e
added minor hpc submodule commits
gspetro Feb 15, 2022
80291a1
Updates to Rocoto Workflow in Quick Start
gspetro Feb 16, 2022
ab97b74
add to HPC-stack intro
gspetro Feb 16, 2022
8056200
submodule updates
gspetro Feb 16, 2022
17504fc
added submodule docs edits
gspetro Feb 17, 2022
357e151
hpc-stack updates & formatting fixes
gspetro Feb 17, 2022
acf555b
hpc-stack intro edits
gspetro Feb 17, 2022
36349a6
bibtex attempted fix
gspetro Feb 18, 2022
838271f
add hpc-stack module edits
gspetro Feb 18, 2022
863b7de
update sphinxcontrib version
gspetro Feb 22, 2022
2b100d9
add .readthedocs.yaml file
gspetro Feb 22, 2022
9e58e67
update .readthedocs.yaml file
gspetro Feb 22, 2022
1830b49
update .readthedocs.yaml file
gspetro Feb 22, 2022
54a647e
update conf.py
gspetro Feb 22, 2022
46d381f
updates .readthedocs.yaml with submodules
gspetro Feb 22, 2022
91af03d
updates .readthedocs.yaml with submodules
gspetro Feb 22, 2022
97616fd
submodule updates
gspetro Feb 22, 2022
21d3e27
submodule updates
gspetro Feb 22, 2022
5af69e5
minor Intro edits
gspetro Feb 23, 2022
ee901e6
minor Intro edits
gspetro Feb 23, 2022
f77cba9
minor Intro edits
gspetro Feb 23, 2022
bc0748c
submodule updates
gspetro Feb 23, 2022
fef6d27
fixed typos in QS
gspetro Feb 23, 2022
0d16101
QS updates
gspetro Feb 24, 2022
418a40b
QS updates
gspetro Feb 24, 2022
77d565d
QS updates
gspetro Feb 25, 2022
2e1a03f
updates to InputOutput and QS
gspetro Feb 25, 2022
80519d4
fix I/O doc typos
gspetro Feb 25, 2022
6f11030
pull updates to hpc-stack docs
gspetro Feb 28, 2022
999a417
pull updates to hpc-stack docs
gspetro Mar 1, 2022
f07fe8a
fix table wrapping
gspetro Mar 1, 2022
14db051
Merge branch 'ufs-community:develop' into develop
gspetro-NOAA Mar 3, 2022
b58d661
updates to QS for cloud
gspetro Mar 3, 2022
301ff5f
Merge branch 'develop' of github.com:gspetro-NOAA/ufs-srweather-app i…
gspetro Mar 3, 2022
0b50e04
fix QS export statements
gspetro Mar 3, 2022
8786b32
fix QS export statements
gspetro Mar 3, 2022
3a442d6
QS edits on bind, config
gspetro Mar 3, 2022
0cae160
add bullet points to notes
gspetro Mar 4, 2022
27247a5
running without rocoto
gspetro Mar 4, 2022
805bb81
add HPC-Stack submodule w/docs
gspetro Mar 4, 2022
29cf292
split QS into container/non-container approaches
gspetro Mar 8, 2022
3e30098
added filepath changes for running in container on Orion, et al.
gspetro Mar 9, 2022
53807fa
edits to overview and container QS
gspetro Mar 10, 2022
93bfe9b
moved CodeReposAndDirs.rst info to the Introduction & deleted file
gspetro Mar 11, 2022
eb00397
continued edits to SRWAppOverview
gspetro Mar 11, 2022
f4d2043
combine overview w/non-container docs
gspetro Mar 15, 2022
d1addf8
finish merging non-container guide & SRWOverview, rename/remove files…
gspetro Mar 16, 2022
fc1a1d4
minor edits for Intro & QS
gspetro Mar 17, 2022
acb77c8
updates to BuildRun doc through 3.8.1
gspetro Mar 17, 2022
70a051b
edits to Build/Run and Components
gspetro Mar 17, 2022
99127e7
remove .gitignore
gspetro Mar 18, 2022
b01268d
fix Ch 3 title, 4 supported platform levels note
gspetro Mar 18, 2022
da35184
fix typos, add term links
gspetro Mar 18, 2022
1302868
other minor fixes/suggestions implemented
gspetro Mar 18, 2022
a704a2f
updated Intro based on feedback; changed SRW to SRW App throughout
gspetro Mar 21, 2022
7fc263d
update comment to Intro citation
gspetro Mar 21, 2022
496fcb3
Merge branch 'develop' into develop
gspetro-NOAA Mar 22, 2022
10de71f
add user-defined vertical levels to future work
gspetro Mar 22, 2022
16b0c1a
Merge branch 'develop' of github.com:gspetro-NOAA/ufs-srweather-app i…
gspetro Mar 22, 2022
e294020
Merge branch 'ufs-community:develop' into develop
gspetro-NOAA Mar 23, 2022
92bddca
Add instructions for srw_common module load
gspetro Mar 23, 2022
698613b
Pull changes from upstream
gspetro Mar 23, 2022
6fa5074
fix typo
gspetro Mar 23, 2022
a5ae76e
update Intro & BuildRunSRW based on Mark's feedback
gspetro Mar 23, 2022
ea17b19
minor intro updates
gspetro Mar 23, 2022
1aa9322
1st round of jwolff's edits
gspetro Mar 25, 2022
3d1cddb
2nd round of jwolff updates
gspetro Mar 28, 2022
173b838
update QS intro
gspetro Mar 29, 2022
09581c8
fix minor physics details
gspetro Mar 29, 2022
a714d43
update citation and physics suite name
gspetro Mar 29, 2022
4757b40
add compute node allocation info to QS
gspetro Mar 29, 2022
2c68823
add authoritative hpc-stack docs to Intro
gspetro Mar 29, 2022
a9702bb
Merge branch 'ufs-community:develop' into develop
gspetro-NOAA Mar 30, 2022
5cb4c2c
edit config intro & platform environment sections
gspetro Mar 30, 2022
96123bb
Merge remote-tracking branch 'origin/develop' into textonly/config
gspetro Mar 30, 2022
0f3598f
edit sections on cron & directory parameters, platform & parameters f…
gspetro Mar 30, 2022
4a53366
edit NCO, file-separator, filename params, add some METplus and model…
gspetro Mar 31, 2022
c425bc5
ConfigWorkflow.rst revisions, added METplus to Components, grid info
gspetro Apr 1, 2022
f596267
add grid config details
gspetro Apr 4, 2022
fb0bc01
changes to readme.md
gspetro Apr 6, 2022
8bb70b1
RTD readme.md edits
gspetro Apr 6, 2022
9d9e079
Merge branch 'ufs-community:develop' into develop
gspetro-NOAA Apr 6, 2022
091574a
create MacOS install/build instructions
gspetro Apr 6, 2022
a5fced9
Merge branch 'develop' of github.com:gspetro-NOAA/ufs-srweather-app i…
gspetro Apr 7, 2022
a73c456
update task run and grid parameters
gspetro Apr 8, 2022
ab13f55
fixed file params & workflow task params
gspetro Apr 8, 2022
1335a4d
1st draft of ConfigParameters.inc
gspetro Apr 12, 2022
19a91cc
Merge branch 'ufs-community:develop' into textonly/config
gspetro-NOAA Apr 12, 2022
563fd8b
minor edits
gspetro Apr 12, 2022
e77110d
Merge branch 'develop' into textonly/config
gspetro Apr 12, 2022
9518b2f
Merge branch 'textonly/config' of github.com:gspetro-NOAA/ufs-srweath…
gspetro Apr 12, 2022
07f4d6e
add stochastic physics var details
gspetro Apr 12, 2022
3070272
update FVCOM, thread affinity params
gspetro Apr 13, 2022
63d2a86
halo_blend, ens, crtm, custom post, subhourly updates
gspetro Apr 13, 2022
7dcb3e7
update HPC-Stack submodule/docs
gspetro Apr 13, 2022
cf0dab0
Merge branch 'develop' into textonly/config
gspetro Apr 13, 2022
2b4d74e
Rocoto WF tasks & params
gspetro Apr 13, 2022
ef234d9
workflow tasks/params, debug, verbose, pre-existing dir, predefined grid
gspetro Apr 14, 2022
7de4653
move Stochastic physics to CCP section; write component edits
gspetro Apr 14, 2022
a552b00
comp'l forecast, grid gen, NOMADS, user-staged files
gspetro Apr 14, 2022
d8d217b
METplus, model config & forecast params, separator
gspetro Apr 15, 2022
43ecc81
2nd draft complete
gspetro Apr 15, 2022
a51165f
physics updates
gspetro Apr 15, 2022
928417a
remove MacInstall empty file
gspetro Apr 15, 2022
a1ad723
undo hpc-stack submodule update (save for separate PR)
gspetro Apr 15, 2022
9241039
undo hpc-stack install doc update (save for separate PR)
gspetro Apr 15, 2022
b9d8087
revisions to SPP & LSM physics
gspetro Apr 19, 2022
987abbd
minor edits
gspetro Apr 19, 2022
aaacfe7
update comments in LAM Grid chapter
gspetro Apr 20, 2022
2956ffc
update LSM_SPP_EACH_STEP
gspetro Apr 20, 2022
4939629
revert LSM_SPP_EACH_STEP to original definition
gspetro Apr 26, 2022
c28b32c
combine config info into one doc instead of two
gspetro Apr 26, 2022
427afb0
Merge branch 'ufs-community:develop' into textonly/vx
gspetro-NOAA Apr 26, 2022
639443c
remove ConfigParameters.inc (contents now appear in ConfigWorkflow.rst)
gspetro Apr 26, 2022
b83a363
update hpc-stack docs submodule
gspetro Apr 26, 2022
104b834
odds & ends
gspetro Apr 27, 2022
0b4e2da
add VX tables, config info, & Rocoto output tables
gspetro Apr 27, 2022
7ba1352
add module use/load statements, fix typos
gspetro Apr 27, 2022
fa05f2c
Merge branch 'ufs-community:develop' into textonly/vx
gspetro-NOAA Apr 28, 2022
41d2195
varied minor details
gspetro Apr 28, 2022
676a351
add workflow svg diagram
gspetro Apr 28, 2022
a6c8143
condense VX task table using ##
gspetro Apr 29, 2022
6f7bbad
update README
gspetro Apr 29, 2022
c0c7949
add png and revert hpc-stack commits until PR#240 (mac docs) is approved
gspetro May 2, 2022
a776900
Merge branch 'ufs-community:develop' into textonly/vx
gspetro-NOAA May 2, 2022
45f2035
jwolff edits
gspetro May 4, 2022
a02ab7d
add info on run_vx.local
gspetro May 5, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,11 @@

The Unified Forecast System (UFS) is a community-based, coupled, comprehensive Earth modeling system. It is designed to be the source system for NOAA’s operational numerical weather prediction applications while enabling research, development, and contribution opportunities for the broader weather enterprise. For more information about the UFS, visit the UFS Portal at https://ufscommunity.org/.

The UFS can be configured for multiple applications (see a complete list at https://ufscommunity.org/#/science/aboutapps). The configuration described here is the UFS Short-Range Weather (SRW) Application, which targets predictions of atmospheric behavior on a limited spatial domain and on time scales from less than an hour out to several days. The development branch of the application is continually evolving as the system undergoes open development. The SRW App v1.0.0 represents a snapshot of this continuously evolving system. The SRW App includes a prognostic atmospheric model, pre- and post-processing, and a community workflow for running the system end-to-end.
The UFS includes multiple applications (see a complete list at https://ufscommunity.org/science/aboutapps/) that support different forecast durations and spatial domains. This documentation describes the UFS Short-Range Weather (SRW) Application, which targets predictions of atmospheric behavior on a limited spatial domain and on time scales from minutes to several days. The development branch of the application is continually evolving as the system undergoes open development. The SRW App release branches represent a snapshot of this continuously evolving system. The SRW Application includes a prognostic atmospheric model, pre- and post-processing, and a community workflow for running the system end-to-end. These components are documented within the User's Guide and supported through a community forum (https://forums.ufscommunity.org/).

The UFS SRW App User's Guide associated with the development branch is at: https://ufs-srweather-app.readthedocs.io/en/latest/, while that specific to the SRW App v1.0.0 release can be found at: https://ufs-srweather-app.readthedocs.io/en/ufs-v1.0.0/. The repository is at: https://github.com/ufs-community/ufs-srweather-app.
The UFS SRW App User's Guide associated with the development branch can be found at: https://ufs-srweather-app.readthedocs.io/en/develop/, while the guide specific to the SRW App v1.0.1 release can be found at: https://ufs-srweather-app.readthedocs.io/en/ufs-v1.0.1/. The GitHub repository link is: https://github.com/ufs-community/ufs-srweather-app.

For instructions on how to clone the repository, build the code, and run the workflow, see:
https://github.com/ufs-community/ufs-srweather-app/wiki/Getting-Started

UFS Development Team. (2021, March 4). Unified Forecast System (UFS) Short-Range Weather (SRW) Application (Version v1.0.0). Zenodo. https://doi.org/10.5281/zenodo.4534994

UFS Development Team. (2021, March 4). Unified Forecast System (UFS) Short-Range Weather (SRW) Application (Version v1.0.0). Zenodo. https://doi.org/10.5281/zenodo.4534994
gspetro-NOAA marked this conversation as resolved.
Show resolved Hide resolved
288 changes: 231 additions & 57 deletions docs/UsersGuide/source/BuildRunSRW.rst

Large diffs are not rendered by default.

13 changes: 9 additions & 4 deletions docs/UsersGuide/source/Components.rst
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,16 @@ further downstream post-processing (e.g., statistical post-processing techniques
METplus Verification Suite
=============================

The Model Evaluation Tools (MET) are a set of verification tools developed by the Developmental Testbed Center (DTC) for use by the numerical weather prediction community to help them assess and evaluate the performance of numerical weather predictions. MET is the core component of the unified METplus verification framework. METplus spans a wide range of temporal (warn-on-forecast to climate) and spatial (storm to global) scales. The core components of the framework include MET, the associated database and display systems called METviewer and METexpress, and a suite of Python wrappers to provide low-level automation and examples, also called use-cases. METplus is intended to be extensible through additional capabilities developed by the community.
The enhanced Model Evaluation Tools (`METplus <https://dtcenter.org/community-code/metplus>`__) verification system has been integrated into the SRW App to facilitate forecast evaluation. METplus is a verification framework that spans a wide range of temporal scales (warn-on-forecast to climate) and spatial scales (storm to global). It is supported by the `Developmental Testbed Center (DTC) <https://dtcenter.org/>`__.

METplus is being actively developed by NCAR/Research Applications Laboratory (RAL), NOAA/Earth Systems Research Laboratories (ESRL), NOAA/Environmental Modeling Center (EMC), and is open to community contributions.
METplus is preinstalled on all `Level 1 <https://github.com/ufs-community/ufs-srweather-app/wiki/Supported-Platforms-and-Compilers>`__ systems; existing builds can be viewed `here <https://dtcenter.org/community-code/metplus/metplus-4-1-existing-builds>`__. METplus can be installed on other systems individually or as part of :term:`HPC-Stack`. Users on non-Level 1 systems can follow the `MET Installation <https://met.readthedocs.io/en/main_v10.1/Users_Guide/installation.html>`__ and `METplus Installation <https://metplus.readthedocs.io/en/v4.1.0/Users_Guide/installation.html>`__ Guides for individual installation. Currently, METplus *installation* is not a supported feature for this release of the SRW App. However, METplus *use* is supported on systems with a functioning METplus installation.
gspetro-NOAA marked this conversation as resolved.
Show resolved Hide resolved

The core components of the METplus framework include the statistical driver, MET, the associated database and display systems known as METviewer and METexpress, and a suite of Python wrappers to provide low-level automation and examples, also called use-cases. MET is a set of verification tools developed for use by the :term:`NWP` community. It matches up grids with either gridded analyses or point observations and applies configurable methods to compute statistics and diagnostics. Extensive documentation is available in the `METplus User’s Guide <https://metplus.readthedocs.io/en/v4.1.0/Users_Guide/overview.html>`__ and `MET User’s Guide <https://met.readthedocs.io/en/main_v10.1/index.html>`__. Documentation for all other components of the framework can be found at the Documentation link for each component on the METplus `downloads <https://dtcenter.org/community-code/metplus/download>`__ page.

Among other techniques, MET provides the capability to compute standard verification scores for comparing deterministic gridded model data to point-based and gridded observations. It also provides ensemble and probabilistic verification methods for comparing gridded model data to point-based or gridded observations. Verification tasks to accomplish these comparisons are defined in the SRW App in :numref:`Table %s <VXWorkflowTasksTable>`. Currently, the SRW App supports the use of :term:`NDAS` observation files in `prepBUFR format <https://nomads.ncep.noaa.gov/pub/data/nccf/com/nam/prod/>`__ (which include conventional point-based surface and upper-air data) for point-based verification. It also supports gridded Climatology-Calibrated Precipitation Analysis (:term:`CCPA`) data for accumulated precipitation evaluation and Multi-Radar/Multi-Sensor (:term:`MRMS`) gridded analysis data for composite reflectivity and :term:`echo top` verification.

METplus is being actively developed by :term:`NCAR`/Research Applications Laboratory (RAL), NOAA/Earth Systems Research Laboratories (ESRL), and NOAA/Environmental Modeling Center (EMC), and it is open to community contributions.

More details about METplus can be found on the `METplus website <https://dtcenter.org/community-code/metplus>`__.

Visualization Example
=====================
Expand All @@ -81,7 +86,7 @@ workflow file that will run each task in the system in the proper sequence (see
This SRW Application release has been tested on a variety of platforms widely used by
researchers, such as the NOAA Research and Development High-Performance Computing Systems
(RDHPCS), including Hera, Orion, and Jet; NOAA’s Weather and Climate Operational
Supercomputing System (WCOSS); the National Center for Atmospheric Research (NCAR) Cheyenne
Supercomputing System (WCOSS); the National Center for Atmospheric Research (:term:`NCAR`) Cheyenne
system; the National Severe Storms Laboratory (NSSL) HPC machine, Odin; the National Science Foundation Stampede2 system; and generic Linux and macOS systems using Intel and GNU compilers. Four `levels of support <https://github.com/ufs-community/ufs-srweather-app/wiki/Supported-Platforms-and-Compilers>`_ have been defined for the SRW Application, including pre-configured (Level 1), configurable (Level 2), limited test platforms (Level 3), and build only platforms (Level 4). Each level is further described below.

On pre-configured (Level 1) computational platforms, all the required libraries for building the SRW Application are available in a central place. That means bundled libraries (NCEPLIBS) and third-party libraries (NCEPLIBS-external) have both been built. The SRW Application is expected to build and run out-of-the-box on these pre-configured platforms.
Expand Down
5 changes: 3 additions & 2 deletions docs/UsersGuide/source/ConfigNewPlatform.rst
Original file line number Diff line number Diff line change
Expand Up @@ -358,8 +358,6 @@ Those requirements highlighted in **bold** are included in the NCEPLIBS-external

* MPI (**MPICH**, OpenMPI, or other implementation)

* wgrib2

* CMake v3.12+

* Software libraries
Expand All @@ -372,6 +370,9 @@ Those requirements highlighted in **bold** are included in the NCEPLIBS-external
* **libPNG**
* **zlib**

..
COMMENT: Update version of ESMF? Need other version updates?
gspetro-NOAA marked this conversation as resolved.
Show resolved Hide resolved

macOS-specific prerequisites:

* brew install wget
Expand Down
Loading