Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reimplement smartredis #159

Merged
merged 10 commits into from
Aug 9, 2022
37 changes: 37 additions & 0 deletions config_src/external/database_comms/MOM_database_comms.F90
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
!> Contains routines necessary to initialize communication with a database
module MOM_database_comms
! This file is part of MOM6. See LICENSE.md for the license.
use MOM_file_parser, only : param_file_type
use MOM_error_handler, only : MOM_error, WARNING
use database_client_interface, only : dbclient_type

implicit none; private

!> Control structure to store Database communication related parameters and objects
type, public :: dbcomms_CS_type
type(dbclient_type) :: client !< The Database client itself
logical :: use_dbclient !< If True, use Database within MOM6
logical :: colocated !< If True, the orchestrator was setup in 'co-located' mode
logical :: cluster !< If True, the orchestrator has three shards or more
integer :: colocated_stride !< Sets which ranks will load the model from the file
!! e.g. mod(rank,colocated_stride) == 0
end type dbcomms_CS_type

public :: database_comms_init
public :: dbclient_type

contains

subroutine database_comms_init(param_file, CS, client_in)
type(param_file_type), intent(in ) :: param_file !< Parameter file structure
type(dbcomms_CS_type), intent(inout) :: CS !< Control structure for Database
type(dbclient_type), optional, intent(in ) :: client_in !< If present, use a previously initialized
!! Database client

call MOM_error(WARNING,"dbcomms_init was compiled using the dummy module. If this was\n"//&
"a mistake, please follow the instructions in:\n"//&
"MOM6/config_src/external/dbclient/README.md")
end subroutine database_comms_init

end module MOM_database_comms

25 changes: 25 additions & 0 deletions config_src/external/database_comms/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Overview
This module is designed to be used in conjunction with the SmartSim and
SmartRedis libraries found at https://github.com/CrayLabs/. These
libraries are used to perform machine-learning inference and online
analysis using a Redis-based database.

An earlier implementation of these routines was used in Partee et al. [2022]:
"Using Machine Learning at scale in numerical simulations with SmartSim:
An application to ocean climate modeling" (doi.org/10.1016/j.jocs.2022.101707)
to predict eddy kinetic energy for use in the MEKE module. The additional
scripts and installation instructions for compiling MOM6 for this case can
be found at: https://github.com/CrayLabs/NCAR_ML_EKE/. The substantive
code in the new implementation is part of `MOM_MEKE.F90`.

# File description

- `MOM_database_comms` contains just method signatures and elements of the
control structure that are imported elsewhere within the primary MOM6
code. This includes: `dbcomms_CS_type`, `dbclient_type`, and `database_comms_init`

- `database_client_interface.F90` contains the methods for a communication client
to transfer data and/or commands between MOM6 and a remote database. This is
roughly based on the SmartRedis library, though only the methods that are most
likely to be used with MOM6 are retained. This is to ensure that the API can be
tested without requiring MOM6 users to compile in the the full library.
Loading