Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add split-explicit AB2 time stepping capability to the ocean component #5989

Merged
merged 12 commits into from
Nov 21, 2023
Merged

Add split-explicit AB2 time stepping capability to the ocean component #5989

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
a365b2b
Add the AB2 time stepping option to MPAS-Ocean
hyungyukang Oct 9, 2023
2ca482d
Update Registry.xml and some codes
hyungyukang Oct 10, 2023
e98913f
Remove unsplit stuffs and revise header in split_ab2.F
hyungyukang Oct 11, 2023
b65f38f
Fix openmp directives and add ab2 option in other places
hyungyukang Oct 12, 2023
6642572
Fixed unterminated openmp directive
hyungyukang Oct 12, 2023
dd88eb5
Fix issues when running on GPUs
hyungyukang Oct 17, 2023
76fabed
Fix the restart issue and add the AB2 option in some descriptions
hyungyukang Oct 26, 2023
1ed4764
Initialize normalVelocity=0 at the top of the domain
hyungyukang Oct 26, 2023
f4d112b
Change the default config_time_integrator to split_explicit_ab2
hyungyukang Oct 31, 2023
77a847e
Fix a bug for the decomposition error after rebase
hyungyukang Nov 2, 2023
d27af88
Change halo computation to nEdgesAll
mark-petersen Nov 2, 2023
b2067b7
Change the header of the AB2 code
hyungyukang Nov 15, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion components/mpas-ocean/bld/namelist_files/namelist_defaults_mpaso.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@
<config_dt ocn_grid="SOwISC12to60E2r4">'00:10:00'</config_dt>
<config_dt ocn_grid="ECwISC30to60E2r1">'00:30:00'</config_dt>
<config_dt ocn_grid="ECwISC30to60E3r2">'00:30:00'</config_dt>
<config_time_integrator>'split_explicit'</config_time_integrator>
<config_time_integrator>'split_explicit_ab2'</config_time_integrator>
<config_number_of_time_levels>2</config_number_of_time_levels>

<!-- hmix -->
Expand Down
2 changes: 1 addition & 1 deletion components/mpas-ocean/bld/namelist_files/namelist_definition_mpaso.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ Default: Defined in namelist_defaults.xml
category="time_integration" group="time_integration">
Time integration method.

Valid values: 'split_explicit', 'RK4', 'unsplit_explicit', 'split_implicit', 'LTS'
Valid values: 'split_explicit', 'RK4', 'unsplit_explicit', 'split_implicit', 'LTS', 'split_explicit_ab2'
Default: Defined in namelist_defaults.xml
</entry>

Expand Down
18 changes: 15 additions & 3 deletions components/mpas-ocean/src/Registry.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -200,9 +200,9 @@
description="Length of model time-step."
possible_values="Any time stamp in 'YYYY-MM-DD_hh:mm:ss' format. Items can be removed from the left if they are unused."
/>
<nml_option name="config_time_integrator" type="character" default_value="split_explicit"
<nml_option name="config_time_integrator" type="character" default_value="split_explicit_ab2"
description="Time integration method."
possible_values="'split_explicit', 'RK4', 'unsplit_explicit', 'split_implicit', 'LTS'"
possible_values="'split_explicit', 'RK4', 'unsplit_explicit', 'split_implicit', 'LTS', 'split_explicit_ab2'"
/>
<nml_option name="config_number_of_time_levels" type="integer" default_value="2"
description="The number of time levels in the time-stepping scheme. This is used for array allocation."
Expand Down Expand Up @@ -1278,7 +1278,7 @@
</nml_record>
<nml_record name="split_timestep_share" mode="forward">
<nml_option name="config_n_ts_iter" type="integer" default_value="2"
description="number of large iterations over stages 1-3"
description="number of large iterations over stages 1-3; For the split_explicit_ab2 time integrator, this value only affects the first time step when it is not a restart run. For restart runs, this value has no effect on the split_explicit_ab2 time integrator."
possible_values="any positive integer, but typically 1, 2, or 3"
/>
<nml_option name="config_n_bcl_iter_beg" type="integer" default_value="1"
Expand Down Expand Up @@ -1590,6 +1590,7 @@
<package name="variableShortwave" description="This package includes variables required to compute spatially variable shortwave extinction coefficients"/>

<package name="splitTimeIntegrator" description="This package includes variables required for either the split or unsplit explicit time integrators."/>
<package name="splitAB2TimeIntegrator" description="This package includes variables required for the split explicit AB2 time integrators."/>
<package name="semiImplicitTimePKG" description="This package includes variables required for split-implicit time integrators."/>
<package name="thicknessFilter" description="This package includes variables required for frequency filtered thickness."/>
<package name="windStressBulkPKG" description="This package includes varibles required for bulk wind stress forcing."/>
Expand Down Expand Up @@ -1914,6 +1915,8 @@
<var name="landIceFloatingFraction"/>
<var name="seaIcePressure"/>
<var name="atmosphericPressure"/>
<var name="normalVelocityTendOld" packages="splitAB2TimeIntegrator" />
<var name="CoriolisTermOld" packages="splitAB2TimeIntegrator" />
<var_array name="tracersSurfaceValue"/>
<var_array name="surfaceVelocity"/>
<var_array name="SSHGradient"/>
Expand Down Expand Up @@ -3403,6 +3406,15 @@
description="gradient of sea surface height perturbation from self-attraction and loading"
packages="tidalPotentialForcingPKG"
/>
<!-- FIELD split_explicit AB2 time stepping -->
<var name="normalVelocityTendOld" type="real" dimensions="nVertLevels nEdges Time" units="m s^-2"
description="time tendency of normal component of velocity at the previous time step used in the split-explicit AB2 time stepping"
packages="splitAB2TimeIntegrator"
/>
<var name="CoriolisTermOld" type="real" dimensions="nVertLevels nEdges Time" units="m s^-2"
description="Coriolis term at the previous time step used in the split-explicit AB2 time stepping"
packages="splitAB2TimeIntegrator"
/>
</var_struct>
<var_struct name="shortwave" time_levs="1">
<!-- **********************************************************************
Expand Down
6 changes: 4 additions & 2 deletions components/mpas-ocean/src/analysis_members/mpas_ocn_high_frequency_output.F
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -352,7 +352,8 @@ subroutine ocn_compute_high_frequency_output(domain, timeLevel, err)!{{{
call mpas_pool_get_array(highFrequencyOutputAMPool, 'vertVelSFC', vertVelSFC)

! split explicit specific arrays
if ( config_time_integrator == trim('split_explicit') .or. config_time_integrator == trim('split_implicit') ) then
if ( config_time_integrator == trim('split_explicit') .or. config_time_integrator == trim('split_implicit') &
.or. config_time_integrator == trim('split_explicit_ab2') ) then
call mpas_pool_get_array(statePool, 'normalBaroclinicVelocity', normalBaroclinicVelocity, 1)
call mpas_pool_get_array(statePool, 'normalBarotropicVelocity', normalBarotropicVelocity, 1)
endif
Expand Down Expand Up @@ -539,7 +540,8 @@ subroutine ocn_compute_high_frequency_output(domain, timeLevel, err)!{{{
meridionalAreaWeightedCellVelAtBottom(iCell) = meridionalAreaWeightedCellVelAtBottom(iCell)/cellArea
end do

if ( config_time_integrator == trim('split_explicit')) then
if ( config_time_integrator == trim('split_explicit') .or. config_time_integrator == trim('split_implicit') &
.or. config_time_integrator == trim('split_explicit_ab2') ) then

do iEdge = 1, nEdges
normalBarotropicVel(iEdge) = normalBarotropicVelocity(iEdge)
Expand Down
11 changes: 10 additions & 1 deletion components/mpas-ocean/src/driver/mpas_ocn_core_interface.F
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,7 @@ function ocn_setup_packages(configPool, packagePool, iocontext) result(ierr)!{{{
logical, pointer :: thicknessFilterActive
logical, pointer :: splitTimeIntegratorActive
logical, pointer :: semiImplicitTimePKGActive
logical, pointer :: splitAB2TimeIntegratorActive
logical, pointer :: windStressBulkPKGActive
logical, pointer :: variableBottomDragPKGActive
logical, pointer :: tracerBudgetActive
Expand Down Expand Up @@ -216,7 +217,8 @@ function ocn_setup_packages(configPool, packagePool, iocontext) result(ierr)!{{{
if ( forwardModeActive ) then
if ( config_time_integrator == trim('split_explicit') &
.or. config_time_integrator == trim('unsplit_explicit') &
.or. config_time_integrator == trim('split_implicit') ) then
.or. config_time_integrator == trim('split_implicit') &
.or. config_time_integrator == trim('split_explicit_ab2') ) then
splitTimeIntegratorActive = .true.
end if
endif
Expand All @@ -227,6 +229,13 @@ function ocn_setup_packages(configPool, packagePool, iocontext) result(ierr)!{{{
end if
endif

call mpas_pool_get_package(packagePool, 'splitAB2TimeIntegratorActive', splitAB2TimeIntegratorActive)
if ( forwardModeActive ) then
if (config_time_integrator == trim('split_explicit_ab2') ) then
splitAB2TimeIntegratorActive = .true.
end if
endif

!
! test for time filtering scheme
!
Expand Down
8 changes: 5 additions & 3 deletions components/mpas-ocean/src/mode_forward/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,14 @@ OBJS = mpas_ocn_forward_mode.o \
mpas_ocn_time_integration_rk4.o \
mpas_ocn_time_integration_si.o \
mpas_ocn_time_integration_split.o \
mpas_ocn_time_integration_lts.o
mpas_ocn_time_integration_lts.o \
mpas_ocn_time_integration_split_ab2.o

all: forward_mode

forward_mode: $(OBJS)

mpas_ocn_time_integration.o: mpas_ocn_time_integration_rk4.o mpas_ocn_time_integration_si.o mpas_ocn_time_integration_split.o mpas_ocn_time_integration_lts.o
mpas_ocn_time_integration.o: mpas_ocn_time_integration_rk4.o mpas_ocn_time_integration_si.o mpas_ocn_time_integration_split.o mpas_ocn_time_integration_lts.o mpas_ocn_time_integration_split_ab2.o

mpas_ocn_time_integration_rk4.o:

Expand All @@ -25,7 +26,8 @@ mpas_ocn_forward_mode.o: mpas_ocn_time_integration.o \
mpas_ocn_time_integration_rk4.o \
mpas_ocn_time_integration_si.o \
mpas_ocn_time_integration_split.o \
mpas_ocn_time_integration_lts.o
mpas_ocn_time_integration_lts.o \
mpas_ocn_time_integration_split_ab2.o

clean:
$(RM) *.o *.mod *.f90
Expand Down
4 changes: 3 additions & 1 deletion components/mpas-ocean/src/mode_forward/mpas_ocn_forward_mode.F
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ module ocn_forward_mode

use ocn_time_integration
use ocn_time_integration_split
use ocn_time_integration_split_ab2
use ocn_time_integration_si
use ocn_tendency
use ocn_diagnostics
Expand Down Expand Up @@ -860,7 +861,8 @@ function ocn_forward_mode_finalize(domain) result(iErr)!{{{
! destroy the ocean mesh structure
call ocn_meshDestroy(ierr)

if ( config_time_integrator == 'split_explicit') then
if ( config_time_integrator == 'split_explicit' .or. &
config_time_integrator == 'split_explicit_ab2' ) then
call mpas_dmpar_exch_group_destroy_reusable_buffers(domain, 'subcycleFields')
else if ( config_time_integrator == 'split_implicit') then
call ocn_time_integrator_si_variable_destroy()
Expand Down
23 changes: 16 additions & 7 deletions components/mpas-ocean/src/mode_forward/mpas_ocn_time_integration.F
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ module ocn_time_integration
use ocn_time_integration_split
use ocn_time_integration_si
use ocn_time_integration_lts
use ocn_time_integration_split_ab2


implicit none
Expand Down Expand Up @@ -65,12 +66,13 @@ module ocn_time_integration
integer :: timeIntegratorChoice

integer, parameter :: &
timeIntUnknown = 0, &! unknown or undefined
timeIntSplitExplicit = 1, &! split-explicit
timeIntUnsplitExplicit = 2, &! unsplit-explicit
timeIntSemiImplicit = 3, &! Semi-implicit
timeIntRK4 = 4, &! 4th-order Runge-Kutta
timeIntLTS = 5 ! local time-stepping
timeIntUnknown = 0, &! unknown or undefined
timeIntSplitExplicit = 1, &! split-explicit
timeIntUnsplitExplicit = 2, &! unsplit-explicit
timeIntSemiImplicit = 3, &! Semi-implicit
timeIntRK4 = 4, &! 4th-order Runge-Kutta
timeIntLTS = 5, &! local time-stepping
timeIntSplitExplicitAB2 = 6 ! split-explicit AB2 baroclinic

!***********************************************************************

Expand Down Expand Up @@ -134,6 +136,8 @@ subroutine ocn_timestep(domain, dt, timeStamp)!{{{
call ocn_time_integrator_rk4(domain, dt)
case (timeIntLTS)
call ocn_time_integrator_lts(domain, dt)
case (timeIntSplitExplicitAB2)
call ocn_time_integrator_split_ab2(domain, dt)
end select

! compute time since start of simulation, in days
Expand Down Expand Up @@ -229,6 +233,10 @@ subroutine ocn_timestep_init(domain, dt, err)!{{{
timeIntegratorChoice = timeIntLTS
call ocn_time_integration_lts_init(domain)

case ('split_explicit_ab2')
timeIntegratorChoice = timeIntSplitExplicitAB2
call ocn_time_integration_split_ab2_init(domain)

case default

err = 1
Expand All @@ -237,7 +245,8 @@ subroutine ocn_timestep_init(domain, dt, err)!{{{
'Incorrect choice for config_time_integrator:' // &
trim(config_time_integrator) // &
' choices are: RK4, split_explicit, ' // &
'unsplit_explicit, split_implicit, LTS', MPAS_LOG_ERR)
'unsplit_explicit, split_implicit, LTS, ' // &
'split_explicit_ab2', MPAS_LOG_ERR)

end select

Expand Down
Loading