Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add haloupdate unit test #820

Merged
merged 13 commits into from
Mar 13, 2023
315 changes: 287 additions & 28 deletions cicecore/cicedyn/infrastructure/comm/mpi/ice_boundary.F90

Large diffs are not rendered by default.

442 changes: 338 additions & 104 deletions cicecore/cicedyn/infrastructure/comm/serial/ice_boundary.F90

Large diffs are not rendered by default.

5 changes: 5 additions & 0 deletions cicecore/cicedyn/infrastructure/ice_grid.F90
Original file line number Diff line number Diff line change
Expand Up @@ -360,6 +360,11 @@ subroutine init_grid1
file=__FILE__, line=__LINE__)
endif

if (grid_type == 'tripole' .and. (mod(nx_global,2)/=0)) then
call abort_ice(subname//'ERROR: grid_type tripole requires even nx_global number', &
file=__FILE__, line=__LINE__)
endif

if (trim(grid_type) == 'displaced_pole' .or. &
trim(grid_type) == 'tripole' .or. &
trim(grid_type) == 'regional' ) then
Expand Down
472 changes: 472 additions & 0 deletions cicecore/drivers/unittest/halochk/CICE_InitMod.F90

Large diffs are not rendered by default.

811 changes: 811 additions & 0 deletions cicecore/drivers/unittest/halochk/halochk.F90

Large diffs are not rendered by default.

3 changes: 1 addition & 2 deletions cicecore/shared/ice_constants.F90
Original file line number Diff line number Diff line change
Expand Up @@ -97,8 +97,7 @@ module ice_constants
field_loc_center = 1, &
field_loc_NEcorner = 2, &
field_loc_Nface = 3, &
field_loc_Eface = 4, &
field_loc_Wface = 5
field_loc_Eface = 4

!-----------------------------------------------------------------
! field type attribute - necessary for handling
Expand Down
6 changes: 4 additions & 2 deletions configuration/scripts/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ AR := ar

.SUFFIXES:

.PHONY: all cice libcice targets target db_files db_flags clean realclean helloworld calchk sumchk bcstchk gridavgchk optargs
.PHONY: all cice libcice targets target db_files db_flags clean realclean helloworld calchk sumchk bcstchk gridavgchk halochk optargs
all: $(EXEC)

cice: $(EXEC)
Expand All @@ -93,7 +93,7 @@ targets:
@echo " "
@echo "Supported Makefile Targets are: cice, libcice, makdep, depends, clean, realclean"
@echo " Diagnostics: targets, db_files, db_flags"
@echo " Unit Tests : helloworld, calchk, sumchk, bcstchk, gridavgchk, optargs"
@echo " Unit Tests : helloworld, calchk, sumchk, bcstchk, gridavgchk, halochk, optargs"
target: targets

db_files:
Expand Down Expand Up @@ -151,6 +151,8 @@ bcstchk: $(EXEC)

gridavgchk: $(EXEC)

halochk: $(EXEC)

# this builds just a subset of source code specified explicitly and requires a separate target

HWOBJS := helloworld.o
Expand Down
6 changes: 4 additions & 2 deletions configuration/scripts/cice_decomp.csh
Original file line number Diff line number Diff line change
Expand Up @@ -156,8 +156,10 @@ if (${ICE_DECOMP_MXBLCKS} > 0) set mxblcks = ${ICE_DECOMP_MXBLCKS}

set decomp = 'cartesian'
set dshape = 'slenderX2'
if (${nxglob} % ${cicepes} != 0) set decomp = 'roundrobin'
if (${mxblcks} * ${blcky} * 2 < ${nyglob}) set decomp = 'roundrobin'
if (${cicepes} % 2 != 0) set decomp = 'roundrobin'
if (${nyglob} % (${blcky} * 2) != 0) set decomp = 'roundrobin'
if (${nxglob} % ${blckx} != 0) set decomp = 'roundrobin'
if (((${nxglob} * 2) % (${cicepes} * ${blckx})) != 0) set decomp = 'roundrobin'

#--- outputs ---

Expand Down
2 changes: 2 additions & 0 deletions configuration/scripts/options/set_env.halochk
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
setenv ICE_DRVOPT unittest/halochk
setenv ICE_TARGET halochk
3 changes: 3 additions & 0 deletions configuration/scripts/options/set_nml.cyclic
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
ew_boundary_type = 'cyclic'
ns_boundary_type = 'cyclic'

3 changes: 3 additions & 0 deletions configuration/scripts/options/set_nml.open
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
ew_boundary_type = 'open'
ns_boundary_type = 'open'

3 changes: 3 additions & 0 deletions configuration/scripts/options/set_nml.tripole
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
grid_type = 'tripole'
ew_boundary_type = 'cyclic'
ns_boundary_type = 'tripole'
3 changes: 3 additions & 0 deletions configuration/scripts/options/set_nml.tripolet
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
grid_type = 'tripole'
ew_boundary_type = 'cyclic'
ns_boundary_type = 'tripoleT'
41 changes: 28 additions & 13 deletions configuration/scripts/tests/unittest_suite.ts
Original file line number Diff line number Diff line change
@@ -1,14 +1,29 @@
# Test Grid PEs Sets BFB-compare
unittest gx3 1x1 helloworld
unittest gx3 1x1 optargs
unittest gx3 1x1 calchk,short
unittest gx3 4x1x25x29x4 sumchk
unittest gx3 1x1x25x29x16 sumchk
unittest tx1 8x1 sumchk
unittest gx3 4x1 bcstchk
unittest gx3 1x1 bcstchk
unittest gx3 8x2 gridavgchk,dwblockall
unittest gx3 12x1 gridavgchk
unittest gx1 28x1 gridavgchk,dwblockall
unittest gx1 16x2 gridavgchk
unittest gbox128 8x2 gridavgchk
unittest gx3 1x1 helloworld
unittest gx3 1x1 optargs
unittest gx3 1x1 calchk,short
unittest gx3 4x1x25x29x4 sumchk
unittest gx3 1x1x25x29x16 sumchk
unittest tx1 8x1 sumchk
unittest gx3 4x1 bcstchk
unittest gx3 1x1 bcstchk
unittest gx3 8x2 gridavgchk,dwblockall
unittest gx3 12x1 gridavgchk
unittest gx1 28x1 gridavgchk,dwblockall
unittest gx1 16x2 gridavgchk
unittest gbox128 8x2 gridavgchk
unittest gbox80 1x1x10x10x80 halochk,cyclic,debug
unittest gbox80 1x1x24x23x16 halochk
unittest gbox80 1x1x23x24x16 halochk,cyclic
unittest gbox80 1x1x23x23x16 halochk,open
unittest tx1 1x1x90x60x16 halochk,dwblockall
unittest tx1 1x1x90x60x16 halochk,dwblockall,tripolet
unittest tx1 1x1x95x65x16 halochk,dwblockall
unittest tx1 1x1x95x65x16 halochk,dwblockall,tripolet
unittest gx3 4x2 halochk,dwblockall,debug
unittest gx3 8x2x16x12x10 halochk,cyclic,dwblockall
unittest gx3 17x1x16x12x10 halochk,open,dwblockall
unittest tx1 4x2 halochk,dwblockall
unittest tx1 4x2 halochk,dwblockall,tripolet
unittest tx1 4x2x65x45x10 halochk,dwblockall
unittest tx1 4x2x57x43x12 halochk,dwblockall,tripolet
47 changes: 45 additions & 2 deletions doc/source/user_guide/ug_implementation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,28 @@ fold, on the logical grid. The point with index i along the ghost T-row
of index :math:`n+1` physically coincides with point
:math:`{\tt nx\_global}-{\tt i}+1` on the T-row of index :math:`n`. The
ghost U-row of index :math:`n+1` physically coincides with the U-row of
index :math:`n-1`.
index :math:`n-1`. In the schematics below, symbols A-H represent
grid points from 1:nx_global at a given j index and the setup of the
tripole seam is depicted within a few rows of the seam.

.. _tab-tripole:

.. table:: Tripole (u-fold) Grid Schematic
:align: center

+--------------+---------------------------------------+--------------+
| global j | | global j |
| index | grid point IDs (i index) | index source |
+==============+====+====+====+====+====+====+====+====+==============+
| ny_global+2 | H | G | F | E | D | C | B | A | ny_global-1 |
+--------------+----+----+----+----+----+----+----+----+--------------+
| ny_global+1 | H | G | F | E | D | C | B | A | ny_global |
+--------------+----+----+----+----+----+----+----+----+--------------+
| ny_global | A | B | C | D | E | F | G | H | |
+--------------+----+----+----+----+----+----+----+----+--------------+
| ny_global-1 | A | B | C | D | E | F | G | H | |
+--------------+----+----+----+----+----+----+----+----+--------------+


In the T-fold tripole grid, the poles have T-index 1 and and
:math:`{\tt nx\_global}/2+1` on the top T-row of the physical grid, and
Expand All @@ -301,6 +322,27 @@ north of U-row :math:`n`. Ghost T-row :math:`n+1` coincides with T-row
:math:`n-1`, and ghost U-row :math:`n+1` coincides with U-row
:math:`n-2`.

.. _tab-tripoleT:

.. table:: TripoleT (t-fold) Grid Schematic
:align: center

+--------------+--------------------------------------------+--------------+
| global j | | global j |
| index | grid point IDs (i index) | index source |
+==============+====+====+====+====+====+====+====+====+====+==============+
| ny_global+2 | | H | G | F | E | D | C | B | A | ny_global-2 |
+--------------+----+----+----+----+----+----+----+----+----+--------------+
| ny_global+1 | | H | G | F | E | D | C | B | A | ny_global-1 |
+--------------+----+----+----+----+----+----+----+----+----+--------------+
| ny_global | A | BH | CG | DF | E | FD | GC | HB | | |
+--------------+----+----+----+----+----+----+----+----+----+--------------+
| ny_global-1 | A | B | C | D | E | F | G | H | | |
+--------------+----+----+----+----+----+----+----+----+----+--------------+
| ny_global-2 | A | B | C | D | E | F | G | H | | |
+--------------+----+----+----+----+----+----+----+----+----+--------------+


The tripole grid thus requires two special kinds of treatment for
certain rows, arranged by the halo-update routines. First, within rows
along the fold, coincident points must always have the same value. This
Expand All @@ -310,7 +352,8 @@ the coincident physical rows. Both operations involve the tripole
buffer, which is used to assemble the data for the affected rows.
Special treatment is also required in the scattering routine, and when
computing global sums one of each pair of coincident points has to be
excluded.
excluded. Halos of center, east, north, and northeast points are supported,
and each requires slightly different halo indexing across the tripole seam.

*****************
Rectangular grids
Expand Down
1 change: 1 addition & 0 deletions doc/source/user_guide/ug_testing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -736,6 +736,7 @@ The following are brief descriptions of some of the current unit tests,
- **calchk** is a unit test that exercises the CICE calendar over 100,000 years and verifies correctness.
This test does not depend on the CICE initialization.
- **gridavgchk** is a unit test that exercises the CICE grid_average_X2Y methods and verifies results.
- **halochk** is a unit test that exercises the CICE haloUpdate methods and verifies results.
- **helloworld** is a simple test that writes out helloworld and uses no CICE infrastructure.
This tests exists to demonstrate how to build a unit test by specifying the object files directly
in the Makefile
Expand Down