Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GPU port of thickness tendency with OpenACC #4792

Merged
merged 2 commits into from
Feb 23, 2022
Merged

GPU port of thickness tendency with OpenACC #4792

merged 2 commits into from
Feb 23, 2022

Conversation

brian-oneill
Copy link
Contributor

@brian-oneill brian-oneill commented Feb 12, 2022

GPU port of ocn_tend_thick and all subprocesses

Changes include:

  • replace mesh pool variables and arrays with counterparts in ocn_mesh module
  • remove meshPool from ocn_tend_thick subroutine calls in time integrator routines
  • resolve OpenACC error in mpas_ocn_diagnostics.F by copying needed arrays onto device in ocn_compute_land_ice_flux_input_fields routine
  • add OpenACC directives to loops in ocn_tend_thick and associated subroutines

Bit-for-bit for QU240 test problem with PGI on Summit in CPU-only and GPU builds. Passed compass PR suite on Chrysalis with Intel/IMPI and GNU/OpenMPI

[BFB]

- replace mesh pool variables and arrays with counterparts in ocn_mesh module
- remove meshPool from ocn_tend_thick subroutine calls in time integrator routines
- resolve OpenACC error in mpas_ocn_diagnostics.F by copying needed
  arrays onto device in ocn_compute_land_ice_flux_input_fields routine
- add OpenACC directives to loops in ocn_tend_thick and associated subroutines
- copyout tendThick, surfaceThicknessFlux, and surfaceThicknessFluxRunoff back
  to host at end of ocn_tend_thick
Copy link
Contributor

@philipwjones philipwjones left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved based on visual inspection and confirmation of earlier bfb tests on Summit in both CPU-only and GPU runs. Also passes compass pr test suite on Compy.

@xylar
Copy link
Contributor

xylar commented Feb 18, 2022

@philipwjones, thanks very much for testing with compass!

Copy link
Contributor

@mark-petersen mark-petersen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This now matches bfb for nightly suite with both gnu and intel between the branch point for #4785 (bc5a2cc) and the head of #4785, plus these two commits cherry-picked on top. That was the easiest way to make a clean comparison. I also tested the head of this PR with gnu and intel 19, optimized and debug, and it passed everything. I also looked through the code, and tested head with:

SMS_D.T62_oQU120_ais20.MPAS_LISIO_TEST.cori-haswell_gnu
SMS_D.T62_oQU120_ais20.MPAS_LISIO_TEST.cori-haswell_intel

@brian-oneill thanks for fixing these problems, and the clean-up of removing the mesh pool.

@jonbob jonbob added the BFB PR leaves answers BFB label Feb 22, 2022
jonbob added a commit that referenced this pull request Feb 22, 2022
)

GPU port of thickness tendency with OpenACC

GPU port of ocn_tend_thick and all subprocesses. Changes include:
* replace mesh pool variables and arrays with counterparts in ocn_mesh
  module
* remove meshPool from ocn_tend_thick subroutine calls in time integrator
  routines
* resolve OpenACC error in mpas_ocn_diagnostics.F by copying needed arrays
  onto device in ocn_compute_land_ice_flux_input_fields routine
* add OpenACC directives to loops in ocn_tend_thick and associated
  subroutines

[BFB]
@jonbob
Copy link
Contributor

jonbob commented Feb 22, 2022

Passes the following e3sm_developer tests on chrysalis:

  • ERS.ne11_oQU240.WCYCL1850NS
  • ERS_Ld5.T62_oQU120.CMPASO-NYF
  • SMS_P12x2.ne4_oQU240.WCYCL1850NS.allactive-mach_mods

Merged to next

@jonbob jonbob merged commit 71508d0 into E3SM-Project:master Feb 23, 2022
@jonbob
Copy link
Contributor

jonbob commented Feb 23, 2022

merged to master

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BFB PR leaves answers BFB mpas-ocean
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants