Enhance "compile_MET_all.sh" to support the new Intel oneAPI compilers and upgrade dependent library versions as needed

[HathewayWill]

Describe the Task

Upgrade some of MET's external dependencies (based on availability on WCOSS2):

• Grib2C from 1.6.0 to 1.6.4
• Jasper from 1.900.1 to 2.0.25
• GSL from 2.1 to 2.7
• BUFRLIB from 11.3.0 to 11.6.0
• Decrease Proj from 9.2.1 to 7.1.0
  and ensure that MET and its dependencies compile with the new Intel oneAPI Compilers.

Also include changes for Enhance MET to compile and link against the Atlas and ecKit libraries #2574. Atlas and ecKit will be optional libraries. We will use the following versions based on availability on WCOSS2:
Atlas 0.30.0
EcKit 1.20.2

Note that METbaseimage will need to be updated along with enhancing the Dockerfile to allow the MET branch and tarfile location to be defined at build time. METbaseimage will provides a test for the GNU compilers, but not Intel ones.

In the second half of 2023 Intel will be ending their classic compilers for the new Intel LLVM-based compilers Intel® oneAPI C++ Compiler (ICX)(https://www.intel.com/content/www/us/en/developer/articles/technical/getting-to-know-llvm-based-oneapi-compilers.html#:~:text=Intel%20oneAPI%20DPC%2B%2B%2FC%2B%2B%20Compiler,-oneAPI's%20Data%20Parallel&text=It's%20built%20on%20LLVM%20and,C%2B%2B%20for%20heterogeneous%20computing%20platforms.)

The porting guide from intel classic compilers to intel llvm compilers can be found here:
https://www.intel.com/content/www/us/en/developer/articles/guide/porting-guide-for-ifort-to-ifx.html
https://www.intel.com/content/www/us/en/developer/articles/guide/porting-guide-for-icc-users-to-dpcpp-or-icx.html

According to intel it should be a simple change to use these exports:

  export CC=icx
  export CXX=icpx
  export FC=ifx
  export F77=ifx
  export F90=ifx
  export MPIFC="mpiifort -fc=ifx"
  export MPIF77="mpiifort -fc=ifx"
  export MPIF90="mpiifort -fc=ifx"
  export MPICC="mpiicc -cc=icx"
  export MPICXX="mpiicpc -cxx=icpx"

Will Hatheway used the above exports MET but found several errors. See attached here in a zip file.
MET_intel_llvm.zip

Time Estimate

3 days

Sub-Issues

Consider breaking the task down into sub-issues.

• Add a checkbox for each sub-issue here.

Relevant Deadlines

12.0.0 release

Funding Source

2702691

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Review default alert labels
• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Milestone as the next official version or Backlog of Development Ideas
• For the next official version, select the MET-X.Y.Z Development project

Define Related Issue(s)

Consider the impact to the other METplus components.
METbaseimage will need to be updated

• METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy

Task Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Development issue
  Select: Milestone as the next official version
  Select: MET-X.Y.Z Development project for development toward the next official release
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.

[jprestop]

@jprestop Reminder to self to update Tor on how this goes.

[jprestop]

Some work done on Derecho to upgrade the following libraries

• Grib2c from1.6.0 to 1.6.4
• Jasper from 1.900.1 to 2.0.25
• GSL from 2.1 to 2.7
• BUFRLIB from 11.3.0 to 11.5.0 or 11.7.0

with new oneAPI compilers. Had success with all but BUFRLIB. Need to get BUFRLIB to compile before I can get MET to compile.

[jprestop]

Update: I was successfully able to install BUFRLIB 11.6.0 using the new oneAPI compilers. I have posted on the BUFRLIB GitHub Discussions forum with further detail about the problems with 11.5.0 and 11.7.0:

BUFRLIB compilation with oneAPI compilers? #530

On to compiling MET...

[HathewayWill]

@jprestop Let me know and I will also verify

…

On Thu, Oct 19, 2023, 3:04 PM Julie Prestopnik ***@***.***> wrote: Update: I was successfully able to install BUFRLIB 11.6.0 using the new oneAPI compilers. I have posted on the BUFRLIB GitHub Discussions forum with further detail about the problems with 11.5.0 and 11.7.0: BUFRLIB compilation with oneAPI compilers? #530 <NOAA-EMC/NCEPLIBS-bufr#530> On to compiling MET... — Reply to this email directly, view it on GitHub <#2611 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A4FUT5RKGWAW2T7FI5C4ICLYAGBV3AVCNFSM6AAAAAA2NCNSWGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZRGYZDSNBVGY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[jprestop]

Update: I was getting this error in the met.make.log file:

/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: cannot find -loopopt=1

After a little research, I ended up on the oneAPI Release Notes page, which indicates that this is a known problem:

Problem
In trying to determine libraries needed to link Fortran code with C or C++ code, GNU Autoconf 2.69 and earlier mistakenly interprets -loopopt=0 in -mllvm -loopopt=0 in verbose compiler output as a linker flag.  GNU Autoconf then adds -loopopt=0 to the FLIBS variable, which is passed to the linker.  The linker then looks for a non-existent library causing the test to fail.

The solution involves installing "GNU Autoconf-2.70 or newer".  On Derecho exists version 2.71 where I was able to run "autoreconf -if" to recreate the configure file using autoconf 2.71.  Using the new configure script I was able to get a successful compilation of MET using the one API compilers on Derecho.  We are using autoconf version 2.71 in our automated Docker builds for our test code.

[jprestop]

This work was completed with PR #2737.