Enhance MET to compile and link against the Atlas and ecKit libraries

[JohnHalleyGotway]

Describe the New Feature

Adding support in MET for unstructured grids requires adding dependencies on the ecKit and/or Atlas libraries. These depend on the Proj library which is being added via MET#2669. This task is to install those libraries on the development machine, seneca, and update MET's configuration script and Makefiles to link to them. Basically, any Makefile currently linking to the vx_grid library should now also link to ecKit and/or Atlas.

Remember to update the development environment setup scripts in MET/internal/scripts/environment accordingly.

Recommend coordinating with @jprestop to discuss the implications on the compile_MET_all.sh script and the various supported platforms.

Recommend updating the User's Guide during this process as well to clarify the dependencies.

Acceptance Testing

List input data types and sources.
Describe tests required for new functionality.

Time Estimate

4 days?

Sub-Issues

Consider breaking the new feature down into sub-issues.
None

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

7705991 MPAS Participation

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Repository and/or Organization level Project(s) or add alert: NEED CYCLE ASSIGNMENT label
• Select Milestone as the next official version or Future Versions

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy
• Update METbaseimage to complete the transition to the Debian 12 (bookworm) base image METbaseimage#12

New Feature Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Development issues
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.

[JohnHalleyGotway]

Note, based on a UGRID project meeting on 8/23/23, it may be the case that only the eckit library is need and not Atlas. Atlas depends on eckit but @hsoh-u may only actually be using eckit features rather than Atlas-specific add ons. At this time, recommend compiling against Proj and eckit.

Note that ecKit contains tags (https://github.com/ecmwf/eckit/tags) but there are no corresponding releases created on GitHub. We'll need to note this detail in the METplus documentation.

Will wait for direction from @hsoh-u as to whether or not Atlas is also required.

[hsoh-u]

Please add atlas library and eckit library. To use KDTree at eckit, more work for point data (what atlas does) should be done.

[JohnHalleyGotway]

Shifting from the beta1 cycle to beta2 since beta1 ends tomorrow. Would ideally like to complete this by the end of this week though.

[JohnHalleyGotway]

@hsoh-u I was going to take a crack at this but am wondering how I can be most helpful to you. Should I use a separate feature_2574_atlas_eckit branch or should I use your existing development branch? Also, should I actually update any of the MET Makefiles to actually like to atlas and eckit? If so, which ones and where?

[hsoh-u]

Either ways work. The duplicated works were done at feature_2574_atlas_eckit branch (Makefile.am files and and others configurations). "development.seneca" maybe modified depending on the location of Atlas and Eckit locations.

[JohnHalleyGotway]

Per @TaraJensen on 9/21/23, make Atlas and ecKit OPTIONAL dependencies rather than required. There is potential that we might need add an ESMF-based solution in the future. And we don't want to paint ourselves into a corner.

[JohnHalleyGotway]

During the METplus Governance Meeting on 9/27/23, @robdarvell from UK Met Office and @easatterfield from NRL both confirmed that making Atlas/ecKit optional rather than required is a good choice. This reinforces the decision last week from @TaraJensen.

[hsoh-u]

Which configuration option for ./configure?

• --enable-ugrid
• --enable-atlas
• --enable-esmf
• --enable-esmf-libs

This implies that the default is disabled

[jprestop]

Great question, @hsoh-u! I think my vote would be for "--enable-atlas", but I can see where one of the others would be preferred. I'm guessing that @TaraJensen and @JohnHalleyGotway would be better advisors than myself. I'm tagging them to help ensure they see your question.

[hsoh-u]

Thank you for the vote. Switching between --enable-atlas and --enable-ugrid is not difficult. I lean forward to --enable-ugrid. The following names will be used internally for the developers. But it may be exposed to MET users as the build documentation: WITH_UGRID or WITH_ATLAS at *cc, *h, and Makefiles, UGRID_LIBS or ATLAS_LIBS at Makefiles).

compile with/without -DWITH_ATLAS (generated by calling ./configure).
ATLAS_LIBS="-lvx_data2d_ugrid -latlas -leckit -leckit_mpi -leckit_geometry 

[JohnHalleyGotway]

Adding @jprestop as an assignee since she'll work on updates to the compile_MET_all.sh script for Atlas/ecKit.

[hsoh-u]

From Downloading and building Atlas

[ ecbuild ]

export ecbuild_ROOT=/d1/personal/hsoh/third_party/install/ecbuild
git clone https://github.com/ecmwf/ecbuild
cd ecbuild
mkdir build && cd build
cmake ../ -DCMAKE_INSTALL_PREFIX=$ecbuild_ROOT
make install

[ ecKit ]
Downloading and building - eckit
Prepare eckit source code:

wget https://github.com/ecmwf/eckit/archive/refs/tags/1.20.2.tar.gz
mv 1.20.2.tar.gz eckit-1.20.2.tar.gz
tar xvzf eckit-1.20.2.tar.gz
cd  eckit-1.20.2

Building eckit at seneca:

export PATH=$ecbuild_ROOT:$PATH
export eckit_ROOT=/d1/personal/hsoh/third_party/install/eckit
mkdir build && cd build
cmake ../ -DCMAKE_INSTALL_PREFIX=$eckit_ROOT
make install

Note: MPI is not enabled (add -DENABLE_MPI=ON to the cmake configuration line to enable MPI)

[ Atlas ]
Prepare atlas source code:

wget https://github.com/ecmwf/atlas/archive/refs/tags/0.30.0.tar.gz
mv 0.30.0.tar.gz atlas-0.30.0.tar.gz
tar xvzf atlas-0.30.0.tar.gz
cd atlas-0.30.0

Building Atlas at seneca:

export PATH=$eckit_ROOT/bin:$PATH
export atlas_ROOT=/d1/personal/hsoh/third_party/install/atlas
mkdir build && cd build
cmake ../ -DCMAKE_INSTALL_PREFIX=$atlas_ROOT
make -j8
make install

[jprestop]

Reopening so that I can add the changes to the compile_MET_all.sh script.

[JohnHalleyGotway]

@jprestop is working on this on derecho. Still planning on doing this during the beta2 dev cycle by Nov 7th.

[jprestop]

@hsoh-u I'm starting to update the compile_MET_all.sh script for ecKit and Atlas. I was looking in configure.ac in your branch to ensure that the environment variables were named MET_ECKIT and MET_ATLAS to specify the paths to the lib and include files and was checking for the -enable-ugrid option, but I don't see any of those in configure.ac. Do you have changes you haven't pushed yet or is that part of the work not completed yet?

[hsoh-u]

I built it yesterday from the feature branch (seneca:/d1/personal/hsoh/git/pull_request/MET_feature_2231_unstructured_grid).
MET_ATLAS
MET_ACKIT
--enable-ugrid

[jprestop]

Thank you, @hsoh-u. It's good to know that I was looking in the wrong branch! I appreciate the help and the confirmation of those variables and option.

[hsoh-u]

Sorry, it's not merged into the develop branch yet (it's in review status)

[jprestop]

@hsoh-u No need to be sorry at all. Being in a feature branch is totally fine. I just needed to know where to find this information to confirm. :)

[jprestop]

I have added eckit and atlas to the compilation script. I am testing on Derecho, but encountering problems with atlas being able to find ecbuild. I will be working on that today.

[jprestop]

That was an easy fix. On to the next problem which should also be an easy fix. Then I will check in the changes to my branch and will start updating the METbaseimage.

[jprestop]

Hi @hsoh-u. I checked out your branch, feature_2231_unstructured_grid, and ran bootstrap. I created a tar file to test with the changes to the compilation script. MET got through the "configure" step, the "make" step, the "make install" step, but failed on the "make test" step. I'm getting the following output in the make_test.log file:

make[1]: *** No rule to make target 'ugrid', needed by 'all'.  Stop.
make[1]: *** Waiting for unfinished jobs....

I have attached the met.make_test.log file here:
met.make_test.log

I'm not quite sure how to resolve this problem.

[hsoh-u]

The ugrid target is disabled for the "test" task.

[jprestop]

I am running ./build_docker_image.sh to test my changes in METbaseimage. It's taking a long time, but has been running well so far.

[jprestop]

This work was completed with PR #2737.