Add support for flang

[mathomp4]

Due to successes by @tclune in looking at flang + GFE (with a shoutout to @rouson), we should work on getting flang supported in Baselibs.

Current progress:

-------+---------+---------+--------------
Config | Install |  Check  |   Package
-------+---------+---------+--------------
  ok   |   ok    |   --    | antlr2
  ok   |   ok    |   --    | gsl
  ok   |   ok    |   --    | jpeg
  ok   |   ok    |   --    | zlib
  ok   |   ok    |   --    | szlib
  ok   |   ok    |   --    | curl
  ok   |   ok    |   --    | hdf4
  ok   |   --    |   --    | hdf5
  --   |   --    |   --    | netcdf
  --   |   --    |   --    | netcdf-fortran
  --   |   --    |   --    | netcdf-cxx4
  ok   |   ok    |   --    | udunits2
  --   |   --    |   --    | nco
  --   |   --    |   --    | cdo
  --   |   --    |   --    | nccmp
  --   |   --    |   --    | libyaml
  --   |   --    |   --    | FMS
  --   |   --    |   --    | esmf
  ok   |   ok    |   --    | xgboost
  ok   |   ok    |   --    | GFE
  --   |   --    |   --    | hdfeos
  --   |   --    |   --    | hdfeos5
  --   |   --    |   --    | SDPToolkit
-------+---------+---------+--------------

[mathomp4]

At the moment we have a couple issues. One, we need to update HDF5 as 1.10 has no idea what flang is. We've held on to 1.10 for now only because of the odd nf90_create issues we saw in GEOS with 1.14.

But, we know GEOS is happy with hdf5 1.14 from spack, so let's just update for testing.

However, I just tried to build HDF5 1.14.4.3 and I believe I hit the same bug as @mjklemm did in llvm/llvm-project#89403. (Indeed, I was about to file a bug in llvm-compiler but, well, I was beaten by a good 3 months!)

This code was added in HDFGroup/hdf5#4246 by @derobins via @brtnfld ad by the time stamp, I'm guessing this is in 1.14.4 range, so I'll back off to 1.14.3 before this code got in.

[mjklemm]

I have a patch for this ready hopefully soon. I was more work than anticipated, but the patch seems to now work for Linux. The Windows part is still in the works.

PR: llvm/llvm-project#95388

PS: Don't try this yet, it's still in the works. I can ping here, once it's ready for a test.

[mathomp4]

I have a patch for this ready hopefully soon. I was more work than anticipated, but the patch seems to now work for Linux. The Windows part is still in the works.

PR: llvm/llvm-project#95388

Thanks!

Note with 1.14.3:

flang-new: warning: argument unused during compilation: '-pthread' [-Wunused-command-line-argument]
error: loc("/ford1/share/gmao_SIteam/Baselibs/ESMA-Baselibs-8.5.0/src/hdf5/fortran/test/tf_gen.F90":208:5): /ford1/share/gmao_SIteam/llvm-flang/tmp/llvm-src/llvm-project-main/flang/include/flang/Optimizer/Support/Utils.h:143: not yet implemented: intrinsic: REAL(KIND=2) in SPACING
LLVM ERROR: aborting
make[4]: *** [Makefile:1054: tf_gen.lo] Error 1

I search a bit and I find something on the hdf5 site by a developer of the name, huh, @mjklemm : HDFGroup/hdf5#4419

So, yeah, I'm definitely late to this party! 😄

[mathomp4]

Note: I'm also using autotools for HDF5 (for...historical reasons more than anything else) so I might be missing errors that CMake would find in different ways.

[mjklemm]

I have a patch for that, too. Problem is that Flang reports that it supports FP16 and BF16 types, which it does for regular arithmetics, but it does not provide all the intrinsic. SPACING being one of them.

I have a hacks patch that disables the tests in HDF5. Let me know if you'd like to get it and I will see how I can best provide it.

Logged at LLVM as llvm/llvm-project#83894.

[brtnfld]

There are other issues with flang and HDF5,
llvm/llvm-project#83894
llvm/llvm-project#56883

[mathomp4]

Adding a comment here about a doc by @jeffhammond saying how he's built HDF5 with flang:

https://github.com/jeffhammond/HPCInfo/blob/master/docs/NetCFD_HDF5_Flang.md

I'll need to give that a try at some point.

[jeffhammond]

I'll note my effort is the same as AMD folks did two weeks ago.

[brtnfld]

Did all the Fortran HDF5 tests pass?

[mathomp4]

Did all the Fortran HDF5 tests pass?

Haven't had a chance to test on my end yet. Other fires being put out at the moment. Whee!

[mathomp4]

Did all the Fortran HDF5 tests pass?

@brtnfld Good news, bad news. First, I built flang-new as of yesterday:

> flang-new --version
flang-new version 20.0.0git ([email protected]:GMAO-SI-Team/build-llvm-flang.git a39b3317362451d53d98542a2760451767a772d8)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /ford1/share/gmao_SIteam/llvm-flang/2024-08-21/bin

With that and with HDF5 1.14.4.3 and using the "comment out REAL(2) and REAL(3) bits in hdf5/fortran/test/tf_gen.F90 (per @jeffhammond) I get past those. However, I now die at:

make[4]: Entering directory '/ford1/share/gmao_SIteam/Baselibs/ClangTest/ESMA-Baselibs-8.6.0-TEST/src/hdf5/fortran/testpar'
  PPFC     mpi_param.o
flang-new: warning: argument unused during compilation: '-pthread' [-Wunused-command-line-argument]
error: loc("/ford1/share/gmao_SIteam/MPI/openmpi/5.0.4/llvm-flang-2024-08-21/lib/mpi_f08.mod":408:24): 'hlfir.declare' op of numeric, logical, or assumed type entity must not have length parameters
error: verification of lowering to FIR failed
make[4]: *** [Makefile:1337: mpi_param.o] Error 1

I see H5_MPI_LOGICAL_KIND in there, so I have the latest version of that file.

So I guess two things:

1. Whose error is this? Flang's? I think "flang" when I see "hlfir".
2. Is there a way around this? Or is this such an important test I don't want to get around it?

[mathomp4]

I did check and I can do a use mpi_f08 version of Hello World, so it's not like mpi_f08 does not work. So that's nice.

[brtnfld]

Do you have the same issue if you use mpich (compiled with flang) instead of openmpi?

[mathomp4]

Do you have the same issue if you use mpich (compiled with flang) instead of openmpi?

Hmm. That is a good question. Let me try and see! (Need to brush up on my "how to build mpich"... 😄 )

[mathomp4]

Update: I built MPICH but it didn't build mpi_f08 support. I'm honestly not sure if that's expected. I didn't pass in --disable-f08 so autotools must have disabled it?

Still, my guess is HDF5 doesn't yet require that, so I'll move forward for now.

[mathomp4]

Update again! MPICH got past the place that Open MPI failed on, so I guess mpi_f08 support isn't required.

I'm going to try make check now to see what happens...

[brtnfld]

Correct, mpi_f08 support is not required for HDF5.

[mathomp4]

Update as of 2024-Oct-23. Built new flang-new and for HDF5, still need to comment out the REAL(KIND=2) and REAL(KIND=3) bits per @jeffhammond. Though maybe the logical bits I had to do before are okay now.

[mathomp4]

Okay, with some hacks to hdf5 and esmf, they build. But a test of MAPL showed a change in flang-new behavior, see llvm/llvm-project#113487

It's like flang OpenMP has...changed. I'll see what they say...

[Krande]

Hey @mathomp4, we're also looking into building hdf5 using flang over at conda-forge/hdf5-feedstock#217 (however in that PR we're focusing mainly on windows).

Just wondering if you have managed to successfully compile hdf5 using flang on linux? And if so, didn't you run into this issue?

[444/3074] Building Fortran object fortran\src\CMakeFiles\hdf5_fortran-shared.dir\H5VLff.F90.obj
FAILED: fortran/src/CMakeFiles/hdf5_fortran-shared.dir/H5VLff.F90.obj mod/shared/h5vl.mod 
%BUILD_PREFIX%\Library\bin\flang-new.exe -I%SRC_DIR%\fortran\src -I%SRC_DIR%\fortran\src -I%SRC_DIR%\build\mod\shared -I%SRC_DIR%\build\fortran -I%SRC_DIR%\build\fortran\src -I%SRC_DIR%\src -I%SRC_DIR%\src\H5FDsubfiling -I%SRC_DIR%\build\src -I%PREFIX%\Library\include -D_CRT_SECURE_NO_WARNINGS -D_MT -D_DLL --target=x86_64-pc-windows-msvc -fms-runtime-lib=dll -module-dirmod\shared -ffixed-line-length-72 -o fortran\src\CMakeFiles\hdf5_fortran-shared.dir\H5VLff.F90.obj -c fortran\src\CMakeFiles\hdf5_fortran-shared.dir\H5VLff.F90-pp.f90
error: loc("D:\\bld\\hdf5_1730135420238\\work\\fortran\\src\\H5VLff.F90":456:32): C:\bld\flang-split_1725521580766\work\flang\lib\Optimizer\CodeGen\Target.cpp:102: not yet implemented: passing VALUE BIND(C) derived type for this target
LLVM ERROR: aborting

[mathomp4]

@Krande Huh. That is a new error to me. I'm still disabling all the REAL(2), REAL(3), and MPI_LOGICAL_KIND in a few places, but it does build.

[Krande]

Interesting. I'll try to compile HDF5 on linux too using the same changes I made to the source code and see if that turns out differently. For reference I was using HDF5 v1.14.5 and flang v19.1.2. I see you mentioned that you were running the latest flang from main. I'll probably give that a go as well. Which HDF5 version did you compile?

[mathomp4]

@Krande It would have been HDF5 1.14.5 as well. But this was, as you say, flang 20.0.0git of whatever it calls the version from git (as of last week).

Note that I'm not quite good enough at autotools or whatever to figure out how to fix the few files I need to correctly in source. From what I see, the changes I make are to files generated during the build. So sadly git diff shows nothing :(