Sync develop changes March 25 - March 28 to hdf5_1_14 (#4284)

* Take user block into account when returning chunk addresses (#4236)

Both H5Dchunk_iter() and H5Dget_chunk_info(_by_coord)() did not take
the size of the user block into account when reporting addresses. Since
the #1 use of these functions is to root around in the file for the raw
data, this is kind of a problem.

Fixes GitHub issue #3003

* Fix a minor warning in h5test.c (#4242)

* Turn on -Werror for Java in GitHub -Werror workflows (#4243)

* Update Windows CI to not install ninja (#4230)

* Rework Fortran macros to use the proper code. (#4240)

* Correct reference copy for 16 API (#4244)

* Determine MPI LOGICAL during build, used in tests. (#4246)

* Skip userblock test in chunk_info.c for multi-file VFDs (#4249)

* Match generators with real cmake -G output on Windows (#4252)

* Add Julia GitHub Actions. (#4123)

* Re-revert to using autoreconf in autogen.sh (#4253)

We previously tried removing the per-tool invocation of the Autotools
and instead simply invoked autoreconf (PR #1906). This was reverted
when it turned out that the NAG Fortran compiler had trouble with an
undecorated -shared linker flag.

It turns out that this is due to a bug in libtool 2.4.2 and earlier.
Since this version of libtool is over a decade old, we're un-reverting
the change. We've added a release note for anyone who has to build
from source on elderly platforms.

Fixes #1343

* Rewrite H5T__path_find_real for clarity (#4225)

* Move conversion path free logic to helper function

* Add tgz extensions on names (#4255)

* Remove an error check regarding large cache objects (#4254)

* Remove an error check regarding large cache objects

In PR#4231 an assert() call was converted to a normal HDF5 error
check. It turns out that the original assert() was added by a
developer as a way of being alerted that large cache objects
existed instead of as a guard against incorrect behavior, making
it unnecessary in either debug or release builds.

The error check has been removed.

* Update RELEASE.txt

* File format security issues (#4234)

* Add job timeout to cygwin workflow (#4260)

* Replace user-define with user-defined (#4261)

* Improve the CMake clang -fsanitize=memory flags (#4267)

-fsanitize=memory is almost useless without
using -fsanitize-memory-track-origins=2 and we shoud probably add
-fno-optimize-sibling-calls as well.

* Add documentation (H5M) (#4259)

* Add documentation (H5P) (#4262)

* MPI type correction (#4268)

* corrected type for MPI_*_f2c APIs

* fixed return type of callback

* reset compilation flags of logical test program

* Clean up test/cmpd_dtransform.c (#4270)

* Clean up test/cmpd_dtransform.c

* Fix uninitialized memory warning from sanitizers
* FAIL_STACK_ERROR --> TEST_ERROR
* Emit output
* Delete test file when done

* Fix typo

* H5Fdelete() --> remove()

* Fix uninitialized memory issues in packet table (#4271)

* replace deprecated CMAKE_COMPILER_IS_GNU** (#4272)

* Prevent stack overflows in H5E__push_stack (#4264)

* Minor fixes after merge of file format security fixes (#4263)

* Update H5_IS_BUFFER_OVERFLOW to account for 'size' of 0

* Invert a few checks to avoid function call

* CHECK --> CHECK_PTR in tmisc.c (#4274)

* Add release note for CVE-2017-17507 (#4275)

* Update Cygwin installation guide (#4265)

* Addresses configuration fortran testing flags (#4276)

* turn warnings to errors in fortran configure test

* Intel fortran test fix

* Merge julia workflows into standard ci format (#4273)

* Fix range check in H5_addr_overlap (#4278)

When the H5_addr_overlap macro was updated to use H5_RANGE_OVERLAP,
it failed to take into account that H5_RANGE_OVERLAP expects the
range to be inclusive. This lead to an assertion failure in
H5MM_memcpy due to a memcpy operation on overlapping memory.
This has been fixed by subtracting 1 from the calculated high
bound values passed to H5_RANGE_OVERLAP

* Fix potential buffer read overflows in H5PB_read (#4279)

H5PB_read previously did not account for the fact that the size of the
read it's performing could overflow the page buffer pointer, depending
on the calculated offset for the read. This has been fixed by adjusting
the size of the read if it's determined that it would overflow the page.

.github/workflows/autotools.yml

@@ -83,4 +83,10 @@ jobs:
       with:
             build_mode: "production"
 
+    call-release-cmake-julia:
+        name: "Autotools Julia Workflows"
+        uses: ./.github/workflows/julia-auto.yml
+        with:
+            build_mode: "production"
+
 

.github/workflows/cmake.yml

@@ -85,4 +85,10 @@ jobs:
         uses: ./.github/workflows/cygwin-cmake.yml
         with:
             build_mode: "Release"
+
+    call-release-cmake-julia:
+        name: "CMake Julia Workflows"
+        uses: ./.github/workflows/julia-cmake.yml
+        with:
+            build_mode: "Release"
 

.github/workflows/cygwin-auto.yml

@@ -15,6 +15,7 @@ jobs:
   cygwin_build_and_test:
     name: "cygwin ${{ inputs.build_mode }}"
     runs-on: windows-latest
+    timeout-minutes: 30
     steps:
       - name: Set git to use LF
         run: |

.github/workflows/cygwin-cmake.yml

@@ -15,6 +15,7 @@ jobs:
   cygwin_build_and_test:
     name: "cygwin-${{ inputs.build_mode }}"
     runs-on: windows-latest
+    timeout-minutes: 30
     steps:
       - name: Set git to use LF
         run: |

.github/workflows/julia-auto.yml

@@ -0,0 +1,79 @@
+name: hdf5 1.14 autotools julia
+
+on:
+  workflow_call:
+    inputs:
+      build_mode:
+        description: "release vs. debug build"
+        required: true
+        type: string
+
+permissions:
+  contents: read
+
+jobs:
+  julia_build_and_test:
+    name: "julia ${{ inputs.build_mode }}"
+    runs-on: ubuntu-latest
+    steps:
+      - name: Get Sources
+        uses: actions/[email protected]
+
+      - name: Install Dependencies
+        shell: bash
+        run: |
+          sudo apt-get update
+          sudo apt-get install autoconf automake libtool libtool-bin libaec-dev
+          sudo apt-get install doxygen graphviz
+          sudo apt install -y zlib1g-dev libcurl4-openssl-dev libjpeg-dev wget curl bzip2
+          sudo apt install -y m4 flex bison cmake libzip-dev openssl build-essential
+
+      - name: Autotools Configure
+        shell: bash
+        run: |
+          sh ./autogen.sh
+          mkdir "${{ runner.workspace }}/build"
+          cd "${{ runner.workspace }}/build"
+          $GITHUB_WORKSPACE/configure \
+            --enable-build-mode=${{ inputs.build_mode }} \
+            --disable-fortran \
+            --enable-shared \
+            --disable-parallel \
+            --prefix=/tmp
+
+      - name: Autotools Build
+        shell: bash
+        run: |
+          make -j3
+        working-directory: ${{ runner.workspace }}/build
+
+      - name: Install HDF5
+        shell: bash
+        run: |
+          make install
+        working-directory: ${{ runner.workspace }}/build
+
+      - name: Install julia
+        uses: julia-actions/setup-julia@latest
+        with:
+          version: '1.6'
+          arch: 'x64'
+
+      - name: Get julia hdf5 source
+        uses: actions/[email protected]
+        with:
+          repository: JuliaIO/HDF5.jl
+          path: .
+
+      - name: Generate LocalPreferences
+        run: |
+          echo '[HDF5]' >> LocalPreferences.toml
+          echo 'libhdf5 = "/tmp/lib/libhdf5.so"' >> LocalPreferences.toml
+          echo 'libhdf5_hl = "/tmp/lib/libhdf5_hl.so"' >> LocalPreferences.toml
+
+      - uses: julia-actions/julia-buildpkg@latest
+
+      - name: Julia Run Tests
+        uses: julia-actions/julia-runtest@latest
+        env:
+          JULIA_DEBUG: Main

.github/workflows/julia-cmake.yml

@@ -0,0 +1,82 @@
+name: hdf5 1.14 CMake julia
+
+on:
+  workflow_call:
+    inputs:
+      build_mode:
+        description: "release vs. debug build"
+        required: true
+        type: string
+
+permissions:
+  contents: read
+
+jobs:
+  julia_build_and_test:
+    name: "julia ${{ inputs.build_mode }}"
+    runs-on: ubuntu-latest
+    steps:
+      - name: Get Sources
+        uses: actions/[email protected]
+
+      - name: Install Dependencies
+        shell: bash
+        run: |
+          sudo apt update
+          sudo apt-get install ninja-build doxygen graphviz
+          sudo apt install libssl3 libssl-dev libcurl4 libcurl4-openssl-dev
+          sudo apt install -y libaec-dev zlib1g-dev wget curl bzip2 flex bison cmake libzip-dev openssl build-essential
+
+      - name: CMake Configure
+        shell: bash
+        run: |
+          mkdir "${{ runner.workspace }}/build"
+          cd "${{ runner.workspace }}/build"
+          cmake -C $GITHUB_WORKSPACE/config/cmake/cacheinit.cmake -G Ninja \
+          -DCMAKE_BUILD_TYPE=${{ inputs.build_mode }} \
+          -DHDF5_ENABLE_SZIP_SUPPORT:BOOL=OFF \
+          -DHDF5_ENABLE_PARALLEL:BOOL=OFF \
+          -DHDF5_BUILD_CPP_LIB:BOOL=OFF \
+          -DLIBAEC_USE_LOCALCONTENT=OFF \
+          -DZLIB_USE_LOCALCONTENT=OFF \
+          -DHDF5_BUILD_FORTRAN:BOOL=OFF \
+          -DHDF5_BUILD_JAVA:BOOL=OFF \
+          -DCMAKE_INSTALL_PREFIX=/tmp \
+          $GITHUB_WORKSPACE
+
+      - name: CMake Build
+        shell: bash
+        run: |
+          cmake --build . --parallel 3 --config ${{ inputs.build_mode }}
+        working-directory: ${{ runner.workspace }}/build
+
+      - name: Install HDF5
+        shell: bash
+        run: |
+          cmake --install .
+        working-directory: ${{ runner.workspace }}/build
+
+      - name: Install julia
+        uses: julia-actions/setup-julia@latest
+        with:
+          version: '1.6'
+          arch: 'x64'
+
+      - name: Get julia hdf5 source
+        uses: actions/[email protected]
+        with:
+          repository: JuliaIO/HDF5.jl
+          path: .
+
+      - name: Generate LocalPreferences
+        run: |
+          echo '[HDF5]' >> LocalPreferences.toml
+          echo 'libhdf5 = "/tmp/lib/libhdf5.so"' >> LocalPreferences.toml
+          echo 'libhdf5_hl = "/tmp/lib/libhdf5_hl.so"' >> LocalPreferences.toml
+
+      - uses: julia-actions/julia-buildpkg@latest
+
+      - name: Julia Run Tests
+        uses: julia-actions/julia-runtest@latest
+        env:
+          JULIA_DEBUG: Main

.github/workflows/main-auto-spc.yml

@@ -417,8 +417,10 @@ jobs:
           sh ./autogen.sh
           mkdir "${{ runner.workspace }}/build"
           cd "${{ runner.workspace }}/build"
-          CFLAGS=-Werror $GITHUB_WORKSPACE/configure \
+          CFLAGS=-Werror JAVACFLAGS=-Werror JNIFLAGS=-Werror \
+          $GITHUB_WORKSPACE/configure \
             --enable-build-mode=debug \
+            --enable-warnings-as-errors \
             --enable-deprecated-symbols \
             --with-default-api-version=v114 \
             --enable-shared \
@@ -478,8 +480,10 @@ jobs:
           sh ./autogen.sh
           mkdir "${{ runner.workspace }}/build"
           cd "${{ runner.workspace }}/build"
-          CFLAGS=-Werror $GITHUB_WORKSPACE/configure \
+          CFLAGS=-Werror JAVACFLAGS=-Werror JNIFLAGS=-Werror \
+          $GITHUB_WORKSPACE/configure \
             --enable-build-mode=production \
+            --enable-warnings-as-errors \
             --enable-deprecated-symbols \
             --with-default-api-version=v114 \
             --enable-shared \

.github/workflows/main-cmake.yml

@@ -135,10 +135,6 @@ jobs:
 
       # CMake gets libaec from fetchcontent
 
-      - name: Install Dependencies (Windows)
-        run: choco install ninja
-        if: matrix.os == 'windows-latest'
-
       - name: Install Dependencies (macOS)
         run: brew install ninja
         if: matrix.os == 'macos-13'