Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API test updates #3018

Merged
merged 19 commits into from
May 26, 2023
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
51d01d7
Remove macros from api tests (#2929)
glennsong09 May 11, 2023
ba6debe
Remove macros and undefined callbacks (#2959)
glennsong09 May 15, 2023
dd91659
Remove remaining macros from H5_api_tests_disabled.h (#2968)
glennsong09 May 18, 2023
5e3adcc
Put some vol capability checks in testpar tests and remove remaining …
glennsong09 May 23, 2023
8f09995
API tests datatype generation cleanup
jhendersonHDF May 11, 2023
b3f47d9
Init parallel API tests with MPI_THREAD_MULTIPLE
jhendersonHDF May 12, 2023
7cc1743
HDF5 API tests - Check VOL connector registration
jhendersonHDF May 12, 2023
3e9e275
Cleanup some usages of H5VL_CAP_FLAG_CREATION_ORDER in API tests
jhendersonHDF May 15, 2023
7c044d1
Remove some now-unused macros from H5_api_tests_disabled.h
jhendersonHDF May 15, 2023
85b7ec0
Enable HDF5 API tests by default
jhendersonHDF May 15, 2023
7d44ac8
Implement CMake option to install HDF5 API tests
jhendersonHDF May 15, 2023
009e66b
Check for invalid AAPL from H5Acreate
jhendersonHDF May 22, 2023
6b0e116
Enable building of VOL connectors alongside HDF5 in CMake
jhendersonHDF May 23, 2023
8a43e33
Prepend CMake VOL URL option indices with 0s so they come in order
jhendersonHDF May 25, 2023
0d5c7f9
Don't turn on API tests by default yet
jhendersonHDF May 25, 2023
b0e0ba6
Document VOL connector FetchContent functionality
jhendersonHDF May 26, 2023
ef1c960
Add release note for API test updates
jhendersonHDF May 26, 2023
8d48dc0
Only install testing library if API tests are installed
jhendersonHDF May 26, 2023
1a1bd6d
Fix grammar
jhendersonHDF May 26, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -844,6 +844,11 @@ include (UserMacros.cmake)
#-----------------------------------------------------------------------------
include (CMakeFilters.cmake)

#-----------------------------------------------------------------------------
# Include external VOL connectors
#-----------------------------------------------------------------------------
include (CMakeVOL.cmake)

#-----------------------------------------------------------------------------
# Option for external libraries on windows
#-----------------------------------------------------------------------------
Expand Down
180 changes: 180 additions & 0 deletions CMakeVOL.cmake
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new file contains all the code to generate CMake options and variables for specifying external VOL connectors and then pulling in, building those connectors and setting up for later testing of the connectors.

Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
#
# Copyright by The HDF Group.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the COPYING file, which can be found at the root of the source code
# distribution tree, or in https://www.hdfgroup.org/licenses.
# If you do not have access to either file, you may request a copy from
# [email protected].
#

include (FetchContent)

# Function to retrieve all of the CMake targets generated
# in a directory and all its subdirectories
function (get_generated_cmake_targets out_var dir)
get_directory_property (dir_targets DIRECTORY "${dir}" BUILDSYSTEM_TARGETS)
get_directory_property (dir_subdirs DIRECTORY "${dir}" SUBDIRECTORIES)

foreach (subdir ${dir_subdirs})
get_generated_cmake_targets(subdir_targets "${subdir}")
list (APPEND dir_targets "${subdir_targets}")
endforeach()

set (${out_var} "${dir_targets}" PARENT_SCOPE)
endfunction ()

# For now, only support building of external VOL connectors with FetchContent
option (HDF5_VOL_ALLOW_EXTERNAL "Allow building of external HDF5 VOL connectors with FetchContent" OFF)
mark_as_advanced (HDF5_VOL_ALLOW_EXTERNAL)
if (HDF5_VOL_ALLOW_EXTERNAL)
if (HDF5_ALLOW_EXTERNAL_SUPPORT MATCHES "NO" OR NOT HDF5_ALLOW_EXTERNAL_SUPPORT MATCHES "GIT")
message (FATAL_ERROR "HDF5_ALLOW_EXTERNAL_SUPPORT must be set to 'GIT' to allow building of external HDF5 VOL connectors")
endif ()

# For compatibility, set some variables that projects would
# typically look for after calling find_package(HDF5)
set (HDF5_FOUND 1)
set (HDF5_LIBRARIES "${HDF5_LIBSH_TARGET};${LINK_LIBS};${LINK_COMP_LIBS};$<$<BOOL:${HDF5_ENABLE_PARALLEL}>:MPI::MPI_C>")
set (HDF5_INCLUDE_DIRS "${HDF5_SRC_INCLUDE_DIRS};${HDF5_SRC_BINARY_DIR};$<$<BOOL:${HDF5_ENABLE_PARALLEL}>:${MPI_C_INCLUDE_DIRS}>")

set (HDF5_MAX_EXTERNAL_VOLS 10)
set (HDF5_EXTERNAL_VOL_TARGETS "")

foreach (vol_idx RANGE 1 ${HDF5_MAX_EXTERNAL_VOLS})
# Generate fixed-width index number prepended with 0s
# so URLs come in order from 1 - HDF5_MAX_EXTERNAL_VOLS
set (vol_idx_num_digits 2) # Based on HDF5_MAX_EXTERNAL_VOLS
set (vol_idx_fixed "${vol_idx}")
string (LENGTH "${vol_idx_fixed}" vol_idx_len)
while (vol_idx_len LESS vol_idx_num_digits)
string (PREPEND vol_idx_fixed "0")
math (EXPR vol_idx_len "${vol_idx_len}+1")
endwhile ()

set (HDF5_VOL_URL${vol_idx_fixed} "" CACHE STRING "Git repository URL of an external HDF5 VOL connector to build")
mark_as_advanced (HDF5_VOL_URL${vol_idx_fixed})

if (NOT "${HDF5_VOL_URL${vol_idx_fixed}}" STREQUAL "")
# Extract the name of the VOL connector
string (FIND "${HDF5_VOL_URL${vol_idx_fixed}}" "/" hdf5_vol_name_pos REVERSE)
if (hdf5_vol_name_pos EQUAL -1)
message (SEND_ERROR "Invalid URL '${HDF5_VOL_URL${vol_idx_fixed}}' specified for HDF5_VOL_URL${vol_idx_fixed}")
endif ()

math (EXPR hdf5_vol_name_pos "${hdf5_vol_name_pos}+1")

string (SUBSTRING "${HDF5_VOL_URL${vol_idx_fixed}}" ${hdf5_vol_name_pos} -1 hdf5_vol_name)
string (REPLACE ".git" "" hdf5_vol_name "${hdf5_vol_name}")
string (STRIP "${hdf5_vol_name}" hdf5_vol_name)
string (TOUPPER "${hdf5_vol_name}" hdf5_vol_name_upper)
string (TOLOWER "${hdf5_vol_name}" hdf5_vol_name_lower)

message (VERBOSE "Building VOL connector '${hdf5_vol_name}' with FetchContent")

# Set some cache variables that can be set by users when building
set ("HDF5_VOL_${hdf5_vol_name_upper}_NAME" "" CACHE STRING "Name of VOL connector to set for the HDF5_VOL_CONNECTOR environment variable")
set ("HDF5_VOL_${hdf5_vol_name_upper}_BRANCH" "main" CACHE STRING "Git branch (or tag) to use when building VOL connector '${hdf5_vol_name}'")
option ("HDF5_VOL_${hdf5_vol_name_upper}_TEST_PARALLEL" "Whether to test VOL connector '${hdf5_vol_name}' against the parallel API tests" OFF)

mark_as_advanced ("HDF5_VOL_${hdf5_vol_name_upper}_NAME")
mark_as_advanced ("HDF5_VOL_${hdf5_vol_name_upper}_BRANCH")
mark_as_advanced ("HDF5_VOL_${hdf5_vol_name_upper}_TEST_PARALLEL")

if (HDF5_TEST_API)
if ("${HDF5_VOL_${hdf5_vol_name_upper}_NAME}" STREQUAL "")
message (SEND_ERROR "HDF5_VOL_${hdf5_vol_name_upper}_NAME must be set to a valid connector name to use VOL connector '${hdf5_vol_name}' for testing")
endif ()
endif ()

if ("${HDF5_VOL_${hdf5_vol_name_upper}_BRANCH}" STREQUAL "")
message (SEND_ERROR "HDF5_VOL_${hdf5_vol_name_upper}_BRANCH must be set to a valid git branch name (or git tag) to build VOL connector '${hdf5_vol_name}'")
endif ()

FetchContent_Declare (HDF5_VOL_${hdf5_vol_name_lower}
GIT_REPOSITORY "${HDF5_VOL_URL${vol_idx_fixed}}"
GIT_TAG "${HDF5_VOL_${hdf5_vol_name_upper}_BRANCH}"
)

FetchContent_GetProperties(HDF5_VOL_${hdf5_vol_name_lower})
if (NOT hdf5_vol_${hdf5_vol_name_lower}_POPULATED)
FetchContent_Populate(HDF5_VOL_${hdf5_vol_name_lower})

if (NOT EXISTS "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/CMakeLists.txt")
message (SEND_ERROR "The git repository branch '${HDF5_VOL_${hdf5_vol_name_upper}_BRANCH}' for VOL connector '${hdf5_vol_name}' does not appear to contain a CMakeLists.txt file")
endif ()

# If there are any calls to find_package(HDF5) in the connector's
# CMakeLists.txt files, remove those since any found HDF5 targets
# will conflict with targets being generated by this build of HDF5
if (EXISTS "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/CMakeLists.txt")
file (READ "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/CMakeLists.txt" vol_cmake_contents)
string (REGEX REPLACE "[ \t]*find_package[ \t]*\\([ \t]*HDF5[^\r\n\\)]*\\)[ \t]*[\r\n]+" "" vol_cmake_contents "${vol_cmake_contents}")
file (WRITE "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/CMakeLists.txt" "${vol_cmake_contents}")
endif ()
if (EXISTS "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/src/CMakeLists.txt")
file (READ "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/src/CMakeLists.txt" vol_cmake_contents)
string (REGEX REPLACE "[ \t]*find_package[ \t]*\\([ \t]*HDF5[^\r\n\\)]*\\)[ \t]*[\r\n]+" "" vol_cmake_contents "${vol_cmake_contents}")
file (WRITE "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/src/CMakeLists.txt" "${vol_cmake_contents}")
endif ()

add_subdirectory (${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR} ${hdf5_vol_${hdf5_vol_name_lower}_BINARY_DIR})

# Get list of targets generated by build of connector
get_generated_cmake_targets (connector_targets ${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR})

# Create a custom target for the connector to encompass all its
# targets and other custom properties set by us for later use
add_custom_target ("HDF5_VOL_${hdf5_vol_name_lower}")

# Define and set a custom property on the VOL connector target to
# capture all of the connector's generated targets
define_property (
TARGET
PROPERTY HDF5_VOL_TARGETS
)

set_target_properties (
"HDF5_VOL_${hdf5_vol_name_lower}"
PROPERTIES
HDF5_VOL_TARGETS "${connector_targets}"
)

# Define and set a custom property on the VOL connector target to
# capture the connector's name to set for the HDF5_VOL_CONNECTOR
# environment variable for testing
define_property (
TARGET
PROPERTY HDF5_VOL_NAME
BRIEF_DOCS "VOL connector name to use for the HDF5_VOL_CONNECTOR environment variable when testing"
)

set_target_properties (
"HDF5_VOL_${hdf5_vol_name_lower}"
PROPERTIES
HDF5_VOL_NAME "${HDF5_VOL_${hdf5_vol_name_upper}_NAME}"
)

# Define and set a custom property on the VOL connector target to
# capture whether the connector should be tested with the parallel
# API tests
define_property (
TARGET
PROPERTY HDF5_VOL_TEST_PARALLEL
BRIEF_DOCS "Whether the VOL connector should be tested with the parallel API tests"
)

set_target_properties (
"HDF5_VOL_${hdf5_vol_name_lower}"
PROPERTIES
HDF5_VOL_TEST_PARALLEL ${HDF5_VOL_${hdf5_vol_name_upper}_TEST_PARALLEL}
)

# Add this connector's target to the list of external connector targets
list (APPEND HDF5_EXTERNAL_VOL_TARGETS "HDF5_VOL_${hdf5_vol_name_lower}")
endif ()
endif ()
endforeach ()
endif ()
195 changes: 195 additions & 0 deletions doc/cmake-vols-fetchcontent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
# Building and testing HDF5 VOL connectors with CMake FetchContent

This document details the process of using CMake options to build and test
an HDF5 VOL connector alongside the HDF5 library when building HDF5 from
source. There are several benefits that this may provide, but among them
are the following:

* A VOL connector built this way can be tested at the same time that
HDF5 is, which eliminates the need to have a multi-step build process
where one builds HDF5, uses it to build the VOL connector and then
uses the external [HDF5 VOL tests](https://github.com/hdfGroup/vol-tests)
repository to test their connector.
* Building VOL connectors in this manner will usually install the built
connector library alongside the HDF5 library, allowing future opportunities
for HDF5 to set a default plugin path such that the HDF5_PLUGIN_PATH
environment variable doesn't need to be set.

## Building

To enable building of an HDF5 VOL connector using HDF5's CMake functionality,
two CMake variables must first be set:

HDF5_VOL_ALLOW_EXTERNAL (Default: OFF)
This variable determines whether or not building of external HDF5 VOL connectors
is enabled.

HDF5_ALLOW_EXTERNAL_SUPPORT (Default: "NO")
This variable is a string that specifies the manner in which the source code for
an external VOL connector will be retrieved. Currently, this variable must be set
to "GIT" for building external VOL connectors.

Once the `HDF5_VOL_ALLOW_EXTERNAL` option is set to ON and the `HDF5_ALLOW_EXTERNAL_SUPPORT`
variable is set to "GIT", the CMake cache will be populated with a predefined
(currently 10) amount of new variables, named:

HDF5_VOL_URL01
HDF5_VOL_URL02
HDF5_VOL_URL03
...

For each of these variables, a URL that points to an HDF5 VOL connector Git
repository can be specified. These URLs should currently be HTTPS URLs. For
example, to specify the HDF5 Asynchronous I/O VOL Connector developed by the
ECP team, one can provide the following option to `cmake`:

-DHDF5_VOL_URL01=https://github.com/hpc-io/vol-async.git

For each URL specified, HDF5's CMake code will attempt to use CMake's
[FetchContent](https://cmake.org/cmake/help/latest/module/FetchContent.html)
functionality to retrieve the source code for a VOL connector pointed to by
that URL and will try to build that VOL connector as part of the HDF5 library
build process. The VOL connector must be able to be built by CMake and currently
must have a CMakeLists.txt file in the top level of the source tree in order to
be buildable by this process. If the source code for a VOL connector is successfully
retrieved, the HDF5 build's CMake cache will be populated with variables from
the VOL connector's CMake code, as if one were building the connector by itself.
This gives one the ability to customize the build of the connector as usual.

The CMake cache will also be populated with a few new variables for each VOL
connector that was successfully retrieved from a given URL. To generate these
variables, the CMake code first creates an internal name for the VOL connector
by stripping off the last part of the Git repository URL given for the connector,
removing the ".git" suffix and any whitespace and then upper-casing the result.
For example, the name of the VOL connector located at the URL
https://github.com/hpc-io/vol-async.git would become "VOL-ASYNC". Then, the following
new variables get created:

HDF5_VOL_<VOL name>_BRANCH (Default: "main")
This variable specifies the git branch name or tag to use when fetching
the source code for the VOL connector with the CMake-internal name
'<VOL name>'.

HDF5_VOL_<VOL name>_NAME (Default: "")
This variable specifies the string that should be used when setting the
HDF5_VOL_CONNECTOR environment variable for testing the VOL connector
with the CMake-internal name '<VOL name>'. The value for this variable
can be determined according to the canonical name given to the connector
by the connector's author(s), as well as any extra info that needs to be
passed to the connector for its configuration (see example below). This
variable must be set in order for the VOL connector to be testable with
HDF5's tests.

HDF5_VOL_<VOL name>_TEST_PARALLEL (Default: OFF)
This variable determines whether the VOL connector with the CMake-internal
name '<VOL name>' should be tested against HDF5's parallel tests.

As an example, this would create the following variables for the
previously-mentioned VOL connector:

HDF5_VOL_VOL-ASYNC_BRANCH
HDF5_VOL_VOL-ASYNC_NAME
HDF5_VOL_VOL-ASYNC_TEST_PARALLEL

**NOTE**
If a VOL connector requires extra information to be passed in its
HDF5_VOL_<VOL name>_NAME variable and that information contains any semi-colons,
those semi-colons should be escaped with a single backslash so that CMake
doesn't parse the string as a list. If `cmake` is run from a shell, extra care
may need to be taken when escaping the semi-colons depending on how the
shell interprets backslashes.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'semicolons' shouldn't have a '-'

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


### Example - Build and test HDF5 Asynchronous I/O VOL connector

Assuming that the HDF5 source code has been checked out and a build directory
has been created, running the following cmake command from that build directory
will retrieve, build and test the HDF5 Asynchronous I/O VOL connector while
building HDF5. Note that `[hdf5 options]` represents other build options that
would typically be passed when building HDF5, such as `CMAKE_INSTALL_PREFIX`,
`HDF5_BUILD_CPP_LIB`, etc.

cmake [hdf5 options]
-DHDF5_ENABLE_THREADSAFE=ON
-DHDF5_ENABLE_PARALLEL=ON
-DALLOW_UNSUPPORTED=ON
-DHDF5_TEST_API=ON
-DHDF5_VOL_ALLOW_EXTERNAL=ON
-DHDF5_ALLOW_EXTERNAL_SUPPORT="GIT"
-DHDF5_VOL_URL01=https://github.com/hpc-io/vol-async.git
-DHDF5_VOL_VOL-ASYNC_BRANCH=develop
-DHDF5_VOL_VOL-ASYNC_NAME="async under_vol=0\;under_info={}"
-DHDF5_VOL_VOL-ASYNC_TEST_PARALLEL=ON ..

Here, we are specifying that:

* HDF5 should be built with thread-safety enabled (required by Async VOL connector)
* HDF5 should be built with parallel enabled (required by Async VOL connector)
* Allow unsupported HDF5 combinations (thread-safety and HL, which is on by default)
* Enable the API tests so that they can be tested with the Async VOL connector
* Build and use the HDF5 Asynchronous I/O VOL connector, located at
https://github.com/hpc-io/vol-async.git
* Clone the Asynchronous I/O VOL connector from the repository's 'develop' branch
* When testing the Asynchronous I/O VOL connector, the `HDF5_VOL_CONNECTOR` environment
variable should be set to "async under_vol=0\;under_info={}", which
specifies that the VOL connector with the canonical name "async" should
be loaded and it should be passed the string "under_vol=0;under_info={}"
for its configuration
* The Asynchronous I/O VOL connector should be tested against HDF5's parallel API tests

Note that this also assumes that the Asynchronous I/O VOL connector's
[other dependencies](https://hdf5-vol-async.readthedocs.io/en/latest/gettingstarted.html#preparation)
are installed on the system in a way that CMake can find them. If that is not
the case, the locations for these dependencies may need to be provided to CMake
by passing extra options, such as:

-DABT_INCLUDE_DIR=/path/to/argobots/build/include
-DABT_LIBRARY=/path/to/argbots/build/lib/libabt.so

which would help CMake find an argobots installation in a non-standard location.

## Testing

To facilitate testing of HDF5 VOL connectors when building HDF5, tests from
the [HDF5 VOL tests](https://github.com/hdfGroup/vol-tests) repository were
integrated back into the library and the following new CMake options were
added to HDF5 builds for the 1.14.1 release:

HDF5_TEST_API (Default: OFF)
This variable determines whether the HDF5 API tests will be built and tested.

HDF5_TEST_API_INSTALL (Default: OFF)
This variable determines whether the HDF5 API test executables will be installed
on the system alongside the HDF5 library.

HDF5_TEST_API_ENABLE_ASYNC (Default: OFF)
This variable determines whether the HDF5 Asynchronous I/O API tests will be
built and tested. These tests will only run if a VOL connector reports that
it supports asynchronous I/O operations when queried via the H5Pget_vol_cap_flags
API routine.

HDF5_TEST_API_ENABLE_DRIVER (Default: OFF)
This variable determines whether the HDF5 API test driver program will be
built and used for testing. This driver program is useful when a VOL connector
uses a client/server model where the server program needs to be up and running
before the VOL connector can function. This option is currently not functional.

When the `HDF5_TEST_API` option is set to ON, HDF5's CMake code builds and tests
the new API tests using the native VOL connector. When one or more external VOL
connectors are built successfully with the process described in this document,
the CMake code will duplicate some of these API tests by adding separate
versions of the tests (for each VOL connector that was built) that set the
`HDF5_VOL_CONNECTOR` environment variable to the value specified for the
HDF5_VOL_<VOL name>_NAME variable for each external VOL connector at build time.
Running the `ctest` command will then run these new tests which load and run with
each VOL connector that was built in turn. When run via the `ctest` command, the
new tests typically follow the naming scheme:

HDF5_VOL_<VOL name lowercase>-h5_api_test_<test name>
HDF5_VOL_<VOL name lowercase>-h5_api_test_parallel_<test name>

**NOTE**
If dependencies of a built VOL connector are installed on the system in
a non-standard location that would typically require one to set `LD_LIBRARY_PATH`
or similar, one should ensure that those environment variables are set before
running tests. Otherwise, the tests that run with that connector will likely
fail due to being unable to load the necessary libraries for its dependencies.
Loading