-
-
Notifications
You must be signed in to change notification settings - Fork 60
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update to 1.14.5 #233
base: main
Are you sure you want to change the base?
Update to 1.14.5 #233
Conversation
Hi! This is the friendly automated conda-forge-linting service. I just wanted to let you know that I linted all conda-recipes in your PR ( |
The osx arm patch needs to be adapted. It looked like it was partially integrated so it is harder than I wanted it to be. |
Just wanted to make you all aware that we're stuck on the 1.14.4 migration in kealib: @gillins and I would like to add a test to hdf5 to check for this but neither of us has had time to write one so far. It would be nice to have this addressed before 1.14.5 gets built, though. |
This might be pretty hard to troubleshoot. I might in a first step do the following locally if I were you:
Those kinds of failures suck. Sorry you are hitting it.... |
@hmaarrfk, that does certainly sound tricky. I don't have access to a non-ARM Mac and I have no expertise in building hdf5 from source. I'm just pointing out the issue encountered by kealib because it essentially snarled up the hdf5 migration before it could get to most downstream packages (notably gdal) and I expect the same to happen with this version unless we (as a community) can track this issue down. My hope is to create a simple reproducer of the OSX issue (on CI here). I'll try to get to it sooner rather than later. |
@xylar @petebunting have added test, can you please check this makes sense? I'm hoping this goes away with the various memory corruption fixes in this release of hdf5, fingers crossed. |
recipe/testhdf5exc.cpp
Outdated
{ | ||
H5::Exception::dontPrint(); | ||
|
||
H5::H5File *keaImgH5File = new H5::H5File("/tmp/test.hd5", H5F_ACC_TRUNC, H5::FileCreatPropList::DEFAULT); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gillins, probably name this something like h5file because the reference to keaImg is confusing.
recipe/testhdf5exc.cpp
Outdated
} | ||
catch (const H5::Exception &e) | ||
{ | ||
std::cout << "Failed to open dataset (expected)\n"; |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
recipe/testhdf5exc.cpp
Outdated
std::cout << "Failed to open dataset (expected)\n"; | ||
} | ||
|
||
return 0; |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
@gillins, I have some suggested changes. We want your test to fail if the exception is not caught. But the gist of the test is exactly what I had in mind. I hope it does the trick and either demonstrated the problem or shows that it has been fixed in this version. It might be worth cherry-picking it into its own PR to see whether it has the expected effect on 1.14.4 as well. |
Thanks @xylar have incorporated these changes. AFAICT the problem we were seeing was that the exception was not caught at all and the process was terminated by the C++ runtime (presumably with a non-zero exit code). |
recipe/build.sh
Outdated
|
||
# test for hdf5 C++ exceptions | ||
$CXX $RECIPE_DIR/testhdf5exc.cpp -o testhdf5exc -L$PREFIX/lib -lhdf5_cpp -lhdf5 -I$PREFIX/include | ||
LD_LIBRARY_PATH=$PREFIX/lib DYLD_LIBRARY_PATH=$PREFIX/lib ./testhdf5exc |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LD_LIBRARY_PATH=$PREFIX/lib DYLD_LIBRARY_PATH=$PREFIX/lib ./testhdf5exc | |
if [[ "${CONDA_BUILD_CROSS_COMPILATION:-}" != "1" || "${CROSSCOMPILING_EMULATOR}" != "" ]]; then | |
./testhdf5exc | |
fi |
do you really need the LD_LIBRARY_PATH?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the tests CANNOT run on OSX (and later windows) for arm64 builds since we don't have emulators for those.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gillins and @hmaarrfk, as you may have seen, the test is failing to build under linux-64:
+ x86_64-conda-linux-gnu-c++ /home/conda/recipe_root/testhdf5exc.cpp -o testhdf5exc -L/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib -lhdf5_cpp -lhdf5 -I/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/include
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: warning: libcrypto.so.3, needed by /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so, not found (try using -rpath or -rpath-link)
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: warning: libcurl.so.4, needed by /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so, not found (try using -rpath or -rpath-link)
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: warning: libsz.so.2, needed by /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so, not found (try using -rpath or -rpath-link)
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: warning: libz.so.1, needed by /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so, not found (try using -rpath or -rpath-link)
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `curl_easy_perform'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `EVP_sha256@OPENSSL_3.0.0'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `curl_easy_init'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `curl_slist_append'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `compress2'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `compressBound@ZLIB_1.2.0'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `HMAC@OPENSSL_3.0.0'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `inflate'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `inflateInit_'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `SHA256@OPENSSL_3.0.0'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `curl_easy_cleanup'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `inflateEnd'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `curl_slist_free_all'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `SZ_encoder_enabled'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `curl_easy_setopt'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `curl_global_init'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `SZ_BufftoBuffDecompress'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `SZ_BufftoBuffCompress'
/home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_build_env/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/conda/feedstock_root/build_artifacts/hdf5_1729557442008/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/libhdf5.so: undefined reference to `curl_global_cleanup'
collect2: error: ld returned 1 exit status
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might make more sense to add the test to run_tests.sh
instead of build.sh
and to build it with cmake
similarly to:
https://github.com/conda-forge/hdf5-feedstock/blob/main/recipe/run_test.sh#L37-L40
https://github.com/conda-forge/hdf5-feedstock/blob/main/recipe/test/CMakeLists.txt
https://github.com/conda-forge/hdf5-feedstock/blob/main/recipe/test/h5_cmprss.c
That's what I originally had in mind. I think cmake
will figure out linking in the necessary libs and it's not worth trying to duplicate that effort some other way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On the other hand, maybe it makes sense to take the test back out since it didn't fail in #234
Ah, excellent point. Thanks! |
recipe/build.sh
Outdated
|
||
# test for hdf5 C++ exceptions | ||
if [[ "${CONDA_BUILD_CROSS_COMPILATION:-}" != "1" || "${CROSSCOMPILING_EMULATOR}" != "" ]]; then | ||
$CXX $RECIPE_DIR/testhdf5exc.cpp -o testhdf5exc -L$PREFIX/lib -lhdf5_cpp -lhdf5 -I$PREFIX/include |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You need to add the standard CXXFLAGS
Should I remove my test from this PR? Didn't seem to actually help us... |
5f9f955
to
afcd1bb
Compare
I would like to continue this effort to get it merged. Ultimately, HDF5 1.14.4 broke so much stuff for windows #240 that h5py is really hesitant to merge their migration PR conda-forge/h5py-feedstock#151 conda-forge/h5py-feedstock#152 I've created a migration branch for 1.14.3 and we can continue to use main to stay updated with the latest version of HDF5 to ensure we don't get too many changes all at once. If you are invested in keeping HDF5 updated for windows, please please comment HDFGroup/hdf5#5037 to raise the urgency of reverting the changes. |
Hi! This is the friendly automated conda-forge-linting service. I just wanted to let you know that I linted all conda-recipes in your PR ( I do have some suggestions for making it better though... For recipe/meta.yaml:
This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/12222809928. Examine the logs at this URL for more detail. |
@@ -23,6 +23,10 @@ h5c++ -showconfig | |||
h5c++ ${CXXFLAGS} ${LDFLAGS} h5tutr_cmprss.cpp -o h5tutr_cmprss | |||
./h5tutr_cmprss | |||
|
|||
echo "test for hdf5 C++ exceptions" | |||
h5c++ ${CXXFLAGS} ${LDFLAGS} testhdf5exc.cpp -o testhdf5exc |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this test can be removed, it didn't help prove anything...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok not doing any harm so i'll leave it for now, but I might just rebase things later.
Gotta get some time to look into the osx-arm failure.
Updates to 1.14.5
Also removes the GIF tools that have CVEs #187
Closes #187