Rebuild #142

dalcinl · 2024-02-05T05:36:29Z

Checklist

Used a personal fork of the feedstock to propose changes
Bumped the build number (if the version is unchanged)
Reset the build number to 0 (if the version changed)
Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
Ensured the license file is being packaged.

conda-forge-webservices · 2024-02-05T05:36:38Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

leofang · 2024-02-05T15:05:43Z

@conda-forge-admin, please rerender

github-actions · 2024-02-05T15:07:52Z

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/openmpi-feedstock/actions/runs/7786526756.

leofang · 2024-02-05T15:19:34Z

@conda-forge/core We need some help here. We keep hitting the unicode error after merging #141 (the CI was green there, but the error started happening at main). Now we can reproduce the error even in the CI, so I can only assume this is due to some change in build tool that unfortunately intervened. I asked in the Gitter channel last week but didn't get any response so far...

2024-02-05T10:57:22.0104022Z Warning: rpath /home/conda/feedstock_root/build_artifacts/openmpi-mpi_1707129643574/_build_env/lib is outside prefix /home/conda/feedstock_root/build_artifacts/openmpi-mpi_1707129643574/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold (removing it)
2024-02-05T10:57:22.0576062Z Warning: rpath /home/conda/feedstock_root/build_artifacts/openmpi-mpi_1707129643574/_build_env/lib is outside prefix /home/conda/feedstock_root/build_artifacts/openmpi-mpi_1707129643574/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold (removing it)
2024-02-05T10:57:22.1059741Z Traceback (most recent call last):
2024-02-05T10:57:22.1060663Z   File "/opt/conda/lib/python3.10/site-packages/conda_build/os_utils/liefldd.py", line 54, in ensure_binary
2024-02-05T10:57:22.1067614Z     return lief.parse(str(file))
2024-02-05T10:57:22.1068672Z TypeError: '/home/conda/feedstock_root/build_artifacts/openmpi-mpi_1707129643574/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/\x01\udce4\x05'
2024-02-05T10:57:22.1069304Z 
2024-02-05T10:57:22.1069548Z During handling of the above exception, another exception occurred:
2024-02-05T10:57:22.1069815Z 
2024-02-05T10:57:22.1070014Z Traceback (most recent call last):
2024-02-05T10:57:22.1070354Z   File "/opt/conda/bin/conda-mambabuild", line 10, in <module>
2024-02-05T10:57:22.1070660Z     sys.exit(main())
2024-02-05T10:57:22.1071014Z   File "/opt/conda/lib/python3.10/site-packages/boa/cli/mambabuild.py", line 256, in main
2024-02-05T10:57:22.1077424Z     call_conda_build(action, config)
2024-02-05T10:57:22.1078241Z   File "/opt/conda/lib/python3.10/site-packages/boa/cli/mambabuild.py", line 228, in call_conda_build
2024-02-05T10:57:22.1078616Z     result = api.build(
2024-02-05T10:57:22.1079028Z   File "/opt/conda/lib/python3.10/site-packages/conda_build/api.py", line 254, in build
2024-02-05T10:57:22.1084360Z     return build_tree(
2024-02-05T10:57:22.1084989Z   File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 3789, in build_tree
2024-02-05T10:57:22.1097473Z     packages_from_this = build(
2024-02-05T10:57:22.1098008Z   File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 2877, in build
2024-02-05T10:57:22.1104652Z     newly_built_packages = bundlers[pkg_type](output_d, m, env, stats)
2024-02-05T10:57:22.1105254Z   File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 2004, in bundle_conda
2024-02-05T10:57:22.1109746Z     files = post_process_files(metadata, initial_files)
2024-02-05T10:57:22.1110373Z   File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 1815, in post_process_files
2024-02-05T10:57:22.1114364Z     post_build(m, new_files, build_python=python)
2024-02-05T10:57:22.1114887Z   File "/opt/conda/lib/python3.10/site-packages/conda_build/post.py", line 1818, in post_build
2024-02-05T10:57:22.1125728Z     post_process_shared_lib(m, f, prefix_files, host_prefix)
2024-02-05T10:57:22.1126377Z   File "/opt/conda/lib/python3.10/site-packages/conda_build/post.py", line 1680, in post_process_shared_lib
2024-02-05T10:57:22.1130273Z     mk_relative_linux(
2024-02-05T10:57:22.1135257Z   File "/opt/conda/lib/python3.10/site-packages/conda_build/post.py", line 612, in mk_relative_linux
2024-02-05T10:57:22.1135716Z     existing2, _, _ = get_rpaths_raw(elf)
2024-02-05T10:57:22.1136164Z   File "/opt/conda/lib/python3.10/site-packages/conda_build/os_utils/liefldd.py", line 206, in get_rpathy_thing_raw_partial
2024-02-05T10:57:22.1136517Z     binary = ensure_binary(file)
2024-02-05T10:57:22.1136924Z   File "/opt/conda/lib/python3.10/site-packages/conda_build/os_utils/liefldd.py", line 56, in ensure_binary
2024-02-05T10:57:22.1137298Z     print(f"WARNING: liefldd: failed to ensure_binary({file})")
2024-02-05T10:57:22.1137708Z UnicodeEncodeError: 'utf-8' codec can't encode character '\udce4' in position 303: surrogates not allowed
2024-02-05T10:57:29.2571498Z 
2024-02-05T10:57:29.2638583Z ##[error]Bash exited with code '1'.
2024-02-05T10:57:29.2801376Z ##[section]Finishing: Run docker build

leofang · 2024-02-05T15:45:03Z

@dalcinl do you think any of the changes that we made could lead to this file being generated?

/home/conda/feedstock_root/build_artifacts/openmpi-mpi_1707129643574/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/\x01\udce4\x05

which seems an odd one to me

dalcinl · 2024-02-05T16:17:06Z

@dalcinl do you think any of the changes that we made could lead to this file being generated?

I don't think so, but I'll double check our PR diff.

hmaarrfk · 2024-02-05T19:31:53Z

it might be a difference in the lief version... maybe compare and try to pin?

leofang · 2024-02-05T21:10:25Z

liblief/py-lief are both staying on 0.12.3, nothing has changed 🤔

hmaarrfk · 2024-02-06T11:27:58Z

sometimes i download both logs, delete everything until Z and just run vimdiff to visually inspect the differences

jakirkham · 2024-02-06T20:25:38Z

Looks like some builds are running out of space. Should we try adding to conda-forge.yml (and re-render)?

azure:
  free_disk_space: true

leofang · 2024-02-06T20:39:18Z

@conda-forge-admin, please rerender

…nda-forge-pinning 2024.02.06.16.43.21

leofang · 2024-02-06T21:57:07Z

No luck, out of disk space doesn't seem to be the issue of having a filename with weird unicode

jakirkham · 2024-02-06T23:10:22Z

Previously saw this error in this CI log

conda.CondaMultiError: Error with archive /home/conda/feedstock_root/build_artifacts/pkg_cache/libstdcxx-devel_linux-64-11.4.0-h922705a_105.conda.  You probably need to delete and re-download or re-create this file.  Message was:

failed with error: [Errno 28] No space left on device
Error with archive /home/conda/feedstock_root/build_artifacts/pkg_cache/gfortran_impl_linux-aarch64-11.4.0-hfbda5c0_5.conda.  You probably need to delete and re-download or re-create this file.  Message was:

failed with error: [Errno 28] No space left on device
Error with archive /home/conda/feedstock_root/build_artifacts/pkg_cache/gxx_impl_linux-aarch64-11.4.0-he533754_5.conda.  You probably need to delete and re-download or re-create this file.  Message was:

failed with error: [Errno 28] No space left on device
Error with archive /home/conda/feedstock_root/build_artifacts/pkg_cache/libgfortran5-13.2.0-ha4646dd_5.conda.  You probably need to delete and re-download or re-create this file.  Message was:

failed with error: [Errno 28] No space left on device
Error with archive /home/conda/feedstock_root/build_artifacts/pkg_cache/binutils_impl_linux-64-2.40-hf600244_0.conda.  You probably need to delete and re-download or re-create this file.  Message was:

failed with error: [Errno 28] No space left on device
Error with archive /home/conda/feedstock_root/build_artifacts/pkg_cache/libsanitizer-11.4.0-h4dcbe23_5.conda.  You probably need to delete and re-download or re-create this file.  Message was:

failed with error: [Errno 28] No space left on device
Error with archive /home/conda/feedstock_root/build_artifacts/pkg_cache/gcc_impl_linux-aarch64-11.4.0-he533754_5.conda.  You probably need to delete and re-download or re-create this file.  Message was:

failed with error: [Errno 28] No space left on device

leofang · 2024-02-06T23:22:01Z

Yeah we were not concerned with that, we've been focusing on fixing this issue: #142 (comment), starting here: #141 (comment).

conda-forge.yml

Co-authored-by: jakirkham <[email protected]>

leofang · 2024-02-07T04:30:33Z

@conda-forge-admin, please rerender

…nda-forge-pinning 2024.02.06.16.43.21

h-vetinari · 2024-02-27T03:41:05Z

Wouldn't we also need to rebuild rdma-core first to produce packages with the fix for openmpi to use?

AFAIU, as long as that malformed (non-UTF-8) path is there - which is still the case - the rdma rebuild shouldn't make a difference. Unless we completely get rid of the symlinks in that package, but that's not a reasonable ask; we should IMO fix (or work around) the bug in patchelf/conda-build.

leofang · 2024-03-07T04:31:38Z

@mbargull any change you have further insight on this issue? 🙂

mbargull · 2024-03-10T18:57:48Z

@mbargull any change you have further insight on this issue? 🙂

Sorry, didn't follow this issue; looks like rdma-core has not been rebuilt yet.
The libraries therein are broken for the versions built with conda-build >=3.28,<24.1.2, see my previous comment #142 (comment) :

The rdma-core=50 and rdma-core=49.1 builds are faulty in that ldconfig -vn "${PREFIX}/lib" creates symlinks for libmana/libmlx5 with those broken filenames.

leofang · 2024-03-10T20:44:04Z

So was my understanding in #142 (comment) incorrect, that there are still issues to fix, regardless whether we fixed conda-build (1)?

mbargull · 2024-03-10T20:57:30Z

So was my understanding in #142 (comment) incorrect, that there are still issues to fix, regardless whether we fixed conda-build (1)?

Yes, it is partially incorrect. patchelf does not call ldconfig.
There's no "gap in this analysis"; it's simply that the combination of bugs in conda-build and patchelf lead to faulty artifacts (in rdma-core) which when processed downstream (this feedstock) with ldconfig causes the creation of symlinks with broken names.

xhochy · 2024-03-11T08:43:12Z

I have put in a PR to rebuild rdma_core: conda-forge/rdma-core-feedstock#17

mbargull · 2024-03-11T13:07:16Z

@conda-forge-admin, please rerender

…nda-forge-pinning 2024.03.10.12.43.11

recipe/meta.yaml

leofang · 2024-03-11T13:54:53Z

@conda-forge-admin, please rerender

recipe/meta.yaml

github-actions · 2024-03-11T13:57:48Z

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/openmpi-feedstock/actions/runs/8234009795.

leofang · 2024-03-11T14:04:32Z

Yes, it is partially incorrect. patchelf does not call ldconfig. (...) which when processed downstream (this feedstock) with ldconfig causes the creation of symlinks with broken names.

@mbargull Forgive my ignorance, so when exactly is ldconfig called and by whom?

mbargull · 2024-03-11T14:22:39Z

@mbargull Forgive my ignorance, so when exactly is ldconfig called and by whom?

https://www.gnu.org/software/libtool/manual/libtool.html#Finish-mode
(You should find the exact invocations in a configure scripts, but build's source archive might also carry a libtool.m4 which would be slightly more readable.)

leofang · 2024-03-11T14:27:27Z

linux-64 CI is finally happy!

This lines up the GLIBC version with the Docker image used in the CentOS 7 case. It also ensures GLIBC is properly constrained in the packages produced.

leofang · 2024-03-11T19:45:49Z

@conda-forge-admin, please rerender

github-actions · 2024-03-11T19:48:51Z

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/openmpi-feedstock/actions/runs/8238815790.

github-actions · 2024-03-11T20:39:22Z

Hi! This is the friendly conda-forge automerge bot!

I considered the following status checks when analyzing this PR:

linter: passed
azure: passed

Thus the PR was passing and merged! Have a great day!

leofang · 2024-03-11T20:54:48Z

Thanks @mbargull @minrk @jakirkham and everyone for helping out! This is one of the most tricky bugs I've ever seen on conda-forge!

rebuild

d735e2c

dalcinl requested review from astrofrog, beckermr, bekozi, leofang, minrk, msarahan and ocefpaf as code owners February 5, 2024 05:36

leofang mentioned this pull request Feb 5, 2024

Rebuild for libhwloc 2.9.3 #141

Merged

Revert change to .gitignore

65e2d8c

Update conda-forge.yml

6462cca

MNT: Re-rendered with conda-build 3.28.4, conda-smithy 3.30.4, and co…

a52c953

…nda-forge-pinning 2024.02.06.16.43.21

jakirkham reviewed Feb 7, 2024

View reviewed changes

conda-forge.yml Show resolved Hide resolved

bump swap size

67bc1ac

Co-authored-by: jakirkham <[email protected]>

MNT: Re-rendered with conda-build 3.28.4, conda-smithy 3.30.4, and co…

674c16f

…nda-forge-pinning 2024.02.06.16.43.21

jakirkham mentioned this pull request Mar 7, 2024

MNT: rerender conda-forge/ucx-split-feedstock#158

Closed

1 task

xhochy mentioned this pull request Mar 11, 2024

Rerender the feedstock to fix downstream issues conda-forge/rdma-core-feedstock#17

Merged

MNT: Re-rendered with conda-build 24.1.2, conda-smithy 3.31.1, and co…

f3ef86f

…nda-forge-pinning 2024.03.10.12.43.11

leofang reviewed Mar 11, 2024

View reviewed changes

recipe/meta.yaml Outdated Show resolved Hide resolved

Update recipe/meta.yaml

65a4e58

leofang reviewed Mar 11, 2024

View reviewed changes

recipe/meta.yaml Outdated Show resolved Hide resolved

jakirkham and others added 2 commits March 11, 2024 12:44

Use sysroot for 2.17

cbf6604

This lines up the GLIBC version with the Docker image used in the CentOS 7 case. It also ensures GLIBC is properly constrained in the packages produced.

remove unsuccessful attempt

c1a080f

leofang marked this pull request as ready for review March 11, 2024 19:47

leofang approved these changes Mar 11, 2024

View reviewed changes

leofang added the automerge Merge the PR when CI passes label Mar 11, 2024

github-actions bot merged commit 8910db8 into conda-forge:main Mar 11, 2024
10 checks passed

jakirkham mentioned this pull request Mar 11, 2024

mpifort 5.0.1 fails with undefined refs to memcpy@GLIBC_2.14 and clock_gettime@GLIBC_2.17 #143

Closed

1 task

jakirkham mentioned this pull request Mar 12, 2024

Mark corrupted rdma-core packages broken conda-forge/admin-requests#956

Merged

11 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rebuild #142

Rebuild #142

dalcinl commented Feb 5, 2024 •

edited

Loading

conda-forge-webservices bot commented Feb 5, 2024

leofang commented Feb 5, 2024

github-actions bot commented Feb 5, 2024

leofang commented Feb 5, 2024

leofang commented Feb 5, 2024

dalcinl commented Feb 5, 2024

hmaarrfk commented Feb 5, 2024

leofang commented Feb 5, 2024

hmaarrfk commented Feb 6, 2024

jakirkham commented Feb 6, 2024

leofang commented Feb 6, 2024

leofang commented Feb 6, 2024

jakirkham commented Feb 6, 2024

leofang commented Feb 6, 2024

leofang commented Feb 7, 2024

h-vetinari commented Feb 27, 2024

leofang commented Mar 7, 2024

mbargull commented Mar 10, 2024

leofang commented Mar 10, 2024

mbargull commented Mar 10, 2024

xhochy commented Mar 11, 2024

mbargull commented Mar 11, 2024

leofang commented Mar 11, 2024

github-actions bot commented Mar 11, 2024

leofang commented Mar 11, 2024 •

edited

Loading

mbargull commented Mar 11, 2024

leofang commented Mar 11, 2024

leofang commented Mar 11, 2024

github-actions bot commented Mar 11, 2024

github-actions bot commented Mar 11, 2024

leofang commented Mar 11, 2024

Rebuild #142

Rebuild #142

Conversation

dalcinl commented Feb 5, 2024 • edited Loading

conda-forge-webservices bot commented Feb 5, 2024

leofang commented Feb 5, 2024

github-actions bot commented Feb 5, 2024

leofang commented Feb 5, 2024

leofang commented Feb 5, 2024

dalcinl commented Feb 5, 2024

hmaarrfk commented Feb 5, 2024

leofang commented Feb 5, 2024

hmaarrfk commented Feb 6, 2024

jakirkham commented Feb 6, 2024

leofang commented Feb 6, 2024

leofang commented Feb 6, 2024

jakirkham commented Feb 6, 2024

leofang commented Feb 6, 2024

leofang commented Feb 7, 2024

h-vetinari commented Feb 27, 2024

leofang commented Mar 7, 2024

mbargull commented Mar 10, 2024

leofang commented Mar 10, 2024

mbargull commented Mar 10, 2024

xhochy commented Mar 11, 2024

mbargull commented Mar 11, 2024

leofang commented Mar 11, 2024

github-actions bot commented Mar 11, 2024

leofang commented Mar 11, 2024 • edited Loading

mbargull commented Mar 11, 2024

leofang commented Mar 11, 2024

leofang commented Mar 11, 2024

github-actions bot commented Mar 11, 2024

github-actions bot commented Mar 11, 2024

leofang commented Mar 11, 2024

dalcinl commented Feb 5, 2024 •

edited

Loading

leofang commented Mar 11, 2024 •

edited

Loading