Use `rdma-core` package (instead of CDT) & add to `linux_aarch64` #14

jakirkham · 2024-03-12T18:29:06Z

Part of issue: conda-forge/cuda-feedstock#28

Previously we used the CDT for rdma-core. However it would be better to use the package. Especially so as this CDT is not provided on AlmaLinux 8. This makes that change. It is also aligned with how we use rdma-core elsewhere.

Additionally it looks like rdma-core would be needed on linux_aarch64, but we didn't have it there before. So this makes sure to add it.

Checklist

Used a personal fork of the feedstock to propose changes
Bumped the build number (if the version is unchanged)
Reset the build number to 0 (if the version changed)
Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
Ensured the license file is being packaged.

Also include on `linux_aarch64` as it uses this dependency too.

…nda-forge-pinning 2024.03.12.14.44.34

conda-forge-webservices · 2024-03-12T18:29:13Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

jakirkham · 2024-03-12T18:31:07Z

Not seeing Azure here. Going to try restarting

@conda-forge-admin , please restart CI

jakirkham · 2024-03-12T18:31:58Z

Looks like that fixed it

jakirkham · 2024-03-12T19:04:11Z

Am seeing the following on CI

sed: can't read /home/conda/feedstock_root/build_artifacts/libcufile_1710268372716/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/lib/pkgconfig/cufile*.pc: No such file or directory

Based on looking locally, this appears to be happening because ${PREFIX}/lib/pkgconfig already exists. So the line below moves pkg-config into ${PREFIX}/lib/pkgconfig, which gives us ${PREFIX}/lib/pkgconfig/pkg-config. Where instead we would want all the .pc files in ${PREFIX}/lib/pkgconfig

libcufile-feedstock/recipe/build.sh

Line 10 in 7f8e1ec

[[ -d pkg-config ]] && mv pkg-config ${PREFIX}/lib/pkgconfig

IOW this looks like a bug in build.sh

jakirkham · 2024-04-10T05:10:12Z

@conda-forge-admin , please re-render

github-actions · 2024-04-10T05:13:16Z

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like I wasn't able to push to the drop_cdt branch of jakirkham-feedstocks/libcufile-feedstock. Did you check the "Allow edits from maintainers" box?

NOTE: Our webservices cannot push to PRs from organization accounts or PRs from forks made from organization forks because of GitHub permissions. Please fork the feedstock directly from conda-forge into your personal GitHub account.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/libcufile-feedstock/actions/runs/8626284824.

…nda-forge-pinning 2024.04.10.04.46.28

jakirkham · 2024-04-10T05:21:59Z

@conda-forge-admin , please re-render

jakirkham · 2024-04-10T05:26:59Z

@conda-forge-admin , please lint

conda-forge-webservices · 2024-04-10T05:27:07Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

jakirkham · 2024-04-10T05:27:30Z

@conda-forge-admin , please re-render

github-actions · 2024-04-10T05:29:14Z

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/libcufile-feedstock/actions/runs/8626433350.

adibbley · 2024-04-10T13:57:36Z

I'm not sure the rdma-core package is providing what we needed from the cdt package originally.

One example from the 12.4.1 build log I see:

INFO (libcufile,targets/x86_64-linux/lib/libcufile_rdma.so.1.9.1): Needed DSO x86_64-conda-linux-gnu/sysroot/usr/lib64/libmlx5.so.1 found in CDT/compiler package conda-forge/noarch::libibverbs-cos7-x86_64==22.4=h9b0a68f_1105

And here I see:

WARNING (libcufile,targets/x86_64-linux/lib/libcufile_rdma.so.1.9.1): $RPATH/libmlx5.so.1 not found in packages, sysroot(s) nor the missing_dso_whitelist.

jakirkham · 2024-06-25T00:46:19Z

@conda-forge-admin , please re-render

github-actions · 2024-06-25T00:47:38Z

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like I wasn't able to push to the drop_cdt branch of jakirkham-feedstocks/libcufile-feedstock. Did you check the "Allow edits from maintainers" box?

NOTE: Our webservices cannot push to PRs from organization accounts or PRs from forks made from organization forks because of GitHub permissions. Please fork the feedstock directly from conda-forge into your personal GitHub account.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/libcufile-feedstock/actions/runs/9654546805.

leofang · 2024-06-25T03:12:23Z

It seems the rerendering failed 😅

…nda-forge-pinning 2024.06.25.03.09.40

conda-forge-webservices · 2024-06-25T06:03:10Z

Hi! This is the friendly automated conda-forge-linting service.

I was trying to look for recipes to lint for you, but it appears we have a merge conflict.
Please try to merge or rebase with the base branch to resolve this conflict.

Please ping the 'conda-forge/core' team (using the @ notation in a comment) if you believe this is a bug.

conda-forge-webservices · 2024-06-25T06:06:29Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

jakirkham · 2024-06-25T06:21:05Z

Not sure why it wouldn't find the libraries, they are all there

rdma-core-52.0-hcccb83c_0/lib
├── libefa.so -> libefa.so.1.3.52.0
├── libefa.so.1 -> libefa.so.1.3.52.0
├── libefa.so.1.3.52.0
├── libhns.so -> libhns.so.1.0.52.0
├── libhns.so.1 -> libhns.so.1.0.52.0
├── libhns.so.1.0.52.0
├── libibmad.so -> libibmad.so.5.3.52.0
├── libibmad.so.5 -> libibmad.so.5.3.52.0
├── libibmad.so.5.3.52.0
├── libibnetdisc.so -> libibnetdisc.so.5.1.52.0
├── libibnetdisc.so.5 -> libibnetdisc.so.5.1.52.0
├── libibnetdisc.so.5.1.52.0
├── libibumad.so -> libibumad.so.3.2.52.0
├── libibumad.so.3 -> libibumad.so.3.2.52.0
├── libibumad.so.3.2.52.0
├── libibverbs.so -> libibverbs.so.1.14.52.0
├── libibverbs.so.1 -> libibverbs.so.1.14.52.0
├── libibverbs.so.1.14.52.0
├── libmana.so -> libmana.so.1.0.52.0
├── libmana.so.1 -> libmana.so.1.0.52.0
├── libmana.so.1.0.52.0
├── libmlx4.so -> libmlx4.so.1.0.52.0
├── libmlx4.so.1 -> libmlx4.so.1.0.52.0
├── libmlx4.so.1.0.52.0
├── libmlx5.so -> libmlx5.so.1.24.52.0
├── libmlx5.so.1 -> libmlx5.so.1.24.52.0
├── libmlx5.so.1.24.52.0
├── librdmacm.so -> librdmacm.so.1.3.52.0
├── librdmacm.so.1 -> librdmacm.so.1.3.52.0
└── librdmacm.so.1.3.52.0

leofang · 2024-06-25T15:59:53Z

I seem to recall we encountered a similar situation elsewhere (but can't find the example now), where a CTK component in $PREFIX/targets/<arch>/lib depends on non-CTK packages in $PREFIX/lib. Maybe we need to patch rpath manually for the cuFILE libraries?

conda-forge-webservices · 2024-07-30T20:37:54Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

jakirkham · 2024-07-31T07:27:02Z

Looks like we need libnuma based on this CI warning:

WARNING (gds-tools,gds/tools/gdsio): $RPATH/libnuma.so.1 not found in packages, sysroot(s) nor the missing_dso_whitelist.

However that is already here

libcufile-feedstock/recipe/meta.yaml

Line 161 in 63ee783

- libnuma # [linux]

Or an RPATH fix?

jakirkham · 2024-08-05T21:06:48Z

Yeah it looks like there is something up with the rpath for gds-tools based on this line from CI:

WARNING (gds-tools,gds/tools/gdsio): $RPATH/libcufile.so.0 not found in packages, sysroot(s) nor the missing_dso_whitelist.

As there is already libcufile present in the dependencies

libcufile-feedstock/recipe/meta.yaml

Line 164 in 6b64cd2

- {{ pin_subpackage("libcufile", max_pin="x") }}

jakirkham · 2024-08-06T22:27:03Z

Yep, this is related to issue: conda-forge/cuda-feedstock#10

jakirkham · 2024-10-09T19:51:37Z

@conda-forge-admin , please restart CI

billysuh7 · 2024-10-09T21:00:29Z

@conda-forge-admin, please restart ci

billysuh7

Sorry I forgot @h-vetinari to approve this. LGTM.

jakirkham · 2024-10-24T02:41:24Z

Thanks Billy! 🙏

Were you meaning PR: conda-forge/cuda-nvcc-impl-feedstock#25 ?

billysuh7 · 2024-10-24T03:09:25Z

Thanks Billy! 🙏

Were you meaning PR: conda-forge/cuda-nvcc-impl-feedstock#25 ?

That is all set also, I approved it and put more comment. Thanks.

jakirkham · 2024-10-24T04:10:32Z

Thanks Billy! 🙏

jakirkham · 2024-10-24T04:12:09Z

The conclusion Billy and I reached after looking into the warnings above more closely is there is likely a remaining RPATH issue in this package, which we will need to look into. This will be follow up work

xref: conda-forge/cuda-feedstock#10 (comment)

jakirkham added 4 commits March 12, 2024 11:23

Switch to rdma-core dependency

097011c

Also include on `linux_aarch64` as it uses this dependency too.

Test rdma-core support on all Linuxes

2de3cae

MNT: Re-rendered with conda-build 24.1.2, conda-smithy 3.31.1, and co…

c886d76

…nda-forge-pinning 2024.03.12.14.44.34

Bump build/number to 1

8decb26

jakirkham requested a review from a team as a code owner March 12, 2024 18:29

conda-forge-webservices bot closed this Mar 12, 2024

conda-forge-webservices bot reopened this Mar 12, 2024

Move pkgconfig content (not directory)

b619ada

jakirkham added 2 commits April 10, 2024 00:20

MNT: Re-rendered with conda-build 24.3.0, conda-smithy 3.34.1, and co…

0233bdd

…nda-forge-pinning 2024.04.10.04.46.28

Merge conda-forge/main into jakirkham-feedstocks/drop_cdt

5fc1045

MNT: Re-rendered with conda-build 24.5.1, conda-smithy 3.36.2, and co…

b29246c

…nda-forge-pinning 2024.06.25.03.09.40

Merge conda-forge/main into jakirkham-feedstocks/drop_cdt

32b691a

Merge conda-forge/main into jakirkham-feedstocks/drop_cdt

63ee783

jakirkham mentioned this pull request Oct 1, 2024

Deprecation of CentOS 7 support conda-forge/cuda-feedstock#28

Open

conda-forge-webservices bot closed this Oct 9, 2024

conda-forge-webservices bot reopened this Oct 9, 2024

conda-forge-webservices bot closed this Oct 9, 2024

conda-forge-webservices bot reopened this Oct 9, 2024

billysuh7 approved these changes Oct 24, 2024

View reviewed changes

jakirkham merged commit 55609a7 into conda-forge:main Oct 24, 2024
4 checks passed

jakirkham deleted the drop_cdt branch October 24, 2024 04:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use `rdma-core` package (instead of CDT) & add to `linux_aarch64` #14

Use `rdma-core` package (instead of CDT) & add to `linux_aarch64` #14

jakirkham commented Mar 12, 2024 •

edited

Loading

conda-forge-webservices bot commented Mar 12, 2024

jakirkham commented Mar 12, 2024

jakirkham commented Mar 12, 2024

jakirkham commented Mar 12, 2024

jakirkham commented Apr 10, 2024

github-actions bot commented Apr 10, 2024

jakirkham commented Apr 10, 2024

jakirkham commented Apr 10, 2024

conda-forge-webservices bot commented Apr 10, 2024

jakirkham commented Apr 10, 2024

github-actions bot commented Apr 10, 2024

adibbley commented Apr 10, 2024

jakirkham commented Jun 25, 2024

github-actions bot commented Jun 25, 2024

leofang commented Jun 25, 2024

conda-forge-webservices bot commented Jun 25, 2024

conda-forge-webservices bot commented Jun 25, 2024

jakirkham commented Jun 25, 2024

leofang commented Jun 25, 2024

conda-forge-webservices bot commented Jul 30, 2024 •

edited by conda-forge-admin

Loading

jakirkham commented Jul 31, 2024 •

edited

Loading

jakirkham commented Aug 5, 2024 •

edited

Loading

jakirkham commented Aug 6, 2024

jakirkham commented Oct 9, 2024

billysuh7 commented Oct 9, 2024

billysuh7 left a comment

jakirkham commented Oct 24, 2024

billysuh7 commented Oct 24, 2024

jakirkham commented Oct 24, 2024

jakirkham commented Oct 24, 2024 •

edited

Loading

Use rdma-core package (instead of CDT) & add to linux_aarch64 #14

Use rdma-core package (instead of CDT) & add to linux_aarch64 #14

Conversation

jakirkham commented Mar 12, 2024 • edited Loading

conda-forge-webservices bot commented Mar 12, 2024

jakirkham commented Mar 12, 2024

jakirkham commented Mar 12, 2024

jakirkham commented Mar 12, 2024

jakirkham commented Apr 10, 2024

github-actions bot commented Apr 10, 2024

jakirkham commented Apr 10, 2024

jakirkham commented Apr 10, 2024

conda-forge-webservices bot commented Apr 10, 2024

jakirkham commented Apr 10, 2024

github-actions bot commented Apr 10, 2024

adibbley commented Apr 10, 2024

jakirkham commented Jun 25, 2024

github-actions bot commented Jun 25, 2024

leofang commented Jun 25, 2024

conda-forge-webservices bot commented Jun 25, 2024

conda-forge-webservices bot commented Jun 25, 2024

jakirkham commented Jun 25, 2024

leofang commented Jun 25, 2024

conda-forge-webservices bot commented Jul 30, 2024 • edited by conda-forge-admin Loading

jakirkham commented Jul 31, 2024 • edited Loading

jakirkham commented Aug 5, 2024 • edited Loading

jakirkham commented Aug 6, 2024

jakirkham commented Oct 9, 2024

billysuh7 commented Oct 9, 2024

billysuh7 left a comment

Choose a reason for hiding this comment

jakirkham commented Oct 24, 2024

billysuh7 commented Oct 24, 2024

jakirkham commented Oct 24, 2024

jakirkham commented Oct 24, 2024 • edited Loading

Use `rdma-core` package (instead of CDT) & add to `linux_aarch64` #14

Use `rdma-core` package (instead of CDT) & add to `linux_aarch64` #14

jakirkham commented Mar 12, 2024 •

edited

Loading

conda-forge-webservices bot commented Jul 30, 2024 •

edited by conda-forge-admin

Loading

jakirkham commented Jul 31, 2024 •

edited

Loading

jakirkham commented Aug 5, 2024 •

edited

Loading

jakirkham commented Oct 24, 2024 •

edited

Loading