Add conda-forge recipie for `dgl` #22691

hmacdope · 2023-04-25T23:36:06Z

Checklist

conda-forge-webservices · 2023-04-25T23:36:21Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/dgl) and found it was in an excellent condition.

mikemhenry · 2023-04-26T00:16:50Z

-- DLPACK_INCLUDE_DIR not defined, searching for dlpack header
CMake Error at /home/conda/staged-recipes/build_artifacts/dgl_1682466386125/_build_env/share/cmake-3.26/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find DLPACK (missing: DLPACK_INCLUDE_DIR)
Call Stack (most recent call first):
  /home/conda/staged-recipes/build_artifacts/dgl_1682466386125/_build_env/share/cmake-3.26/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
  cmake/modules/FindDLPACK.cmake:39 (find_package_handle_standard_args)
  CMakeLists.txt:279 (find_package)

hmacdope · 2023-04-26T03:17:18Z

@mikemhenry this is the same issue I was seeing locally, I will continue to investigate. 😺

mikemhenry · 2023-04-26T16:04:44Z

Few hints:
https://github.com/conda-forge/dlpack-feedstock/blob/main/recipe/meta.yaml#L26
https://github.com/conda-forge/libtvm-feedstock/blob/main/recipe/build.sh#L48

hmacdope · 2023-04-26T20:01:50Z

Few hints:

https://github.com/conda-forge/dlpack-feedstock/blob/main/recipe/meta.yaml#L26

https://github.com/conda-forge/libtvm-feedstock/blob/main/recipe/build.sh#L48

Ah awesome should help a lot

bartekkuncer · 2023-05-24T12:19:57Z

Hi @hmacdope, @mikemhenry! First of all thanks for driving the initiative of adding dgl to conda-forge channel. I have two questions:

how are the works going - is there any major blocker atm?
I noticed that in build.sh you are setting USE_LIBXSMM flag to 'OFF' even though the library is available on conda-forge channel and the flag by default is set to 'ON'. Why is that?

mikemhenry · 2023-05-24T17:05:21Z

👋 Hey @bartekkuncer Check out this thread here: dmlc/dgl#1855 (you may have seen it) @hmacdope is in Australia so there is a bit of a time delay. We are still in the middle of working on this and will post an update soon. Right now I think that flag controls if to build with libxsmm, and doesn't control using the vendored lib or the lib from conda-forge. Before we merge in this feedstock, we will have it using every lib that is on codna-forge and we will add the missing ones as well.

hmacdope · 2023-05-30T05:24:05Z

@mikemhenry this will compile now at least (import failing with some linker error) which means the mechanism for inserting CF deps into the build is working. I think I will now try and make this work with the latest version + vendored thrust + up to date deps.

conda-forge-webservices · 2023-06-02T07:04:16Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/dgl) and found some lint.

Here's what I've got...

For recipes/dgl:

requirements: build: parallel-hashmap == 1.32 should not contain a space between relational operator and the version, i.e. parallel-hashmap ==1.32

conda-forge-webservices · 2023-06-02T07:06:26Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/dgl) and found it was in an excellent condition.

mikemhenry · 2023-08-04T19:10:32Z

@jakirkham no worries! I saw that it was missing from the docs you linked but I wasn't sure if it was an intentional omission.

mikemhenry · 2023-08-04T19:11:29Z

I am now trying locally a build where I don't have anything added to the test env to see if adding liblapacke is enough

mikemhenry · 2023-08-04T20:24:15Z

Without - liblapack =*=*mkl in test we still get libdgl.so: undefined symbol: dgemm_

So I know that having - liblapack =*=*mkl in test gets CI to pass, but I am worried about users running into this issue.

I uploaded the built package and then installed it, and get libdgl.so: undefined symbol: dgemm_ locally.

Then if I install liblapack =*=*mkl

   - liblapack=*[build=*mkl]


  Package            Version  Build                        Channel           Size
───────────────────────────────────────────────────────────────────────────────────
  Install:
───────────────────────────────────────────────────────────────────────────────────

  + libhwloc           2.9.2  nocuda_h7313eea_1008         conda-forge        3MB
  + libiconv            1.17  h166bdaf_0                   conda-forge     Cached
  + libxml2           2.11.4  h0d562d8_0                   conda-forge     Cached
  + llvm-openmp       16.0.6  h4dfa4b3_0                   conda-forge     Cached
  + mkl             2022.2.1  h84fe81f_16997               conda-forge     Cached
  + rocm-smi           5.6.0  h59595ed_1                   conda-forge        4MB
  + tbb            2021.10.0  h00ab1b0_0                   conda-forge      186kB

  Remove:
───────────────────────────────────────────────────────────────────────────────────

  - nomkl                1.0  h5ca1d4c_0                   conda-forge     Cached

  Change:
───────────────────────────────────────────────────────────────────────────────────

  - _openmp_mutex        4.5  2_gnu                        conda-forge     Cached
  + _openmp_mutex        4.5  2_kmp_llvm                   conda-forge     Cached
  - libblas            3.9.0  17_linux64_openblas          conda-forge     Cached
  + libblas            3.9.0  16_linux64_mkl               conda-forge     Cached
  - libcblas           3.9.0  17_linux64_openblas          conda-forge     Cached
  + libcblas           3.9.0  16_linux64_mkl               conda-forge     Cached
  - liblapack          3.9.0  17_linux64_openblas          conda-forge     Cached
  + liblapack          3.9.0  16_linux64_mkl               conda-forge     Cached
  - liblapacke         3.9.0  17_linux64_openblas          conda-forge     Cached
  + liblapacke         3.9.0  16_linux64_mkl               conda-forge     Cached
  - pytorch            2.0.0  cpu_generic_py310h7ffd2bf_1  conda-forge     Cached
  + pytorch            2.0.0  cpu_mkl_py310h402c8e3_101    conda-forge       71MB

The error goes away.

mikemhenry · 2023-08-04T20:25:39Z

So I don't know why, but it looks like we are dependent on the mkl build of blas, despite not having that in our host env explicitly.

jakirkham · 2023-08-04T20:26:38Z

Am confused by the _. The BLAS symbol name is dgemm (no _). Do we know where it is coming up with that?

mikemhenry · 2023-08-04T20:27:13Z

If you want to play around with this, this command will install dgl (the build that came from CI)
micromamba create -n dlg-cpi -c conda-forge -c "mmh/label/cpu-test" dgl

and python -c "import dgl" should give you an error unless you first do micromamba install "liblapack =*=*mkl"

mikemhenry · 2023-08-04T20:30:49Z

It looks like they all come from third_party/libxsmm and there is something weird https://github.com/libxsmm/libxsmm/blob/ef75fe7c814c1a8a51f8c142db71059caab65e02/samples/utilities/wrap/dgemm.c#L19

hmacdope · 2023-08-06T09:01:21Z

Can we just use MKL for now and then unwind the issues in a feedstock @mikemhenry @jakirkham?

mikemhenry · 2023-08-07T14:44:09Z

I am going to try moving liblapack =*=*mkl to run: and see if that works (locally) and if it does, push it to CI and we can figure out a better way after we get this merged

mikemhenry · 2023-08-07T16:24:22Z

Okay so once CI passes, I will remove the stuff I did for the swapfile, that might break the build but then it should be good to go...

mikemhenry · 2023-08-07T20:31:00Z

@conda-forge-admin, please ping team

conda-forge-webservices · 2023-08-07T20:31:03Z

Hi! This is the friendly automated conda-forge-webservice.

I was asked to ping @conda-forge/staged-recipes and so here I am doing that.

mikemhenry · 2023-08-07T20:32:56Z

Okay so this might fail without the swap file stuff, but this is ready to merge 😄 @carterbox

recipes/dgl/meta.yaml

jakirkham · 2023-08-07T20:37:35Z

recipes/dgl/meta.yaml

+    - metis ==5.1.1
+    - nanoflann
+    - parallel-hashmap
+    - python =3.10


Was this just due to CI limits?

Good catch! Yes! I will fix that now

Co-authored-by: jakirkham <[email protected]>

mikemhenry · 2023-08-08T17:58:14Z

@jakirkham this is ready to merge, we won't get a green check without the swap fix

jakirkham · 2023-08-08T21:48:12Z

Thanks Mike! 🙏

LGTM

@carterbox did you have any more thoughts here?

Also Mike it might be worth sending a separate PR to add the swapfile changes to staged-recipes if you are up for it 🙂

mikemhenry · 2023-08-09T15:00:23Z

Also Mike it might be worth sending a separate PR to add the swapfile changes to staged-recipes if you are up for it

Will do!

jakirkham · 2023-08-09T19:30:05Z

Thanks all! 🙏

commit nessecary files

7d89fcf

mikemhenry mentioned this pull request Apr 25, 2023

Add dgl #18620

Closed

11 tasks

hmacdope mentioned this pull request Apr 26, 2023

PyPi and conda release choderalab/mtenn#20

Closed

kaminow mentioned this pull request Apr 26, 2023

Make the project conda or pip installable asapdiscovery/asapdiscovery#20

Closed

mikemhenry added 3 commits April 26, 2023 09:06

see if this helps find dlpack

dd16c51

use correct cmake var

23aec9c

maybe it needs to be the build prefix

064b809

hmacdope added 6 commits May 29, 2023 14:29

Merge branch 'conda-forge:main' into dgl_v2

0116aa6

Merge remote-tracking branch 'upstream/main' into dgl_v2

525cfdc

remove junk

59e9f6b

improve build and patch

3fb01dd

Merge branch 'dgl_v2' of github.com:hmacdope/staged-recipes into dgl_v2

2ba8d37

try fixing patch

28052c1

hmacdope mentioned this pull request May 29, 2023

Publish DGL Package in conda-forge channel dmlc/dgl#5718

Closed

hmacdope added 3 commits May 29, 2023 15:54

improve build and make everything a lot nicer

3b54b01

allow building torch backend

c85470e

dont actually build torch backend

4879a25

move to up to date dgl

cd2e217

change back to plane phmap

79d99d7

fix build script

359777f

Add liblapack =*=*mkl as run time dep and sort things

c54ebe8

mikemhenry added 3 commits August 7, 2023 13:29

undo changes to make swap file

e4255d7

undo changes to make swap file (missed one)

1373840

Merge branch 'main' into dgl_v2

b8cf878

github-actions bot added review-requested and removed Awaiting author contribution labels Aug 7, 2023

jakirkham reviewed Aug 7, 2023

View reviewed changes

recipes/dgl/meta.yaml Show resolved Hide resolved

jakirkham reviewed Aug 7, 2023

View reviewed changes

mikemhenry and others added 2 commits August 7, 2023 13:40

Update recipes/dgl/meta.yaml

62d34a5

Co-authored-by: jakirkham <[email protected]>

build all the python

71c3daf

Merge branch 'main' into dgl_v2

1c9fa48

carterbox merged commit 68adcb3 into conda-forge:main Aug 9, 2023

ArnNag mentioned this pull request Jul 27, 2024

About Conda Environment baker-laboratory/rf_diffusion_all_atom#2

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add conda-forge recipie for `dgl` #22691

Add conda-forge recipie for `dgl` #22691

hmacdope commented Apr 25, 2023 •

edited by jakirkham

Loading

conda-forge-webservices bot commented Apr 25, 2023

mikemhenry commented Apr 26, 2023

hmacdope commented Apr 26, 2023

mikemhenry commented Apr 26, 2023

hmacdope commented Apr 26, 2023

bartekkuncer commented May 24, 2023 •

edited

Loading

mikemhenry commented May 24, 2023

hmacdope commented May 30, 2023

conda-forge-webservices bot commented Jun 2, 2023

conda-forge-webservices bot commented Jun 2, 2023

mikemhenry commented Aug 4, 2023

mikemhenry commented Aug 4, 2023

mikemhenry commented Aug 4, 2023

mikemhenry commented Aug 4, 2023

jakirkham commented Aug 4, 2023

mikemhenry commented Aug 4, 2023

mikemhenry commented Aug 4, 2023

hmacdope commented Aug 6, 2023

mikemhenry commented Aug 7, 2023

mikemhenry commented Aug 7, 2023

mikemhenry commented Aug 7, 2023

conda-forge-webservices bot commented Aug 7, 2023

mikemhenry commented Aug 7, 2023

jakirkham Aug 7, 2023

mikemhenry Aug 7, 2023

mikemhenry commented Aug 8, 2023

jakirkham commented Aug 8, 2023

mikemhenry commented Aug 9, 2023

jakirkham commented Aug 9, 2023

Add conda-forge recipie for dgl #22691

Add conda-forge recipie for dgl #22691

Conversation

hmacdope commented Apr 25, 2023 • edited by jakirkham Loading

conda-forge-webservices bot commented Apr 25, 2023

mikemhenry commented Apr 26, 2023

hmacdope commented Apr 26, 2023

mikemhenry commented Apr 26, 2023

hmacdope commented Apr 26, 2023

bartekkuncer commented May 24, 2023 • edited Loading

mikemhenry commented May 24, 2023

hmacdope commented May 30, 2023

conda-forge-webservices bot commented Jun 2, 2023

conda-forge-webservices bot commented Jun 2, 2023

mikemhenry commented Aug 4, 2023

mikemhenry commented Aug 4, 2023

mikemhenry commented Aug 4, 2023

mikemhenry commented Aug 4, 2023

jakirkham commented Aug 4, 2023

mikemhenry commented Aug 4, 2023

mikemhenry commented Aug 4, 2023

hmacdope commented Aug 6, 2023

mikemhenry commented Aug 7, 2023

mikemhenry commented Aug 7, 2023

mikemhenry commented Aug 7, 2023

conda-forge-webservices bot commented Aug 7, 2023

mikemhenry commented Aug 7, 2023

jakirkham Aug 7, 2023

Choose a reason for hiding this comment

mikemhenry Aug 7, 2023

Choose a reason for hiding this comment

mikemhenry commented Aug 8, 2023

jakirkham commented Aug 8, 2023

mikemhenry commented Aug 9, 2023

jakirkham commented Aug 9, 2023

Add conda-forge recipie for `dgl` #22691

Add conda-forge recipie for `dgl` #22691

hmacdope commented Apr 25, 2023 •

edited by jakirkham

Loading

bartekkuncer commented May 24, 2023 •

edited

Loading