How to install GPBoost on an M1 chip

[Androniki]

Installation to M1 chip Macbook is tricky, do you have some steps to help install it in M1?

[fabsig]

Unfortunately not. I don't have a Mac with an M1 chip, so I cannot investigate this. You might want to have a look at LightGBM and check whether they have some guidance or solution as we share a lot of the infrastructure.

[estsauver]

@fabsig, if you're interested in maintaining this on an m1 chip, we can donate an m1 instance on scaleway to the project. I also realize maintaining on another architecture might be a significant burden for the team, but just thought I'd mention it as an option. (https://www.scaleway.com/en/pricing/#apple-silicon)

[fabsig]

@estsauver : Thanks a lot for the offer, much appreciated! Unfortunately, I currently do not have the time to work on this. If anybody could support me with this, it would be highly appreciated. If myself or somebody else has time to work on this in the future, I might get back to you.

[ahalev]

@Androniki were you able to successfully install GPBoost?

[Androniki]

@ahalev not yet. We will let you know when we are successful

[fabsig]

I have now tried installing the gpboost Python package on a M1 Mac mini on scaleway and it works without any problems for me both using clang or gcc after following these instructions. Note that compilation from source needs to be done (but this is done automatically) since there is currently no macOS arm64 version on PyPI.

@Androniki @ahalev : what is the error message you get when installing the package?

@estsauver : In case your offer is still valid, I would be glad about it. I could then also provide pre-compiled macOS arm64 wheel versions on PyPI so that users do not need to do the compilation.

[estsauver]

It’s definitely still valid. Will you reach out to me at ***@***.*** and I’ll figure out how to get you into our account? ~Earl

…

On Wed, May 25 2022 at 4:06 PM, Fabio Sigrist ***@***.***> wrote: I have now tried installing the gpboost Python package on a M1 Mac mini on scaleway and it works without any problems for me both using clang or gcc after following these instructions <https://github.com/fabsig/GPBoost/blob/master/docs/Installation_guide.rst#apple-clang>. Note that compilation from source needs to be done (but this is done automatically) since there is currently no macOS arm64 version on PyPI. @Androniki <https://github.com/Androniki> @ahalev <https://github.com/ahalev> : what is the error message you get when installing the package? @estsauver <https://github.com/estsauver> : In case your offer is still valid, I would be glad about it. I could then also provide pre-compiled macOS arm64 wheel versions on PyPI so that users do not need to do the compilation. — Reply to this email directly, view it on GitHub <#51 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAF3ICNVV66B2WH72IRVYJLVLYXXNANCNFSM5QX5BE5Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[fabsig]

@estsauver : Thanks a lot! I don't have your email address. You can contact me on my public university email address.

[PlanktonicParadox]

Is it possible to install gpboost for R on Mac M1 chip? I am considering buying a new laptop and will most like be a Mac with M1 chip.

[fabsig]

Yes, this should be no problem as a macOS arm64 chip version is on CRAN.

If you want to compile the library yourself, you can also do it. I had no problems doing this with Python, and I don't see a reason why it should not work on R; see the R package instructions and also the CLI instructions for the requirements when doing the compilation yourself.

[PlanktonicParadox]

Much appreciated Fabio. Thanks.

[joselznom]

Hi @fabsig
I tried to install the python package following the instructions that you made. When the build is done, my conda enviroment can't find the package, do you know why this happens?
I compile the package from my home directory but it don't appear in my site-packages folder from the enviroment, I also tried to install directly on the site-packages with the same result.

[fabsig]

One possible explanation might be that the installation from source moved the package to another location than your conda environment. You might check in the installation messages where the package was installed to. Unfortunately, I cannot reproduce this. As I wrote above, when I tried installing it on a M1 Mac mini on scaleway, it all worked for me without any problems.

[fabsig]

A macOS arm64 chip pre-compiled version is now on PyPI. You should now be able to install with pip on M1 / M2 Macs without any problems as compilation is not required anymore. Can somebody please double check whether it works?

[fabsig]

There is a caveat: I have so far not managed to enable OpenMP support for the macOS arm64 version. I.e., things are not parallelized and might be considerably slower because of that.

[fabsig]

The issue with OpenMP support is now also solved. Starting with version 0.7.8.4, the pre-compiled macOS arm64 on PyPI (installation with pip install -U gpboost) should work as expected.

[joselznom]

A new error comes with the pip install -U gpboost:

Reason: tried: '/usr/local/opt/libomp/lib/libomp.dylib' (no such file), '/usr/local/lib/libomp.dylib' (no such file), '/usr/lib/libomp.dylib' (no such file)

libomp was already installed with brew install libomp

In a linux machine, after the install the error was: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.27' not found
I can not use the library in both machines

[fabsig]

@joselznom: Thanks for your feedback.

Concerning Mac: Are you compiling from source or is the pip pre-compiled wheel used? Have you uninstalled and reinstalled brew after you migrated to an M1/M2 Mac? I needed to do this. If you migrate, brew will be in x86_64 mode and not arm64, but you need the arm64 version.

Concerning Linux: In this case, you need to install from source (i.e. build the shared library on the machine that you use); see here for instructions how to do this. That is a known issue resulting from LightGBM (see e.g. microsoft/LightGBM#701) which is used by GPBoost for the tree-boosting part. See also #2 and #41.

[joselznom]

On Mac: I have tried both versions, the pip pre-compiled was the only one that I could install but I have the issue with libomp. and brew was installed natively on M1. From source a fatal error occurs and I can not install the package.

On linux: Installing from source with the pip command solve the problem, thank you :)

Finally, one question regarding the package, could the scikit-learn API could be used to train mixed-effect or random-effects models, or only with this can be done only with the GPModel function from the package? I do not see the grouped_data parameter for the scikit-learn API

[fabsig]

@joselznom: glad to hear that it works on Linux.

I keep wondering why it does not work on your Mac with an arm64 processor. Initially, I also had a similar problem as you on my machine, but then I solved it. It would be good to know what is the root cause. Also, I use mini forge instead of conda (it might be that conda has issues with Mac arm64 despite having an arm64 version, see microsoft/LightGBM#4843). I assume you use the arm64 version of conda? If you have time, you could try using mini forge.

The scikit-learn API cannot be used to train (G)LMMs. This can only be done using the native gpboost GPModel class.

[joselznom]

@fabsig Yes, I am using mini-forge.
The log shows an issue with cmake when I try the source installation but I do not why. I will try to reinstall all to check if the problem can be solved.
Thank you for the support :)

[fabsig]

I reset my arm64 Mac to factory level to double check whether it works. After migrating from an old non-arm64 Mac, I did the following steps such that installation with pip install gpboost works:

• remove existing python environments
• uninstall homebrew (see https://stackoverflow.com/questions/72890277/i-cant-uninstall-brew-on-macos-apple-silicon)
• install homebrew (https://treehouse.github.io/installation-guides/mac/homebrew)
• install OpenMP (brew install libomp)
• install Miniforge (brew install miniforge; conda init "$(basename "${SHELL}")")

With this everything runs smoothly. It might be that Miniforge is not needed and Anaconda works also, but matplotlib crashed on my machine when using Anaconda, so I used Miniforge anyway...