Intel mkl dependency may break the installation for ARM Macs

[SeaCelo]

ARM Macs cannot use the Intel MKL library. Other libraries are available. See for example this comment thread: https://developer.apple.com/forums/thread/695963.

The installation of OG-CORE requires this MKL library in two places: environment.yml and setup.py
This seems (not confirmed) to be breaking the installation for Macs running the ARM chips (M1, M2).

There are two ways to getting around this for now.

*First, make sure you installed anaconda correctly (I prefer miniforge). See links below for some help.

*Second:
A) line 8 of environment.yml should be commented if you are running ARM Mac.
B) setup.py introduced the package dependency requirement of 'mkl' in this commit: https://github.com/PSLmodels/OG-Core/commit/4a09c5d1a27946af941038a3a8526ddde92c6dc3

PS: If you have an M1 Mac you should be installing anaconda for the right architecture. Some (older) guides:
https://www.youtube.com/watch?v=w2qlou7n7MA
https://towardsdatascience.com/how-to-easily-set-up-python-on-any-m1-mac-5ea885b73fab
https://stackoverflow.com/questions/65534042/anaconda-and-upgrading-to-new-m1-mac

[rickecon]

@SeaCelo. Can you post the error message you are getting? Or maybe you've already fixed it. I have not run into this problem on my machine (MacBook Pro, Apple M1 Max Chip). Our recommended installation steps are the following:

1. Install the Anaconda distribution of Python

• If already installed, make sure:
  • your conda package manager is up-to-date: conda update conda
  • the Anaconda distribution of Python is up-to-date: conda update anaconda

2. Create or update the ogcore-dev conda environment

• If creating the environment, navigate to the OG-Core directory on your local machine in your terminal (Mac and Linux) or Anaconda prompt (Windows) and type conda env create -f environment.yml
• If updating the environment, I recommend removing the existing environment conda remove --name ogcore-dev --all and installing it new conda env create -f environment.yml

3. Activate the new ogcore-dev conda environment by typing conda activate ogcore-dev and add the ogcore package to that environment by typing pip install -e . in your terminal or Anaconda prompt window in the top OG-Core directory.

You can now test whether this works by being in your terminal or Anaconda prompt and having the ogcore-dev conda environment active and typing python ./run_examples/run_ogcore_example.py.

After completing these steps, everything should be set up on your machine to run OG-Core. We have tested this on Mac, Windows, and Linux. Let me know if you still have problems running OG-Core after completing these steps.

[rickecon]

@jdebacker and @SeaCelo. I am currently running time and run tests of the run_ogcore_example.py script for the specifications in the table below. In particular, I am testing the performance of the model with and without the mkl for operating systems Mac OSx (both M1 and non M1 Anaconda distribution), Windows, and Linux. I want to try to get rid of the mkl dependency, which causes problems with the Anaconda M1 distribution of Python. [I am currently running these and filling out the rows.]

OS	Python	Cores	SciPy	mkl	Baseline	Reform	Tests pass	Tests time
OSx	3.10.8	7	1.9.3	2022.2.1	1802.11	1908.19	yes	25060.85(?)
OSx	3.10.8	7	1.9.3	none	1896.64	1815.10	yes	20323.71
OSx	3.10.8 (M1)	n/a	n/a	yes	n/a	n/a	n/a	n/a
OSx	3.10.8 (M1)	7	?	none	1054.77	1042.07	no
Windows	?	7	?	?
Windows	?	7	?	none
Linux	3.10.8	7	1.9.3	2022.2.1	3188.15	3093.35	no
Linux	3.10.8	7	1.9.3	none	3120.38	3158.75	no

• I ran the three OSx runs (rows 1, 2, and 4 of table) on my M1 Apple MacBook Pro, M1 Max, 64GB, Ventura 13.0.1, 10 cores (8 performance, 2 efficiency)
• The two runs on OSx with the ARM M1 chip optimization required me to uninstall my traditional Anaconda distribution of Python and install the Anaconda 64-bit M1 distribution for Mac OSx.
• The first run of the M1 Python distribution for OSx (third row of table) was not possible because of the incompatibility between the mkl package and the M1 distribution packages (see @SeaCelo comments above and below).
• I ran the two Linux runs (rows 7 and 8 of table) on a Google Cloud virtual machine instance with the following specs:

Google Cloud Platform (GSP) Virtual Machine (VM) with Linux

Characteristic	Description
Operating system	Linux, Debian 11 bullseye v20221206
Machine type	e2-standard-8 (8-core)
CPU platform	Intel Broadwell
Architecture	x86/64
Boot disk size	25 GB

[SeaCelo]

@rickecon Please also check Activity Monitor when running the test. Depending on how I installed Python it would report either Intel or Apple under "Kind". As far as I can tell there is no easy way to ask Python to self-report its build architecture. That is the only way I found to check if Python is running under Rosetta.

More on our tests here: EAPD-DRB/OG-ZAF#15

Whether Python was running under Rosetta or natively had a large performance delta in my tests:

OS	Python	Cores	SciPy	math library	Baseline + Reform (mins)
OSX	3.9.12 ARM	8+2	1.9.3	"openblas"	36.29
OSX	3.9.12 ARM	8+2	1.9.3	"accelerate"	37.00
OSX	3.9.12 Intel	8+2	1.9.3	"mkl"	59.29

[SeaCelo]

Linking to the discussion that explains the issue and a work around.
Explanation

Workaround