Maintaining isolated virtual environments when building from Source for Apple Silicon

[oradwastaken]

Hey all,

I am following up on some discussion that I started having in under the "Improved Apple Silicon Support" issue: #1853

I have always used miniconda to set up my virtual environments and to maintain my package dependencies. I have recently switched to a new M1 Mac, and now need to "build from source" for the first time.

My current approach has been to use conda to create a virtual environment, install whatever packages I can via conda, then to supplement with pip installs. Finally, I download the remaining dependencies and install them with ./configure and the appropriate flags, as suggested in the link above.

Once all of the installations are done, I can import meep successfully if I add it to the $PYTHONPATH environment variable. I found that adding the following to .zshrc seems to work:
export PYTHONPATH=$PYTHONPATH:/usr/local/lib/python3.9/site-packages

If I look in this directory, I find only one thing: meep.

With this configuration, if I am in the right conda environment, my meep python codes seem to run fine in the command line.

First of all, this doesn't seem to work seamlessly with PyCharm. I need to add this path almost by hand every time I open the software into the custom interpreter configuration. That might be a bug with PyCharm on macOS, and is not the issue I would like to raise here; however, I suspect if we correct the real issue, this problem might go away as well.

Second of all, (and more importantly) I found it to be surprising that the path reference python 3.9 given that conda was installing python 3.8. It seems like the "from source" installation does a system-wide installation of meep as opposed to one confined within my meep conda environment. If I'm correct, I'm not happy with that situation.

I tried to find other 'site-packages' folders. When I run:
find . -name site-packages
in the command line, I find one related to my miniconda installation: /Users/orad/miniconda3/envs/pmeep/lib/python3.9/site-packages.

However, there is no 'meep' in this folder. Indeed, if I add this to my PYTHONPATH, I cannot run any meep in python.

I redid all of the installations, but redirected the "PYTHON=" flag to my miniconda installation, e.g.,
./configure CPPFLAGS="-I$(brew --prefix)/include" LDFLAGS="-L$(brew --prefix)/lib" PYTHON=/Users/orad/miniconda3/envs/pmeep/bin/python && make && sudo make install

I expected this to add meep to that miniconda site-packages, but it doesn't do that.

At this point, I'm at an impasse. So, what is the correct way of installing meep and associating it with an isolated conda virtual environment?

Thanks in advance for the help!

[oradwastaken]

I figured this out a while ago but never posted my solution. In conda, you don't use the PYTHONPATH. Instead, you need to add meep to your conda environments site-packages folder. This line works for me:

ln -s /usr/local/lib/python3.9/site-packages/meep $CONDA_PREFIX/lib/python3.9/site-packages

[thomaslima]

I don't know if it helps at this stage, but I successfully installed meep app to a Conda environment by adding the following flags to the ./configure command (the rest is as you did): ./configure ... PYTHON=$CONDA_PREFIX/bin/python --with-python_prefix=$CONDA_PREFIX

[stevengj]

Right, the correct way to fix this is to use --with-python_prefix=$CONDA_PREFIX to tell it where to install python files (instead of installing in the wrong place and then linking to the correct place)

[stevengj]

As noted by @thomaslima above, do ./configure ... PYTHON=$CONDA_PREFIX/bin/python --with-python_prefix=$CONDA_PREFIX