update conda executable location for Linux/macOS

[Quar]

For changes since [email protected]#recommended-change-to-enable-conda-in-your-shell, when non-root Conda Environment is used, executable file conda will not necessarily exists under bin_dir(ROOTENV), which causing Conda.jl constantly try to install Miniconda.

Using environment variable $CONDA_EXE to locate conda executable is recommend.

For example:

# creating non-root conda env
conda create -n conda_jl python conda
# add following line to `~/.bashrc`, `~/.zshrc` or other rc files to avoid typing everytime
export CONDA_JL_HOME="/path/to/miniconda/envs/conda_jl"
# rebuild Conda
julia -e 'using Pkg; Pkg.build("Conda")'
#  try to list installed Conda packages
julia -e 'using Conda; Conda.list()'  # will still trying to install miniconda

Test passed with test Conda.

[tkf]

Thanks for the PR! Is CONDA_EXE documented somewhere? It looks like conda/conda#6810 is the only source (except the code)?

[tkf]

This whole if can be written as get(ENV, "CONDA_EXE", joinpath(bin_dir(ROOTENV), "conda")).

[Quar]

Thanks for the clean solution, I am still quite new to Julia -- TIL!

You are right, CONDA_EXE is not documented in the quite obsolete user document, albeit being mentioned in release note.

However conda/conda#7126 (with detailed gitter chat record) might be quite evidential that $CONDA_EXE is considerably the de facto way to refer to conda executable after conda 4.4+...

[tkf]

Continuing #146 (comment). Good to know. Thanks for clarification.

I just had a look at Conda.jl to refresh my memory and I noticed a few things:

First of all, if ENV["CONDA_JL_HOME"] is not set, we should use const conda = joinpath(bin_dir(ROOTENV), "conda") always in *nix. Conda.jl is designed to use its own miniconda installation by default. This behavior should not be changed if the user has another conda environment.

Also, I think ENV["CONDA_JL_HOME"] is actually incomplete as a configuration option because conda needs the command itself and the place for the environment. I think we need (say) ENV["CONDA_JL_CONDA_EXE"] and save it as Conda.conda. (In principle, we can use ENV["CONDA_EXE"] only if ENV["CONDA_JL_HOME"] is set. But it sounds complicated.) Since Conda.jl tries to install miniconda whenever the path Conda.conda does not exist, this condition can never be met if Conda.conda is not in ENV["CONDA_JL_HOME"]:

Conda.jl/src/Conda.jl

Lines 151 to 153 in 3d84627

	function _install_conda(env::Environment, force::Bool=false)
	if force || !isfile(Conda.conda)
	Compat.@info("Downloading miniconda installer ...")

I think the right solution is to save the environment variable (say) CONDA_JL_CONDA_EXE = get(ENV, "CONDA_JL_CONDA_EXE", nothing) here

Conda.jl/deps/build.jl

Lines 36 to 39 in 3d84627

	deps = """
	const ROOTENV = "$(escape_string(ROOTENV))"
	const MINICONDA_VERSION = "$(escape_string(MINICONDA_VERSION))"
	"""

and then just error out in _install_conda if CONDA_JL_CONDA_EXE !== nothing and Conda.conda (hence CONDA_JL_CONDA_EXE) does not exist. This requires users to re-build Conda.jl if conda executable at ENV["CONDA_JL_CONDA_EXE"] is removed or relocated. This should be mentioned in the error message.

Also, ideally we better check it in Conda.__init__ and throw an error at import time.

[Quar]

Perhaps something crudely like:

master...Quar:feat_conda_exe
(since code snippet preview doesn't supported cross repo, a simple compare might be easier to glance at)

Additionally, would it be helpful to add another function (say) Conda.info() to print out current environment settings and path to conda executable (as Conda.conda recognized)? Such that one can easily check system environment settings, and other Conda functions may reuse when emitting exception messages?

Still working on Conda.__init__ and adding automated tests.

[marius311]

Just want to confirm that on a system where I use a non-root environment, e.g.

$ conda env list
# conda environments:
#
local                 *  /global/homes/m/marius/.conda/envs/local
base                     /usr/common/software/python/3.7-anaconda-2019.07

I need this PR otherwise Conda.jl doesn't work. Not sure if there's other better workarounds or if this should be merged.

[stevengj]

We don't want to use whatever Conda the user might have installed — we want to use specifically the conda executable installed by Conda.jl. So under ordinary circumstances, we shouldn't be looking at the CONDA_EXE environment variable set by the user (which might refer to a completely different Anaconda installation).

It seems like you guys are referring to the situation where the user has configured Conda.jl with a different CONDA_JL_HOME referring to a pre-existing Anaconda installation … in that case, it sounds like we need to try harder to find the conda executable.

[stevengj]

(Honestly, using CONDA_JL_HOME to specify an Anaconda distro seems to mostly defeat the purpose of Conda.jl, which is not having to manage your own Conda installation … I'd be happier just removing it.)

[tkf]

Having multiple installations of conda actually defeats the purpose of conda; handle multiple environment, centralize resources like download cache, de-duplicate files as much as possible, etc. I sympathise with you that dealing with broken conda environments questions from newbies is not fun. But keeping escape hatch for users already familiar with conda sounds reasonable to me provided that it wouldn't be enabled unless the user explicitly asks for it.

we shouldn't be looking at the CONDA_EXE environment variable set by the user

That's why I suggested CONDA_JL_CONDA_EXE above.

[ViralBShah]

This doesn't seem necessary any longer (in that Conda.jl works well nowadays). Please reopen if still something we should do.