Multiple fixes to get conda setup working in more environments (e-mis…

…sion#741)

* Multiple fixes to get conda setup working in more environments

My personal laptop had a version of conda even older than the docker container.
So old that it only supported `source activate` and not `conda activate`.
So we used `source activate` in the setup script and checked for the CI
variable before using it in `conda activate`.

This:
- supports `setup_conda` on multiple platforms
- upgrades all setup scripts to use `conda install`
- stops exiting from `check_for_conda` so it doesn't kill the shell
- updates the manual install test to also include osx, and to add osx to the
  list of oses that we test against
    - also pass in the platform correctly per OS so that the installation will
      happen correctly

* Mongodb install action only works on ubuntu

So we test install only ubuntu + osx
and go back to linux only for install + tests

This should also catch any inconsistencies between the dev setup and the test
setup scripts (similar to e-mission/e-mission-docs#513 (comment))

* Remove miniconda check since osx doesn't have miniconda preinstalled

* Add exit codes so we can see why this is failing

* Checking to see if the error is with which conda

* Check the conda path only on linux

Since it inexplicably fails on OSX
+ remove exit 1 since it is confusing in source scripts

* Final fixes

- switch to conda on teardown as well
- actually switch to the environment after installing it to ensure that it exists

* Some more clarity into what we are doing

So that there is more indication of what failed if something fails

.github/workflows/osx-ubuntu-manual-install.yml

@@ -0,0 +1,73 @@
+name: osx-ubuntu-manual-install
+
+# Controls when the action will run. Triggers the workflow on push or pull request
+# events but only for the master branch
+on:
+  push:
+    branches: [ master ]
+  pull_request:
+    branches: [ master ]
+  schedule:
+    # * is a special character in YAML so you have to quote this string
+    - cron:  '5 4 * * 0'
+
+# A workflow run is made up of one or more jobs that can run sequentially or in parallel
+jobs:
+  # This workflow contains a single job called "build"
+  build:
+    # The type of runner that the job will run on
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [ubuntu-16.04, ubuntu-18.04, ubuntu-latest, macos-latest]
+        include:
+        - os: macos-latest
+          PLATFORM: MacOSX-x86_64
+        - os: ubuntu-16.04
+          PLATFORM: Linux-x86_64
+        - os: ubuntu-18.04
+          PLATFORM: Linux-x86_64
+        - os: ubuntu-latest
+          PLATFORM: Linux-x86_64
+
+    # Steps represent a sequence of tasks that will be executed as part of the job
+    steps:
+    # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
+    - uses: actions/checkout@v2
+
+    - name: Install miniconda
+      shell: bash -l {0}
+      run: |
+        source setup/setup_conda.sh ${{ matrix.PLATFORM }}
+        if [ ${{ matrix.PLATFORM }} == "Linux-x86_64" ]; then
+            echo "Linux install, going to check conda path"
+            which conda
+        else
+            echo "OSX install, skipping conda path check"
+            echo "    since 'which conda' inexplicably returns with status code 1"
+            echo "    only in the CI environment"
+        fi
+
+    - name: Check whether the CI environment variable is set
+      shell: bash -l {0}
+      run: |
+        source "$HOME/miniconda/etc/profile.d/conda.sh"
+        echo $CI
+
+    - name: Setup the emission environment
+      shell: bash -l {0}
+      run: |
+        conda --version
+        which conda
+        source "$HOME/miniconda/etc/profile.d/conda.sh"
+        conda --version
+        source setup/setup.sh
+        echo "About to activate environment"
+        conda activate emission
+        conda env list
+
+    - name: Teardown the emission environment
+      shell: bash -l {0}
+      run: |
+        source "$HOME/miniconda/etc/profile.d/conda.sh"
+        source setup/teardown.sh

.github/workflows/test-with-manual-install.yml

@@ -1,6 +1,6 @@
 # This is a basic workflow to help you get started with Actions
 
-name: test-with-manual-install
+name: ubuntu-only-test-with-manual-install
 
 # Controls when the action will run. Triggers the workflow on push or pull request
 # events but only for the master branch
@@ -40,7 +40,7 @@ jobs:
     - name: Install miniconda
       shell: bash -l {0}
       run: |
-        source setup/setup_conda.sh
+        source setup/setup_conda.sh Linux-x86_64
         which conda
 
     - name: Check whether the CI environment variable is set

setup/checks/check_for_conda.sh

@@ -4,7 +4,6 @@ EXP_CONDA_VER=4.5.12
 if [ $CURR_CONDA_VER == $EXP_CONDA_VER ]; then
     echo "For conda, found $CURR_CONDA_VER, expected $EXP_CONDA_VER, all is good!"
 else
-    echo "For conda, found $CURR_CONDA_VER, expected $EXP_CONDA_VER, run setup/setup_conda.sh to get the correct version"
-    exit 1
+    echo "For conda, found $CURR_CONDA_VER, expected $EXP_CONDA_VER, run 'bash setup/setup_conda.sh' to get the correct version"
 fi
 

setup/setup.sh

@@ -9,7 +9,7 @@ source setup/checks/check_for_conda.sh
 
 echo "Setting up blank environment"
 conda create --name emission python=3.6
-source activate emission
+conda activate emission
 
 echo "Downloading packages"
 curl -o /tmp/cachetools-2.1.0-py_0.tar.bz2 -L https://anaconda.org/conda-forge/cachetools/2.1.0/download/noarch/cachetools-2.1.0-py_0.tar.bz2

setup/setup_conda.sh

@@ -1,9 +1,18 @@
 EXP_CONDA_VER=4.5.12
 
-wget https://repo.continuum.io/miniconda/Miniconda3-$EXP_CONDA_VER-Linux-x86_64.sh -O miniconda.sh;
-bash miniconda.sh -b -p $HOME/miniconda
-source "$HOME/miniconda/etc/profile.d/conda.sh"
-hash -r
-conda config --set always_yes yes --set changeps1 no
-# Useful for debugging any issues with conda
-conda info -a
+PLATFORM=$1
+echo "Installing for platform $PLATFORM"
+
+if [ -z $PLATFORM ]; then
+    echo "Usage: setup_conda.sh <platform>"
+    echo "   Platform options are Linux-x86_64, MacOSX-x86_64"
+    echo "   For Windows, manually download and install https://repo.anaconda.com/miniconda/Miniconda3-$EXP_CONDA_VER-Windows-x86_64.exe"
+else
+    curl -o miniconda.sh -L https://repo.continuum.io/miniconda/Miniconda3-$EXP_CONDA_VER-$PLATFORM.sh;
+    bash miniconda.sh -b -p $HOME/miniconda
+    source "$HOME/miniconda/etc/profile.d/conda.sh"
+    hash -r
+    conda config --set always_yes yes --set changeps1 no
+    # Useful for debugging any issues with conda
+    conda info -a
+fi

setup/setup_tests.sh

@@ -4,11 +4,7 @@ source setup/checks/check_for_conda.sh
 
 echo "Setting up blank environment"
 conda create --name emissiontest python=3.6
-if [ ${CI} == "true" ] ; then
-    conda activate emissiontest
-else
-    source activate emissiontest
-fi
+conda activate emissiontest
 
 echo "Downloading packages"
 curl -o /tmp/cachetools-2.1.0-py_0.tar.bz2 -L https://anaconda.org/conda-forge/cachetools/2.1.0/download/noarch/cachetools-2.1.0-py_0.tar.bz2

setup/teardown.sh

@@ -1,2 +1,2 @@
-source deactivate emission
+conda deactivate emission
 conda env remove --name emission

setup/teardown_tests.sh

@@ -1,11 +1,7 @@
 #!/bin/bash
 
 echo "Removing environment from "${CONDA_TEMP_PREFIX}
-if [ ${CI} == "true" ] ; then
-    conda deactivate
-else
-    source deactivate emissiontest
-fi
+conda deactivate
 
 conda env remove --yes --name emissiontest
 # rm conf/net/ext_service/habitica.json