Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating install scripts for Ubuntu 18.04 - WIP #578

Closed
wants to merge 5 commits into from
Closed
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 10 additions & 29 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,33 +1,14 @@
astroplan
astropy >= 3.0.0
ccdproc
codecov
coveralls
dateparser
# These are packages that are NOT available via our
# choice of conda repositories; there are less well
# known repositories that do offer them, but that is
# more of a security risk.

# Python wrapper for ffmpeg.
ffmpy
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could probably lose the option. It is just a thin wrapper around ffmpeg. Since we don't have many varying options it would be trivial to write our own script.

gcloud
google-cloud-storage
matplotlib >= 2.0.0
mocket
numpy >= 1.12.1
pycodestyle == 2.3.1
pymongo >= 3.2.2
pyserial >= 3.1.1
pytest >= 3.4.0
python_dateutil >= 2.5.3
PyYAML >= 3.11
pyzmq >= 15.3.0
readline
requests
scikit_image >= 0.12.3
scipy >= 0.17.1

# Jupyter-notebook requires Tornado versions between 4 and 5. See:
# https://stackoverflow.com/questions/49141525/install-jupyter-notebook-on-miniconda
# https://github.com/ipython/ipython/issues/8249
# https://github.com/jupyter/help/issues/324
# Mock object support for Python testing.
mocket
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could potentially work around this. Since we have requests elsewhere now it might be cleaner to switch TheSkyX calls to requests and then use some of the mock tools provided for it, which are available.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to keep changes in this PR to a minimum (i.e. it will be large enough as is). I'd like to focus on just the switch to conda and 18.04. In fact, maybe I already shoved to much into this, and should extract the 18.04 (Mongodb) changes. Sigh. Lots of slow tests required.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, don't want to do it for this PR, mostly just commenting on what I think we could eventually get rid of.


tornado == 4.5.3
# State machine library for Python.
transitions >= 0.4.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And technically I'm still a maintainer on this project though I haven't done anything for years. 😕 I can find out if there is any plan.

tweepy
wcsaxes

32 changes: 31 additions & 1 deletion scripts/install/conda-packages-list.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,33 @@
pandas
astroplan
astropy >= 3.0.0
ccdproc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though conda fixes the problems with this (via astroscrappy), we are not actually using it anywhere in the repo. Might as well just remove.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove ccdproc?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so. It will come through in PIAA probably anyway but might as well get rid of it here since it's not an actual dependency.

codecov
coveralls
cython
dateparser
google-cloud-sdk
google-cloud-storage
matplotlib >= 2.0.0
numpy >= 1.12.1
pycodestyle == 2.3.1
pymongo >= 3.2.2
pyserial >= 3.1.1
pytest >= 3.4.0
pandas
python
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this actually install the python version from here and if so should we specify a version?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specifying a version is problematic. If you say ==3.7, then the constraints are unsolvable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha.

python-dateutil >= 2.5.3
PyYAML >= 3.11
pyzmq >= 15.3.0
readline
requests
scikit-image >= 0.12.3
scipy >= 0.17.1

# Jupyter-notebook requires Tornado versions between 4 and 5. See:
# https://stackoverflow.com/questions/49141525/install-jupyter-notebook-on-miniconda
# https://github.com/ipython/ipython/issues/8249
# https://github.com/jupyter/help/issues/324

tornado == 4.5.3
tweepy
wcsaxes
24 changes: 22 additions & 2 deletions scripts/install/install-dependencies.sh
Original file line number Diff line number Diff line change
Expand Up @@ -469,6 +469,24 @@ if [[ "${DO_CONDA}" -eq 1 ]] ; then
install_conda_if_missing
fi

# Add the astropy channel, i.e. an additional repository in which to
# look for packages. With conda 4.1.0 and later, by default the highest
# priority repository that contains a package is used as the source for
# that package, even if there is a newer version in a lower priority
# package. And by default the most recently added repository is treated
# as the highest priority repository. Here we use prepend to be clear
# that we want astropy to be highest priority.
if [[ -n "$(conda config --show channels | fgrep astropy)" ]] ; then
conda config --remove channels astropy
fi
conda config --prepend channels astropy

# And put conda-forge at the back of the line.
if [[ -n "$(conda config --show channels | fgrep conda-forge)" ]] ; then
conda config --remove channels conda-forge
fi
conda config --append channels conda-forge

# Use the base Anaconda environment until we're ready to
# work with the PANOPTES environment.
conda activate base
Expand All @@ -491,7 +509,6 @@ if [[ "${DO_CONDA}" -eq 1 || "${DO_CREATE_CONDA_ENV}" -eq 1 || \
echo
echo "Updating base conda installation."
conda update --quiet --yes -n base conda
conda update --quiet --all --yes
fi

if [[ "${DO_CREATE_CONDA_ENV}" -eq 1 ]] ; then
Expand All @@ -509,7 +526,8 @@ if [[ "${DO_CONDA}" -eq 1 || "${DO_CREATE_CONDA_ENV}" -eq 1 || \
echo_bar
echo
echo "Updating packages in panoptes-env."
conda update --quiet --all --yes
conda update -n panoptes-env --quiet --yes conda
conda update -n panoptes-env --quiet --yes --all
fi

if [[ "${DO_INSTALL_CONDA_PACKAGES}" -eq 1 ]] ; then
Expand All @@ -523,6 +541,8 @@ echo_bar
echo "PATH=$PATH"
echo_bar

exit

if [[ "${DO_PIP_REQUIREMENTS}" -eq 1 ]] ; then
echo_bar
echo
Expand Down