Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add scripts/install/auto-install.sh #824

Merged
merged 43 commits into from
May 21, 2019
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
9917cf4
First draft of an auto-install script for POCS.
jamessynge Apr 14, 2019
d4ebdf8
Fix name of default branch (develop, not master).
jamessynge Apr 14, 2019
8f17431
Add cloning/pulling of PAWS
jamessynge Apr 14, 2019
b5a9f2d
Re-enable calling install-dependencies.sh
jamessynge Apr 14, 2019
7cdc970
Add some more whitespace, so it is easier to parse the output
jamessynge Apr 14, 2019
52dd23a
Add a separator line (echo_bar) between major commands
jamessynge Apr 14, 2019
41cdf85
Remove debugging option
jamessynge Apr 14, 2019
c16558b
Adapt to change in astrometry version string format.
jamessynge Apr 14, 2019
d11f8fe
Correctly punctuate a multi-line pipe
jamessynge Apr 14, 2019
4031a81
Add install-gcloud.sh, which installs Google Cloud SDK
jamessynge Apr 15, 2019
9795b1e
Add additional documentation
jamessynge Apr 16, 2019
9efc869
Add info about how create the PANUSER.
jamessynge May 4, 2019
ef1cd69
Provide more guidance on how to run the script.
jamessynge May 4, 2019
3f60c0d
Add missing fi
jamessynge May 4, 2019
9627824
Fix then
jamessynge May 4, 2019
13c2c91
wip
jamessynge May 4, 2019
452d8bc
fix redirects
jamessynge May 4, 2019
de5a509
wip
jamessynge May 4, 2019
ec2d089
wip
jamessynge May 4, 2019
1abd4e1
wip
jamessynge May 4, 2019
e1ed9cf
Set group owner of /var/panoptes
jamessynge May 4, 2019
32f3095
Try $COLUMNS for width of terminal.
jamessynge May 4, 2019
c4188fa
Try "tput cols" for width of terminal.
jamessynge May 4, 2019
8523041
Add build-essentials dependency
jamessynge May 4, 2019
c5431c6
Fix build-essential dependency
jamessynge May 4, 2019
35e8bdb
Add urllib3 to requirements.txt
jamessynge May 5, 2019
e8cdc37
Restrict version of urllib3
jamessynge May 5, 2019
c84fdda
Support both origin and upstream for POCS clone.
jamessynge May 5, 2019
f452e32
Fix bug in update_shell_rc_file.
jamessynge May 5, 2019
42c290d
Set name of remote during clone.
jamessynge May 5, 2019
1b14b96
Add missing punctuation, fix comments.
jamessynge May 5, 2019
db22f84
Add function ensure_directory_ownership, use for $PANDIR
jamessynge May 5, 2019
a0e2ad2
Add a question for Wilfred about whether to execute
jamessynge May 5, 2019
e2753a1
Change format of an output string so that shellcheck isn't confused b…
jamessynge May 5, 2019
02c3d1e
Add support for conditionally invoking install-gcloud.sh
jamessynge May 5, 2019
8118344
Remove verbose logging from install-apt-packages.sh.
jamessynge May 5, 2019
96350bd
Reformat message from echo_running_sudo.
jamessynge May 5, 2019
7d7b52d
Replace use of curl with wget.
jamessynge May 5, 2019
3955798
Make apt-get quiet.
jamessynge May 5, 2019
433471f
Remove duplicate sources for the gcloud sdk.
jamessynge May 5, 2019
732a2ca
Reformat sudo message.
jamessynge May 5, 2019
0f77dc6
Don't clone POCS or PAWS if it appears they exist; instead issue
jamessynge May 11, 2019
f742e52
Change clean_PATH so it can be standalone.
jamessynge May 12, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
147 changes: 147 additions & 0 deletions scripts/install/auto-install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
#!/bin/bash
# auto-install.sh installs git if it isn't installed, then clones POCS and
# installs its dependencies.
#
# To fetch this from github and execute it immediately, run one of these two
# commands in a bash shell, the first if you have wget installed, the second
# if you have curl installed:
#
# 1a) wget -O - https://raw.githubusercontent.com/panoptes/POCS/develop/scripts/install/auto-install.sh | bash
# 1b) bash <(wget -O - https://raw.githubusercontent.com/panoptes/POCS/develop/scripts/install/auto-install.sh)
#
# 2) bash <(curl -s https://raw.githubusercontent.com/panoptes/POCS/develop/scripts/install/auto-install.sh)
#
# To determine whether you have these commands install, type these commands:
#
# which wget
# which curl

################################################################################
# Env vars used for debugging of this script; these allow you to point to your
# fork of POCS on github, so that you can download your fork instead of the
# primary repo. There is no support for doing the same with PAWS.

[[ -z "${POCS_GITHUB_USER}" ]] && export POCS_GITHUB_USER="panoptes"
[[ -z "${POCS_GIT_URL}" ]] && export POCS_GIT_URL="https://github.com/${POCS_GITHUB_USER}/POCS.git"
[[ -z "${POCS_BRANCH}" ]] && export POCS_BRANCH="develop"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is fine, but I have been thinking lately that we should go back to pushing release versions to master and having non-developer units always pull from master. Mostly the onus here would be on me to actually be good about making releases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that might be a good idea to use master that way. I.e. on rare occasions, and after testing on multiple units, we declare a new version that non-developers should use.


################################################################################
# Functions COPIED from install-helper-functions.sh
# Print the disk location of the first arg.
function safe_which() {
type -p "${1}" || /bin/true
}

# Print a separator bar of # characters.
function echo_bar() {
local terminal_width
if [ -n "${TERM}" ] && [ -t 0 ] ; then
if [[ -n "$(which resize)" ]] ; then
terminal_width="$(resize 2>/dev/null | grep COLUMNS= | cut -d= -f2)"
elif [[ -n "$(which stty)" ]] ; then
terminal_width="$(stty size 2>/dev/null | cut '-d ' -f2)"
fi
fi
printf "%${terminal_width:-80}s\n" | tr ' ' '#'
}
################################################################################

function do_sudo() {
if [ "$(id -u -n)" == "root" ] ; then
echo "Running ${*}"
"$@"
else
echo "Running sudo ${*}; you may be prompted for your password."
(set -x ; sudo "$@")
fi
}

function clone_or_update() {
local -r REPO_DIR="${1}"
local -r REPO_URL="${2}"
local -r BRANCH="${3}"

echo_bar

if [ ! -d "${REPO_DIR}/.git" ]
then
if [ -d "${REPO_DIR}" ]
then
echo 2> "
The directory (${REPO_DIR}) already exists, but doesn't appear
to be a valid git repository. Please remove it (or move it out
of the way) and re-run this script.
"
exit 1
fi
echo "
Cloning ${REPO_URL} into ${REPO_DIR}
"
(set -x ; git clone "${REPO_URL}" "${REPO_DIR}")
cd "${REPO_DIR}"
git checkout "${BRANCH}"
jamessynge marked this conversation as resolved.
Show resolved Hide resolved
else
echo "
Pulling the latest software into the worktree at ${REPO_DIR}
jamessynge marked this conversation as resolved.
Show resolved Hide resolved
"
cd "${REPO_DIR}"
git fetch --all
git checkout "${BRANCH}"
git pull
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If they have already set up the repo they might have origin set to their repo and this pull might not necessarily be doing what we want. Or they might have files in staging, in which case this will fail.

I think if the dir is there and a repo we just do nothing other than maybe print a message reminding them to update. Thoughts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, clearly if there are already files staged then we need to avoid messing with the repo.
Of course, if they have a repo, they don't need this auto-install script, and can run install-dependencies.sh themselves.

I'll adjust the script to stop if there is a POCS or PAWS repo.

fi
}

# Exit immediately if a command fails:
set -e

# COPIED from default-env-vars.sh
[[ -z "${PANUSER}" ]] && export PANUSER="panoptes" # Default user
[[ -z "${PANDIR}" ]] && export PANDIR="/var/panoptes" # Main Dir
[[ -z "${PANLOG}" ]] && export PANLOG="${PANDIR}/logs" # Log files
[[ -z "${POCS}" ]] && export POCS="${PANDIR}/POCS" # Main Observatory Control
[[ -z "${PAWS}" ]] && export PAWS="${PANDIR}/PAWS" # Web Interface

if [[ "$(whoami)" != "${PANUSER}" ]] ; then
echo >2 "Please run this script as ${PANUSER}, not as $(whoami)"
jamessynge marked this conversation as resolved.
Show resolved Hide resolved
exit 1
fi

# Let's assume we'll need to run apt-get install, so first run apt-get update
# which will refresh caches used during apt-get install.
echo_bar
do_sudo apt-get update

if [ ! -x "$(safe_which git)" ]
then
echo_bar
echo "
git is not installed, so installing it...
"
do_sudo apt-get install -y git
fi

echo_bar
echo "
Ensuring that ${PANDIR} exists
jamessynge marked this conversation as resolved.
Show resolved Hide resolved
"
if [ ! -d "${PANDIR}" ]
then
do_sudo mkdir -p "${PANDIR}"
fi

echo_bar
echo "
Ensuring that ${PANDIR} is owned by user ${PANUSER}
jamessynge marked this conversation as resolved.
Show resolved Hide resolved
"
do_sudo chown "${PANUSER}" "${PANDIR}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
do_sudo chown "${PANUSER}" "${PANDIR}"
do_sudo chown -R "${PANUSER}:${PANUSER}" "${PANDIR}"

I added the recursive option because I was thinking the subdirectories existed at this point but it looks like not but the option shouldn't hurt. Also not sure if wanted the group changed or not, I usually default to changing both.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried that a while ago, even had it in this code earlier today, but then I remembered that it was really very slow if the full tree was already there. I can reconsider/retest if you like.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. It seems like there shouldn't be that many files in the tree. But okay not doing it. That means are assuming if it already exists then the user has already set their permissions accordingly, which I think is fine.

Copy link
Contributor Author

@jamessynge jamessynge May 10, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added it, and it worked fine. I now recall that the slow case was inside of docker.


clone_or_update "${POCS}" "${POCS_GIT_URL}" "${POCS_BRANCH}"
clone_or_update "${PAWS}" "https://github.com/panoptes/PAWS.git" "develop"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could obviously add the ability to clone a forked version of PAWS but I suppose not entirely necessary at this point. Can leave until we overhaul PAWS, which needs to happen soonish.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly.


echo_bar
echo "
Executing ${POCS}/scripts/install/install-dependencies.sh, which will
install the tools needed to run POCS.
"

${POCS}/scripts/install/install-dependencies.sh
5 changes: 4 additions & 1 deletion scripts/install/install-dependencies.sh
Original file line number Diff line number Diff line change
Expand Up @@ -574,7 +574,10 @@ function maybe_install_conda() {
function get_installed_astrometry_version() {
local -r solve_field="${ASTROMETRY_DIR}/bin/solve-field"
if [[ -x "${solve_field}" ]] ; then
"${solve_field}" --help|(grep -E '^Revision [0-9.]+,' || /bin/true)|cut -c10-|cut -d, -f1
("${solve_field}" --help \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use $SOLVE_FIELD elsewhere to point to the solve-field executable. Here it is just checking a revision number, correct? (Also, I keep meaning to submit a PR to astrometry.net to add a simple --version flag. Very annoying it doesn't have one.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, just checking version number.

| (grep -E --only-matching '^Revision [0-9.]+' || /bin/true) \
| cut -c10- \
| cut -d, -f1)
fi
}

Expand Down