Merge pull request #3 from gruntwork-io/custom-repo

Allow specifying a custom repo for gruntwork-install

README.md

@@ -1,9 +1,10 @@
 # Gruntwork Installer
 
-[Gruntwork Script Modules](https://github.com/gruntwork-io/script-modules) is a private repo that contains scripts and
-applications developed by [Gruntwork](http://www.gruntwork.io) for common infrastructure tasks such as setting up
-continuous integration, monitoring, log aggregation, and SSH access. This repo provides a script called
-`gruntwork-install` that makes it as easy to install a Script Module as using apt-get, brew, or yum.
+At [Gruntwork](http://www.gruntwork.io/), we've developed a number of scripts and apps, most of them in private repos
+such as [script-modules](https://github.com/gruntwork-io/script-modules), that perform common infrastructure tasks such
+as setting up continuous integration, monitoring, log aggregation, and SSH access. This repo provides a script called
+`gruntwork-install` that makes it as easy to install a Gruntwork script from a private repo as using apt-get, brew, or
+yum.
 
 For example, in your Packer and Docker templates, you can use `gruntwork-install` to install the [vault-ssh-helper
 module](https://github.com/gruntwork-io/script-modules/tree/master/modules/vault-ssh-helper) as follows:
@@ -15,7 +16,7 @@ gruntwork-install --module-name 'vault-ssh-helper' --tag '0.0.3'
 ## Installing gruntwork-install
 
 ```bash
-curl -Ls https://raw.githubusercontent.com/gruntwork-io/gruntwork-installer/master/bootstrap-gruntwork-installer.sh | bash /dev/stdin --version 0.0.6
+curl -Ls https://raw.githubusercontent.com/gruntwork-io/gruntwork-installer/master/bootstrap-gruntwork-installer.sh | bash /dev/stdin --version 0.0.7
 ```
 
 Notice the `--version` parameter at the end where you specify which version of `gruntwork-install` to install. See the
@@ -27,9 +28,9 @@ For paranoid security folks, see [is it safe to pipe URLs into bash?](#is-it-saf
 
 #### Authentication
 
-Since the [Script Modules](https://github.com/gruntwork-io/script-modules) repo is private, you must set your
-[GitHub access token](https://help.github.com/articles/creating-an-access-token-for-command-line-use/) as the
-environment variable `GITHUB_OAUTH_TOKEN` so `gruntwork-install` can use it to access the repo:
+To install scripts from private Gruntwork repos, you must create a [GitHub access
+token](https://help.github.com/articles/creating-an-access-token-for-command-line-use/) and set it as the environment
+variable `GITHUB_OAUTH_TOKEN` so `gruntwork-install` can use it to access the repo:
 
 ```bash
 export GITHUB_OAUTH_TOKEN="(your secret token)"
@@ -44,6 +45,7 @@ Option           | Required | Description
 `--module-name`  | Yes      | The name of the Script Module to install. Can be any folder within the `modules` directory of the [Script Modules Repo](https://github.com/gruntwork-io/script-modules).
 `--tag`          | Yes      | The version of the Script Module to install. Follows the syntax described at [Tag Constraint Expressions](https://github.com/gruntwork-io/fetch#tag-constraint-expressions).
 `--module-param` | No       | A key-value pair of the format `key=value` you wish to pass to the module as a parameter. May be used multiple times. See the documentation for each module to find out what parameters it accepts.
+`--repo`         | No       | `gruntwork-install` assumes that all scripts are in the [script-modules](https://github.com/gruntwork-io/script-modules) repo. To use a different repo, specify the `--repo` flag.
 `--help`         | No       | Show the help text and exit.
 
 #### Examples
@@ -63,6 +65,13 @@ parameters to it:
 gruntwork-install --module-name 'vault-ssh-helper' --tag '0.0.3' --module-param 'install-dir=/opt/vault-ssh-helper' --module-param 'owner=ubuntu'
 ```
 
+Install [ecs-scripts](https://github.com/gruntwork-io/module-ecs/tree/master/modules/scripts) from the
+[module-ecs repo](https://github.com/gruntwork-io/module-ecs):
+
+```bash
+gruntwork-install --module-name 'ecs-scripts' --tag '0.0.3' --repo "https://github.com/gruntwork-io/module-ecs"
+```
+
 And finally, to put all the pieces together, here is an example of a Packer template that installs `gruntwork-install`
 and then uses it to install several modules:
 
@@ -100,15 +109,16 @@ and then uses it to install several modules:
 
 `gruntwork-install` is a fairly simple script that does the following:
 
-1. Uses [fetch](https://github.com/gruntwork-io/fetch) to download the version of the module requested from
-   [script-modules](https://github.com/gruntwork-io/script-modules).
+1. Uses [fetch](https://github.com/gruntwork-io/fetch) to download the specified version of the module from the
+   `/modules` folder of either the [script-modules](https://github.com/gruntwork-io/script-modules) repo or the repo
+   specified via the `--repo` option.
 1. Runs the `install.sh` script inside that module.
 
 Future versions of `gruntwork-install` may do more (e.g. verify checksums, manage dependencies), but for now, that's
 all there is to it.
 
-That means that to add a new module to script-modules, all you have to do is include an `install.sh` script and it'll
-automatically be installable!
+That means that to add a new module, all you have to do is include an `install.sh` script and it'll automatically be
+installable!
 
 ## Running tests
 

gruntwork-install

@@ -1,6 +1,6 @@
 #!/bin/bash
 #
-# This script downloads a Gruntwork Script Module from https://github.com/gruntwork-io/script-modules using fetch
+# This script can be usedGruntwork Script Modules from https://github.com/gruntwork-io/script-modules using fetch
 # (https://github.com/gruntwork-io/fetch), and then runs it. The main motivation in writing it is to make installing
 # Gruntwork Script Modules feel as easy as installing a package using apt-get, brew, or yum.
 #
@@ -19,13 +19,14 @@ function print_usage {
   echo
   echo "Usage: gruntwork-install [OPTIONS]"
   echo
-  echo "Download a Gruntwork Script Module from https://github.com/gruntwork-io/script-modules and install it."
+  echo "Download a Gruntwork Script Module and install it."
   echo
   echo "Options:"
   echo
-  echo -e "  --module-name\t\tRequired. The name of the Script Module to install. Can be any folder within the $MODULES_DIR directory of $SCRIPT_MODULES_REPO."
-  echo -e "  --tag\t\t\tRequired. The version of the Script Module to install. Follows the syntax described at https://github.com/gruntwork-io/fetch#tag-constraint-expressions."
-  echo -e "  --branch\t\tOptional. Download the latest commit from this branch. This is an alternative to --tag for development purposes."
+  echo -e "  --module-name\t\tRequired. The name of the module to install. Can be any folder within the $MODULES_DIR directory of the --repo option."
+  echo -e "  --tag\t\t\tRequired. The version of the module to install. Follows the syntax described at https://github.com/gruntwork-io/fetch#tag-constraint-expressions."
+  echo -e "  --branch\t\tOptional. Download the latest commit from this branch in --repo. This is an alternative to --tag for development purposes."
+  echo -e "  --repo\t\tOptional. The repo to download modules from. If not specified, defaults to $SCRIPT_MODULES_REPO."
   echo -e "  --module-param\tOptional. A key-value pair of the format key=value you wish to pass to the module as a parameter. May be used multiple times."
   echo
   echo "Example:"
@@ -72,6 +73,7 @@ function fetch_script_module {
   local readonly branch="$3"
   local readonly github_token="$4"
   local readonly download_path="$5"
+  local readonly repo="$6"
 
   # We want to make sure that all folders down to $download_path/$module_name exists, but that $download_path/$module_name itself is empty.
   mkdir -p "$download_path/$module_name/"
@@ -80,8 +82,8 @@ function fetch_script_module {
   # Note that fetch can safely handle blank arguments for --tag or --branch
   # If both --tag and --branch are specified, --branch will be used
   # TODO: fetch should read GITHUB_OAUTH_TOKEN as an environment variable too
-  echo "Downloading module $module_name from $SCRIPT_MODULES_REPO"
-  fetch --repo="$SCRIPT_MODULES_REPO" --tag="$tag" --branch="$branch" --github-oauth-token="$github_token" "/modules/$module_name" "$download_path/$module_name" >/dev/null
+  echo "Downloading module $module_name from $repo"
+  fetch --repo="$repo" --tag="$tag" --branch="$branch" --github-oauth-token="$github_token" "/modules/$module_name" "$download_path/$module_name" >/dev/null
 }
 
 # Validate that at least one file was downloaded from the module; otherwise throw an error.
@@ -90,9 +92,10 @@ function validate_module {
   local readonly download_path="$2"
   local readonly tag="$3"
   local readonly branch="$4"
+  local reaodnly repo="$5"
 
   if [[ ! -e "$download_path/$module_name" ]]; then
-    echo "ERROR: No files were downloaded. Are you sure \"$module_name\" is a valid Script Module in $SCRIPT_MODULES_REPO (tag = $tag, branch = $branch)?"
+    echo "ERROR: No files were downloaded. Are you sure \"$module_name\" is a valid Script Module in $repo (tag = $tag, branch = $branch)?"
     exit 1
   fi
 }
@@ -119,6 +122,7 @@ function install_script_module {
   local tag=""
   local branch=""
   local module_name=""
+  local repo="$SCRIPT_MODULES_REPO"
   local module_params=()
 
   while [[ $# > 0 ]]; do
@@ -137,6 +141,10 @@ function install_script_module {
         module_name="$2"
         shift
         ;;
+      --repo)
+        repo="$2"
+        shift
+        ;;
       --module-param)
         local readonly param=$(convert_module_params_format "$2")
         module_params+=("$param")
@@ -158,11 +166,12 @@ function install_script_module {
 
   assert_is_installed fetch
   assert_not_empty "--module-name" "$module_name"
+  assert_not_empty "--repo" "$repo"
   assert_env_var_not_empty "GITHUB_OAUTH_TOKEN"
 
   echo "Installing $module_name..."
-  fetch_script_module "$module_name" "$tag" "$branch" "$GITHUB_OAUTH_TOKEN" "$MODULES_DOWNLOAD_DIR"
-  validate_module "$module_name" "$MODULES_DOWNLOAD_DIR" "$tag" "$branch"
+  fetch_script_module "$module_name" "$tag" "$branch" "$GITHUB_OAUTH_TOKEN" "$MODULES_DOWNLOAD_DIR" "$repo"
+  validate_module "$module_name" "$MODULES_DOWNLOAD_DIR" "$tag" "$branch" "$repo"
   run_module "$module_name" "${module_params[@]}"
   echo "Success!"
 }

test/integration-test.sh

@@ -12,3 +12,6 @@ echo "Using local copy of bootstrap installer to install local copy of gruntwork
 echo "Using gruntwork-install to install a few modules from script-modules"
 gruntwork-install --module-name "vault-ssh-helper" --tag "~>0.0.21"
 
+# TODO: Can't commit this until there is a tagged release of module-ecs due to a bug in fetch: https://github.com/gruntwork-io/fetch/issues/10
+# echo "Using gruntwork-install to install a module from the module-ecs repo"
+# gruntwork-install --module-name "ecs-scripts" --repo "https://github.com/gruntwork-io/module-ecs" --tag "~>0.0.1"