Update workstation scripts to be more broadly useful (#64)

* Import print_env_info script from capi-ci

- It is no longer needed by capi-ci, since removing Slack
  integration from the bosh-lite pool.
- It will be less lonely here, since this is where the rest of the
  bosh-lite pool scripts live.

* Simplify claim_bosh_lite function

- Remove dependency on git-together
- Remove dependency on direnv
- Remove dependency on lastpass CLI
- Remove Pivotal Tracker integration
- No longer prompts for Pivotal Tracker story id
- Adds Jira integration (not really; just kidding)
- No longer changes your working directory
- Fix hostname for zsh (no $HOSTNAME environment variable)

* Tidy up target_bosh function

- Mostly just adding some stdout lines to clarify what the command is
  doing

* Tidy unclaim_bosh_lite function

- Remove dependency on git-together
- Unset more environment variables
- Fix hostname for zsh (no $HOSTNAME environment variable)

* Use git commit instead of git ci

- cause not everyone uses aliases

* Simplify print_env_info

- Remove story column
- Remove staleness. Bosh lites are eternally fresh!
- Gracefully handle case when no environment are claimed. Previously
  this would would trigger an "ambiguous ref" error from git, due to
  weirdness when globbing empty directories.

* Delete can-i-push

- No longer used
- Depends the dead canibump app (RIP)

* Unclaim also unsets CREDHUB_CA_CERT env var

* Don't log in to CredHub

- Per CredHub CLI output:
  Login ignored as the CREDHUB_* environment variables are present.
  You do NOT need to run this "login" command in order to run other
  "credhub" commands when you set the CREDHUB_* environment variables.

  The command only validates your credentials. For backwards compatibility
  reasons, it (in this particular use case) does not error out. You should
  not use it with the environment variables set.
  The "credhub login" command is useful in the case you do not have the
  environment variables set and you pass your credentials to "credhub login"
  through command line arguments.

- Add some guards for commands that require a targeted bosh. For
  instance, if they require credhub env vars to be set =)

* Remove some dubious commands

- Remove the `set_capi_*_pipeline` scripts. They didn't do that much and
  have some baked-in assumptions (e.g. having a fly target named
  "capi"). They didn't seem worth parameterizing to make them more
  portable, but we can do it if people care enough.
- Remove `gcs_to_claimed` script. I'm not really sure what this did, but
  it doesn't seem to be widely used, and it's not very portable. It also
  duplicates some of the behavior in `claim_bosh_lite`, which I'd rather
  not have to maintain independently.

* Add shebang to bin/fixcommitter

* Clean up deploy scripts

- Add additional stoutput to help explain what is going on
- Add guards to make sure there is an environment targeted
- Reincarnate `use-created-capi.yml` ops file, formerly of capi-ci, but
  now gone. This ops file is used to build and deploy a capi release
  from your local filesystem, thanks to bosh handling local file urls.
- Loosen assumptions about local directory structure. The commands now
  let you provide environment variables to override the default
  assumption that you have a relevant repos in a `~/workspace`
  directory.
- Miscellaneous other small style changes, because I can

* Additional minor tidying

- Soften more directory structure opinions
- Change some output formatting to be more consistent with other
  commands
- Remove indenting output by piping to sed, since it doesn't work for
  non-stdout terminal output (e.g. bosh CLI output is often not
  pipe-able)

* Remove bin/story

- Most CAPI work is tracked via GitHub issues now; Tracker is not used
  by most/all CAPI contributors.
- Story is no longer needed for claiming bosh lites (via prior commit)

* Tidy seed_users

- Take org/space name as arguments, akin to bootstrap_cf
- Change default names to "org" and "space", instead of "test", so that
  they match bootstrap_cf

* Remove commit_with_shortlog

- Does not work; It attempted to call bin/staged_shortlog, which doesn't
  exist.
- The actual staged_shortlog is in capi-release/scripts

* Fix run_bridge_tests

- The script assumed that it was inside capi-release's scripts
  directory. It was not.

* Remove watch_cc_units

- Consistent with removing other minor concourse-related scripts (see
  previous commit)
- Provides dubious value vs using fly yourself, and it has a hard-coded
  target name

* Remove bin/load-key

- Part of larger effort to remove lastpass dependency
- Most CAPI contributors don't use lastpass anymore

* Remove bin/pivotal_login

* Update capidoc with recent changes

- Add some missing scripts

* Small UI improvements to scripts

* Reorganize README

- Move heavy-weight installation instructions to the bottom, since they
  are not in common use
- Emphasize the light-weight installation instructions (adding scripts
  to path and sourcing functions)
- Add section espousing the wonders of the bosh lite pool
- Remove some "Pivotal" references

* Purge lib directory

- Remove most opinions about development environment (vim, ruby dev,
  git, GOPATH, etc)
- Limit to dev-machine-agnostic functions for CAPI development

assets/capidoc.yml

@@ -26,10 +26,6 @@ check_certificate_expirations:
   description: Reads YAML files, checks for expired (or premature) certs. Useful for pipeline troubleshooting
   location: capi-workspace/bin
   group: testing
-commit_with_shortlog:
-  description: Git commit with a staged_shortlog
-  location: capi-workspace/bin
-  group: git
 compare_db_rows:
   description: Compares counts of two db's rows to validate backup & restore
   location: capi-workspace/bin
@@ -50,10 +46,14 @@ fixcommitter:
   description: Cleans up committer/author after a rebase
   location: capi-workspace/bin
   group: git
-gcs_to_claimed:
-  description: Convert bosh-lite environment files stored in gcs to a claimed bosh-lite
+generate_integration_config:
+  description: Create an integration_config.json for running CATS against current bosh target.
   location: capi-workspace/bin
-  group: bosh-lite-pool
+  group: testing
+git-open:
+  description: Open a git repo in your browser, courtesy of Paul Irish.
+  location: capi-workspace/bin
+  group: git
 mysql_bosh_lite:
   description: Connect to current bosh target's MySQL DB
   location: capi-workspace/bin
@@ -147,14 +147,6 @@ gi:
   description: gem install
   location: capi-workspace/custom-bash-it-plugins
   group: ruby-dev
-set_capi_pipeline:
-  description: Set the capi pipeline to your local config file (fly set-pipeline)
-  location: capi-workspace/bin
-  group: ci
-watch_cc_units:
-  description: Watch CAPI CI unit tests. For a specific run, -b build_number (fly watch)
-  location: capi-workspace/bin
-  group: ci
 cfu:
   description: Alias for seed_users
   location: capi-workspace/bin
@@ -200,10 +192,6 @@ attach_debugger:
   description: Attaches the rubymine debugger to your bosh lite
   location: capi-release/src/cloud_controller_ng/scripts/short-circuit-cc
   group: bosh-lite-pool
-load-key:
-  description: Loads the an ssh key in the currently logged in lastpass called "github-private-key".  Run eval $(ssh-agent) if it fails
-  location: capi-workspace/bin
-  group: auth
 capi-workspaces:
   description: List and connect to existing K8s CAPI workspaces
   location: capi-workspace/bin

assets/use-created-capi.yml

@@ -0,0 +1,7 @@
+---
+- type: replace
+  path: /releases/name=capi
+  value:
+    name: capi
+    version: create
+    url: file://((capi_release_dir))

bin/cf_admin_password

@@ -4,24 +4,13 @@ set -e
 
 get_cf_admin_password() {
   echo "Getting CF admin password from credhub... "
-
-  # Newer bosh-lites use $CREDHUB_SECRET instead of $CREDHUB_PASSWORD
-  # The credhub cli is aware of $CREDHUB_SECRET so it does not need it passed in
-  # If there are no bosh-lites around anymore that were created before 01/25/2018
-  # you can delete the else condition
-  set +u # $CREDHUB_PASSWORD may be unbound
-  if [[ -z $CREDHUB_PASSWORD ]]; then
-    credhub login --skip-tls-validation
-  else
-    credhub login -s "$CREDHUB_SERVER" -u "$CREDHUB_USERNAME" -p "$CREDHUB_PASSWORD" --skip-tls-validation
-  fi
   set -u
   cf_admin_password=$(credhub get --name '/bosh-lite/cf/cf_admin_password' --output-json | jq -r '.value')
 }
 
 main() {
   if [ -z "$BOSH_ENVIRONMENT" ]; then
-    echo "No bosh targeted. Use \"target_bosh\" before getting credhub password"
+    echo "No bosh targeted. Use \"target_bosh\" before getting admin password."
   else
     get_cf_admin_password
     echo "CF Admin Password: ${cf_admin_password}"

bin/create_and_deploy

@@ -1,12 +1,18 @@
 #!/bin/bash
-set -exu
+set -eu
+
+cfd_dir="${CF_DEPLOYMENT_DIR:-$HOME/workspace/cf-deployment}"
+capi_ci_dir="${CAPI_CI_DIR:-$HOME/workspace/capi-ci}"
 
 create_and_upload
 
-# deploy to current bosh env
-bosh deploy ~/workspace/cf-deployment/cf-deployment.yml \
+printf "\nDeploying uploaded release...\n"
+echo "bosh deploy cf-deployment.yml -o..."
+
+bosh deploy "${cfd_dir}/cf-deployment.yml" \
   -v system_domain=$BOSH_LITE_DOMAIN \
-  -o ~/workspace/capi-ci/cf-deployment-operations/skip-cert-verify.yml \
-  -o ~/workspace/cf-deployment/operations/bosh-lite.yml \
-  -o ~/workspace/cf-deployment/operations/use-compiled-releases.yml \
-  -o ~/workspace/capi-ci/cf-deployment-operations/use-latest-capi.yml $@
+  -o "${capi_ci_dir}/cf-deployment-operations/skip-cert-verify.yml" \
+  -o "${cfd_dir}/operations/bosh-lite.yml" \
+  -o "${cfd_dir}/operations/use-compiled-releases.yml" \
+  -o "${capi_ci_dir}/cf-deployment-operations/use-latest-capi.yml" \
+  $@

bin/create_and_upload

@@ -1,9 +1,28 @@
 #!/bin/bash
-set -eu
+set -e
 
-# create and upload a release
-pushd ~/workspace/capi-release
+if [ -z "$BOSH_ENVIRONMENT" ]; then
+  echo "No bosh targeted. Use \"target_bosh\" before creating and uploading a release."
+  exit 1
+fi
+echo "Uploading new release to $BOSH_ENVIRONMENT."
+
+set -u
+
+capi_release_dir="${CAPI_RELEASE_DIR:-$HOME/workspace/capi-release}"
+
+pushd "${capi_release_dir}" > /dev/null
+
+  printf "\nSyncing bosh blobs...\n"
+  echo "bosh sync-blobs"
   bosh sync-blobs
+
+  printf "\nCreating bosh release...\n"
+  echo "create-release --force --name capi"
   bosh create-release --force --name capi
+
+  printf "\nUploading release to bosh director...\n"
+  echo "bosh upload-release --rebase"
   bosh upload-release --rebase
-popd
+
+popd > /dev/null

bin/deploy

@@ -1,4 +1,4 @@
 #!/bin/bash
-set -exu
+set -e
 
 create_and_deploy $@ -n

bin/deploy_only_new_capi

@@ -1,13 +1,19 @@
 #!/usr/bin/env bash
 
-# you provide your own capi-release directory via the first positional argument
-if [[ -n "$1" ]] ; then
-  CAPI_RELEASE_DIR="$1" ; shift
-else
-  CAPI_RELEASE_DIR="$HOME/workspace/capi-release"
+set -e
+
+capi_release_dir="${CAPI_RELEASE_DIR:-$HOME/workspace/capi-release}"
+__dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+if [ -z "$BOSH_ENVIRONMENT" ]; then
+  echo "No bosh targeted. Use \"target_bosh\" before deploying new CAPI."
+  exit 1
 fi
+echo "Building a new CAPI from local filesystem and deploying to $BOSH_ENVIRONMENT."
+
+echo "bosh deploy <(bosh manifest) -o ...use-created-capi.yml ..."
 
 bosh -n deploy <(bosh manifest) \
-  -o ~/workspace/capi-ci/cf-deployment-operations/use-created-capi.yml \
-  -v capi_release_dir="${CAPI_RELEASE_DIR}" \
-  "$@"
+  -o "${__dir}/../assets/use-created-capi.yml" \
+  -v capi_release_dir="${capi_release_dir}" \
+  $@

bin/fixcommitter

@@ -1,3 +1,5 @@
+#!/usr/bin/env bash
+
 do_fixcommitter () 
 { 
     ( committer=$(git show HEAD -s --pretty="tformat:%b" | grep Signed-off | cut -d: -f2);

bin/mysql_bosh_lite

@@ -5,13 +5,6 @@ set -e
 set -o pipefail
 
 get_database_password(){
-	set +u # $CREDHUB_PASSWORD may be unbound
-  if [[ -z $CREDHUB_PASSWORD ]]; then
-    credhub login --skip-tls-validation
-  else
-    credhub login -s "$CREDHUB_SERVER" -u "$CREDHUB_USERNAME" -p "$CREDHUB_PASSWORD" --skip-tls-validation
-  fi
-  set -u
 	cc_database_password=$(credhub get -n '/bosh-lite/cf/cc_database_password' --output-json | jq -r '.value')
 }
 
@@ -45,6 +38,13 @@ kill_tree() {
 	kill $pid
 }
 
+set +u
+if [ -z "$BOSH_ENVIRONMENT" ]; then
+  echo "No bosh targeted. Use \"target_bosh\" before connecting to MySQL."
+  exit 1
+fi
+set -u
+
 {
 	get_database_password &&
 	ssh_tunnel &&