Control Tower aims to provide a better abstraction over Concourse pipelines so that it's easier to manage and fly. For now, it's used internally in PKS networking team. Thus some commands are only available for a few built-in type of pipelines.
# clone control-tower repo
git clone https://github.com/maplain/control-tower
# build ct binary
make build-ctct will be built under bin. You may want to optionally move it under your $PATH.
Or you can simply do a:
make buildProfile is an encrypted key-value yaml file managed by ct. They are the source of truth when it comes to configure a templated pipeline. You can create a profile from command line:
➜ control-tower git:(master) ✗ ct p c -n test --vars a=b,c=d
profile test is created successfully
➜ control-tower git:(master) ✗ ct p v -n test
a: b
c: dOr from a key-value yaml file:
➜ control-tower git:(master) ✗ ct p c -n test-2 --var-file <(ct p v -n test)
profile test-2 is created successfully
➜ control-tower git:(master) ✗ ct p v -n test-2
a: b
c: dIt's ok to create profile with some fields templated by (( )). For example:
➜ control-tower git:(master) ✗ ct p c -n test-3 --var-file template.yml --template
profile test-3 is created successfully
➜ control-tower git:(master) ✗ ct p v -n test-3
name: ((name))
➜ control-tower git:(master) ✗ cat template.yml
name: ((name))Note: you need to use --template flag to inform ct that it's a templated profile.
When a template profile is used to create a pipeline, you'll be asked for the values by ct and one corresponding concrete profile will be optionally created.
Generally speaking, following practices are recommended:
- create normal profiles for static shared secrets
- create template profiles for dynamic configurations that vary from feature to feature, eg: branches, pool-name etc
This profiles are encrypted using a default key which might not suite your use case. To encrypt it using another key:
➜ control-tower git:(master) ✗ ct s g > secret
➜ control-tower git:(master) ✗ ct p c -n test-3 --vars a=b,c=d -k $(<secret)
profile test-3 is created successfully
➜ control-tower git:(master) ✗ ct p v -n test-3
2019/01/28 14:31:17 error: yaml: invalid leading UTF-8 octet
➜ control-tower git:(master) ✗ ct p v -n test-3 -k $(<secret)
a: b
c: dTo setup kubo-related profiles automatically:
make set-kuboTo update kubo related profiles forcefully:
make set-kubo-forceKubo related profiles will be setup automatically for you. To check them out:
➜ control-tower git:(master) ✗ ct p l -t kubo
+---------------------------+------+------------+
| PROFILE NAME | TAGS | ISTEMPLATE |
+---------------------------+------+------------+
| common-secrets | kubo | false |
| deploy-kubo | kubo | false |
| kubo-fangyuanl | kubo | false |
| pks-nsx-t-release-secrets | kubo | false |
+---------------------------+------+------------+To setup raas-related profiles automatically:
make set-raasTo update kubo related profiles forcefully:
make set-raas-forceRAAS related profiles will be setup automatically for you. To check them out:
ct p l -t releng
+----------------------------+--------+------------+
| PROFILE NAME | TAGS | ISTEMPLATE |
+----------------------------+--------+------------+
| nsx-t-secrets | releng | false |
| pks-releng-write-locks-bot | releng | false |
| raas-credentials | releng | false |
| vsphere-nsx-variables | releng | false |
+----------------------------+--------+------------+
ct p l -t releng-template
+----------------+-----------------+------------+
| PROFILE NAME | TAGS | ISTEMPLATE |
+----------------+-----------------+------------+
| raas-variables | releng-template | true |
+----------------+-----------------+------------+You can group profiles by applying tags.
➜ control-tower git:(master) ✗ ct p t -n test -t test,example
tags of profile test are updated to example,testTo view profiles by tags:
➜ control-tower git:(master) ✗ ct p l -t example
+--------------+--------------+------------+
| PROFILE NAME | TAGS | ISTEMPLATE |
+--------------+--------------+------------+
| test | example,test | false |
+--------------+--------------+------------+
➜ control-tower git:(master) ✗ ct p l -t test
+--------------+--------------+------------+
| PROFILE NAME | TAGS | ISTEMPLATE |
+--------------+--------------+------------+
| test | example,test | false |
| test-2 | test | false |
+--------------+--------------+------------+Tags are useful when:
- you want to deploy a pipeline with multiple profiles
- you want to separate similar configurations based on feature
context is an alias for concourse target, team and pipeline. ct provides a rich set of utilities that are associated with one context.
Let's say you control following pipelines on concourse:
➜ control-tower git:(master) ✗ fly -t npks ps | grep kubo
fangyuanl-kubo-2 no noCreate a context for pipeline fangyuanl-kubo-2 by:
➜ control-tower git:(master) ✗ ct c c -n test --pipeline-name fangyuanl-kubo-2 -t npks
context test is created
use ct c v -n test to check detailsSwitch current context to it:
ct c set -n testSee the pipeline in browser:
ct oCheck out other contexts in browser:
ct o [another context name]Check out jobs in current context:
ct f j
+------+---------------------------+
| ID | NAME |
+------+---------------------------+
| 9049 | claim-lock-kubo |
| 9050 | run-precheck-release-lock |
| 9051 | run-precheck |
| 9052 | deploy-kubo-release-lock |
| 9053 | claim-lock-for-outputs |
| 9054 | outputs |
| 9055 | outputs-release-lock |
| 9056 | deploy-kubo |
| 9057 | claim-lock-for-deletion |
| 9058 | delete-kubo |
| 9059 | delete-kubo-release-lock |
+------+---------------------------+Check out pipeline status in current context:
➜ control-tower git:(master) ✗ ct f s
+--------+------+------+-----------+-----------------+
| ID | TEAM | NAME | STATUS | JOB |
+--------+------+------+-----------+-----------------+
| 533232 | nsxt | 1 | succeeded | claim-lock-kubo |
| 539343 | nsxt | 2 | succeeded | claim-lock-kubo |
| 562675 | nsxt | 3 | succeeded | claim-lock-kubo |
| 562858 | nsxt | 4 | succeeded | claim-lock-kubo |
+--------+------+------+-----------+-----------------+
+--------+------+------+-----------+---------------------------+
| ID | TEAM | NAME | STATUS | JOB |
+--------+------+------+-----------+---------------------------+
| 539080 | nsxt | 1 | succeeded | run-precheck-release-lock |
+--------+------+------+-----------+---------------------------+
+--------+------+------+-----------+--------------+
| ID | TEAM | NAME | STATUS | JOB |
+--------+------+------+-----------+--------------+
| 573958 | nsxt | 23 | succeeded | run-precheck |
| 574034 | nsxt | 24 | succeeded | run-precheck |
| 574161 | nsxt | 25 | failed | run-precheck |
| 574171 | nsxt | 26 | failed | run-precheck |
| 574604 | nsxt | 27 | failed | run-precheck |
+--------+------+------+-----------+--------------+
+--------+------+------+-----------+--------------------------+
| ID | TEAM | NAME | STATUS | JOB |
+--------+------+------+-----------+--------------------------+
| 553764 | nsxt | 1 | succeeded | deploy-kubo-release-lock |
| 562845 | nsxt | 2 | succeeded | deploy-kubo-release-lock |
| 563411 | nsxt | 3 | succeeded | deploy-kubo-release-lock |
+--------+------+------+-----------+--------------------------+
+--------+------+------+-----------+-------------+
| ID | TEAM | NAME | STATUS | JOB |
+--------+------+------+-----------+-------------+
| 563232 | nsxt | 13 | succeeded | deploy-kubo |
| 567257 | nsxt | 14 | succeeded | deploy-kubo |
| 567721 | nsxt | 15 | failed | deploy-kubo |
| 573968 | nsxt | 16 | errored | deploy-kubo |
| 574036 | nsxt | 17 | errored | deploy-kubo |
+--------+------+------+-----------+-------------+Check out logs for a specific build:
➜ control-tower git:(master) ✗ ct f logs -i 533232
acquiring lock on: fangyuanl
Cloning into '/tmp/build/get'...
5478c9162c fangyuanl-kubo-2/claim-lock-kubo build 1 claiming: nsx2You can also provide customized outputs for built-in type pipelines:
➜ control-tower git:(master) ✗ ct c set kubo
current context is set to kubo
➜ control-tower git:(master) ✗ ct c v
target: npks
team: nsxt
pipeline: fangyuanl-kubo-2
type: kubo
inuse: true
➜ control-tower git:(master) ✗ ct f o
build-id: "567257"
kubeconfig-path: fangyuanl/kubeconfig-0.270.0.tgz
lock-name: nsx3The outputs for type kubo pipeline is defined to be a key-value yaml file so that it can be directly piped into another command's input. See examples below.
To deploy a pipeline, basically there are three kinds of information needed:
- pipeline yaml file which can be specified by either
--templateor--template-type.--templateaccepts a filepath, whereas--template-typeaccepts a list of reserved static types. See more details below; 2.(optional) fly target and pipeline name. If not provided, values in current context are used; 3.(optional) profiles that used to populate pipeline template yaml file. Three flags are available:--profile-name,--profile-pathand--profile-tag.
To login the context:
ct f loginDeploy the pipeline:
pushd $GOPATH/src/gitlab.eng.vmware.com/PKS/pks-nsx-t-release
# ci/pks-nsx-t-release.yml is templated using erb semantics
ct d --profile-tag kubo -m <(erb ci/pks-nsx-t-release.yml) -n [pipeline name]
popdDeploy kubo pipeline
ct d --profile-tag=kubo --template-type kubo -n kubo --target npksIf your current context is on a kubo type pipeline and it has run through successfully, then you can deploy a nsx-acceptance-tests pipeline with artifacts from that kubo pipeline.
➜ control-tower git:(master) ✗ ct c c -n ntests --pipeline-name nsx-acceptance-tests --target npks
context ntests is created
use ct c v -n ntests to check details
➜ control-tower git:(master) ✗ ct c set ntests
current context is set to ntests
➜ control-tower git:(master) ✗ ct d --profile-tag=kubo --template-type nsx-acceptance-tests --profile-path=<(ct f outputs) In above example, we use the outputs from a kubo type pipeline to configure a nsx-acceptance-tests type pipeline so that the latter will use the exact versions of kubeconfig and lock generated by the first one.
To check all jobs defined for nsx-acceptance-tests type of pipeline:
➜ control-tower git:(master) ✗ ct f j
+------+------------------------------------+
| ID | NAME |
+------+------------------------------------+
| 9427 | claim-lock |
| 9428 | run-release-tests-release-lock |
| 9429 | run-release-tests |
| 9430 | run-conformance-tests-release-lock |
| 9431 | run-conformance-tests-delete-kubo |
| 9432 | run-conformance-tests |
| 9433 | run-release-tests-delete-kubo |
+------+------------------------------------+To unpause the pipeline:
➜ control-tower git:(master) ✗ ct f p -u
pipeline fangyuanl-kubo-2 is unpaused.To trigger the first job:
➜ control-tower git:(master) ✗ ct f trigger -j claim-lock
started nsx-acceptance-tests/claim-lock #1To pause the pipeline:
➜ control-tower git:(master) ✗ ct f p
pipeline nsx-acceptance-tests is paused.To get pipeline configuration yaml:
ct f cTo get job configuration:
ct f c -j [job-name]To delete the pipeline:
➜ control-tower git:(master) ✗ ct f d
pipeline nsx-acceptance-tests is deleted. now ntests is a dangling context%
➜ control-tower git:(master) ✗ ct c d -n ntests
context ntests is deleted successfully
This pipeline will build a PKS tile based on your configuration
ct d --profile-tag=releng --profile-tag=releng-template --template-type build-tile -n fangyuanl-kubo-2 --target npksThis pipeline will install a PKS tile to the specified Nimbus testbed
ct d --profile-tag=releng --profile-tag=releng-template --template-type install-tile -n fangyuanl-kubo-2 --target npksThis pipeline will deploy a k8s cluster in given PKS deployment and run acceptance-tests
ct d --profile-path <(echo "releng-tests-lock-name: nsx1") -p common-secrets --profile-tag releng --profile-tag nodes_dns --template-type releng-acceptance-testscheck out doc