Skip to content

Latest commit

 

History

History
147 lines (113 loc) · 7.18 KB

README.md

File metadata and controls

147 lines (113 loc) · 7.18 KB

Running Buildkite agent

The buildkite agent is a small, reliable and cross-platform build runner that makes it easy to run automated builds on your own infrastructure. Its main responsibilities are polling buildkite.com for work, running build jobs, reporting back the status code and output log of the job, and uploading the job's artifacts. It is simple, lightweight hosted Buildkite CI/CD system which only requires to host agents in your Kubernetes cluster.

Introduction

This chart bootstraps a buildkite agent builder on a Kubernetes cluster using the Helm package manager. As it sets service account it can be used to build Docker images and deploy them using kubectl and helm clients in the same cluster where agents run, without any extra setup.

Add Buildkite Helm chart repository:

helm repo add buildkite https://buildkite.github.io/charts/
helm repo update

Installing the Chart

In order for the chart to configure the Buildkite Agent properly during the installation process, you must provide some minimal configuration which can't rely on defaults. This includes at least one element in the agent list token:

To install the chart with the release name bk-agent:

helm install --name bk-agent --namespace buildkite buildkite/agent \
    --set agent.token="BUILDKITE_AGENT_TOKEN"

To install the chart with the release name bk-agent and set Agent tags and git repo SSH key:

helm install --name bk-agent --namespace buildkite buildkite/agent \
  --set agent.token="$(cat buildkite.token)",agent.tags="role=production" \
  --set privateSshKey="$(cat buildkite.key)"  \
  --set registryCreds.gcrServiceAccountKey="$(cat gcr_service_account.key | base64)"

Alternatively, an external secret can be referenced for the agent token and agent SSH key:

helm install --name bk-agent --namespace buildkite buildkite/agent \
  --set agent.externalSecretName="buildkite-agent-secret",agent.tags="role=production" \
  --set agent.externalSecretTokenKey="agent-token",agent.externalSecretSSHKey="agent-ssh"

Note: if your pipeline uses docker for build images or run containers, you must set dind.enabled to true.

Where --set values contain:

agentToken: Buildkite token read from file
agentMeta: tagging agent with - role=production (to add multiple tags, you must separate them with an escaped comma, like this: role=production\,queue=kubernetes)
privateSshKey: private SSH key read from file
registryCreds.gcrServiceAccountKey: base64 encoded gcr_service_account.key json file

Tip: List all releases using helm list

Uninstalling the Chart

To uninstall/delete the bk-agent release:

helm delete bk-agent

The command removes all the Kubernetes components associated with the chart and deletes the release.

Configuration

The following table lists the configurable parameters of the buildkite chart and their default values.

Parameter Description Default
replicaCount Replicas count 1
image.repository Image repository buildkite/agent
image.tag Image tag ``
image.pullPolicy Image pull policy IfNotPresent
agent.externalSecretName Name of a Secret to load the agent token and agent private SSH key from. Takes precedence over .agent.token and .privateSshKey nil
agent.externalSecretTokenKey Name of the Key in the above secret where the agent token is located agent-token
agent.externalSecretSSHKey Name of the key in the above secret where the agent private SSH is located nil
agent.token Agent token Must be specified unless agent.externalSecretName is set
agent.tags Agent tags role=agent
enableHostDocker Mount docker socket true
podSecurityContext Pod security context to set {}
securityContext Container security context to set {}
extraEnv Agent extra env vars nil
privateSshKey Agent ssh key for git access. Also see .agent.externalSecretName nil
registryCreds.gcrServiceAccountKey GCP Service account json key nil
registryCreds.dockerConfig Private registry docker config.json nil
entrypointd Add files to /docker-entrypoint.d/ via a ConfigMap {}
serviceAccount.annotation Extra annotations for the generated ServiceAccount {}
rbac.create Whether to create RBAC resources to be used by the pod false
rbac.role.rules List of rules following the role specification See values.yaml
volumeMounts Extra volumeMounts configuration nil
volumes Extra volumes configuration nil
resources Liveness probe for docker socket {}
livenessProbe Pod resource requests & limits {}
nodeSelector Node labels for pod assignment {}
tolerations Node tolerations for pod assignment {}
affinity Node/pod affinity {}
podAnnotations Extra annotation to apply to the pod {}
podContainers Extra pod container or sidecar configuration nil
podInitContainers Extra pod init containers nil
dind.enabled Enable preconfigured Docker-in-Docker (DinD) pod configuration false
dind.image Image to use for Docker-in-Docker (DinD) pod container docker:19.03-dind
dind.port Port Docker-in-Docker (DinD) daemon listens on as REST request proxy 2375
dind.mtu The MTU used for Docker-inDocker (DinD) daemon. Must be lower than pod networking interface 1500
dind.resources Pod resource requests & limits for dind sidecar (if enabled) {}
dind.volumeMounts Extra volumeMounts configuration nil
terminationGracePeriodSeconds Duration in seconds the pod needs to terminate gracefully 30
nameOverride Provide a name to override the default $name template variable nil
fullnameOverride Provide a name to substitute for the full names of resources nil

Specify each parameter using the --set key=value[,key=value] argument to helm install.

Alternatively, a YAML file that specifies the values for the above parameters can be provided while installing the chart. For example:

helm install --name bk-agent --namespace buildkite buildkite/agent -f values.yaml

Tip: You can use the default values.yaml file

Buildkite pipeline examples

Check for examples of pipeline.yml and build/deploy scripts here.

Adding agent hooks to agent pods

Adding your own hooks (e.g. environment hooks) depends on whether you use DinD or not.

Without Docker-in-Docker

Without using DinD, you can follow the lower part of the guide here

With Docker-in-Docker

As the hooks directory is set to a shared dir, currently the best way to add your own hooks while using DinD consists of two steps.

  1. Follow the guide above for usage without DinD.
  2. Add an entrypoint script to your values.yml that copies the hooks from the image to the shared dir. E.g:
entrypointd: 
  01-copy-hooks: |
    #!/bin/sh
    set -euo pipefail
    mkdir -p /var/buildkite/hooks
    cp /buildkite/hooks/* /var/buildkite/hooks/.