Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add kustomize argument for kubectl #1169

Merged
merged 2 commits into from
Sep 8, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions examples/kubernetes-kustomize-example/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Kubernetes Kustomize Example

This folder contains a minimal Kubernetes resource config file to demonstrate how you can use Terratest to write
automated tests for Kubernetes.

This resource file deploys an nginx container as a single pod deployment with a node port service attached to it.

See the corresponding terratest code for an example of how to test this resource config:
- [kubernetes_kustomize_example_test.go](../../test/kubernetes_kustomize_example_test.go)


## Deploying the Kubernetes resource

1. Setup a Kubernetes cluster. We recommend using a local version:
- [minikube](https://github.com/kubernetes/minikube)
- [Kubernetes on Docker For Mac](https://docs.docker.com/docker-for-mac/kubernetes/)
- [Kubernetes on Docker For Windows](https://docs.docker.com/docker-for-windows/kubernetes/)

1. Install and setup [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) to talk to the deployed
Kubernetes cluster.
1. Run `kubectl apply -k .`


## Running automated tests against this Kubernetes deployment

1. Setup a Kubernetes cluster. We recommend using a local version:
- [minikube](https://github.com/kubernetes/minikube)
- [Kubernetes on Docker For Mac](https://docs.docker.com/docker-for-mac/kubernetes/)
- [Kubernetes on Docker For Windows](https://docs.docker.com/docker-for-windows/kubernetes/)

1. Install and setup [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) to talk to the deployed
Kubernetes cluster.
1. Install [Golang](https://golang.org/) and make sure this code is checked out into your `GOPATH`.
1. `cd test`
1. `dep ensure`
1. `go test -v -tags kubernetes -run TestKubernetesKustomizeExample`
20 changes: 20 additions & 0 deletions examples/kubernetes-kustomize-example/deployment.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
selector:
matchLabels:
app: nginx
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.15.7
ports:
- containerPort: 80
3 changes: 3 additions & 0 deletions examples/kubernetes-kustomize-example/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
resources:
- ./deployment.yaml
- ./service.yaml
13 changes: 13 additions & 0 deletions examples/kubernetes-kustomize-example/service.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
kind: Service
apiVersion: v1
metadata:
name: nginx-service
spec:
selector:
app: nginx
ports:
- protocol: TCP
targetPort: 80
port: 1080
type: NodePort
22 changes: 22 additions & 0 deletions modules/k8s/kubectl.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,17 @@ func KubectlDeleteE(t testing.TestingT, options *KubectlOptions, configPath stri
return RunKubectlE(t, options, "delete", "-f", configPath)
}

// KubectlDeleteFromKustomize will take in a kustomization directory path and delete it from the cluster targeted by KubectlOptions. If there are any
// errors, fail the test immediately.
func KubectlDeleteFromKustomize(t testing.TestingT, options *KubectlOptions, configPath string) {
require.NoError(t, KubectlDeleteFromKustomizeE(t, options, configPath))
}

// KubectlDeleteFromKustomizeE will take in a kustomization directory path and delete it from the cluster targeted by KubectlOptions.
func KubectlDeleteFromKustomizeE(t testing.TestingT, options *KubectlOptions, configPath string) error {
return RunKubectlE(t, options, "delete", "-k", configPath)
}

// KubectlDeleteFromString will take in a kubernetes resource config as a string and delete it on the cluster specified
// by the provided kubectl options.
func KubectlDeleteFromString(t testing.TestingT, options *KubectlOptions, configData string) {
Expand Down Expand Up @@ -83,6 +94,17 @@ func KubectlApplyE(t testing.TestingT, options *KubectlOptions, configPath strin
return RunKubectlE(t, options, "apply", "-f", configPath)
}

// KubectlApplyFromKustomize will take in a kustomization directory path and apply it to the cluster targeted by KubectlOptions. If there are any
// errors, fail the test immediately.
func KubectlApplyFromKustomize(t testing.TestingT, options *KubectlOptions, configPath string) {
require.NoError(t, KubectlApplyFromKustomizeE(t, options, configPath))
}

// KubectlApplyFromKustomizeE will take in a kustomization directory path and delete it from the cluster targeted by KubectlOptions.
func KubectlApplyFromKustomizeE(t testing.TestingT, options *KubectlOptions, configPath string) error {
return RunKubectlE(t, options, "apply", "-k", configPath)
}

// KubectlApplyFromString will take in a kubernetes resource config as a string and apply it on the cluster specified
// by the provided kubectl options.
func KubectlApplyFromString(t testing.TestingT, options *KubectlOptions, configData string) {
Expand Down
60 changes: 60 additions & 0 deletions test/kubernetes_kustomize_example_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
//go:build kubeall || kubernetes
// +build kubeall kubernetes

// NOTE: we have build tags to differentiate kubernetes tests from non-kubernetes tests. This is done because minikube
// is heavy and can interfere with docker related tests in terratest. Specifically, many of the tests start to fail with
// `connection refused` errors from `minikube`. To avoid overloading the system, we run the kubernetes tests and helm
// tests separately from the others. This may not be necessary if you have a sufficiently powerful machine. We
// recommend at least 4 cores and 16GB of RAM if you want to run all the tests together.

package test

import (
"fmt"
"path/filepath"
"strings"
"testing"

"github.com/stretchr/testify/require"

"github.com/gruntwork-io/terratest/modules/k8s"
"github.com/gruntwork-io/terratest/modules/random"
)

// An example of how to test the Kubernetes resource config in examples/kubernetes-kustomize-example using Terratest.
func TestKubernetesKustomizeExample(t *testing.T) {
t.Parallel()

// website::tag::1::Path to the Kubernetes resource config we will test
kubeResourcePath, err := filepath.Abs("../examples/kubernetes-kustomize-example/")
require.NoError(t, err)

// To ensure we can reuse the resource config on the same cluster to test different scenarios, we setup a unique
// namespace for the resources for this test.
// Note that namespaces must be lowercase.
namespaceName := fmt.Sprintf("kubernetes-kustomize-example-%s", strings.ToLower(random.UniqueId()))

// website::tag::2::Setup the kubectl config and context.
// Here we choose to use the defaults, which is:
// - HOME/.kube/config for the kubectl config file
// - Current context of the kubectl config file
// - Random namespace
options := k8s.NewKubectlOptions("", "", namespaceName)

k8s.CreateNamespace(t, options, namespaceName)
// website::tag::5::Make sure to delete the namespace at the end of the test
defer k8s.DeleteNamespace(t, options, namespaceName)

// website::tag::6::At the end of the test, run `kubectl delete -f RESOURCE_CONFIG` to clean up any resources that were created.
defer k8s.KubectlDeleteFromKustomize(t, options, kubeResourcePath)

// website::tag::3::Apply kubectl with 'kubectl apply -f RESOURCE_CONFIG' command.
// This will run `kubectl apply -f RESOURCE_CONFIG` and fail the test if there are any errors
k8s.KubectlApplyFromKustomize(t, options, kubeResourcePath)

// website::tag::4::Check if NGINX service was deployed successfully.
// This will get the service resource and verify that it exists and was retrieved successfully. This function will
// fail the test if the there is an error retrieving the service resource from Kubernetes.
service := k8s.GetService(t, options, "nginx-service")
require.Equal(t, service.Name, "nginx-service")
}