Use docker testcontainer able to launch: local docker registry, etc for testing cases

[cmoulliard]

Proposition

I would like to propose to use this project https://golang.testcontainers.org/quickstart/ for our integration tests when it is needed to launch some containers. Example: To execute this image_test.go test, a local registry container is needed

We can easily use testcontainer to bootstrap a container as you can see hereafter:

package gitserver

import (
	"context"
	"github.com/cnoe-io/idpbuilder/api/v1alpha1"
	"github.com/cnoe-io/idpbuilder/pkg/apps"
	"github.com/cnoe-io/idpbuilder/pkg/docker"
	"github.com/docker/docker/api/types"
	"github.com/docker/docker/api/types/container"
	"github.com/docker/go-connections/nat"
	"github.com/testcontainers/testcontainers-go"
	"github.com/testcontainers/testcontainers-go/wait"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	controllerruntime "sigs.k8s.io/controller-runtime"
	logf "sigs.k8s.io/controller-runtime/pkg/log"
	"sigs.k8s.io/controller-runtime/pkg/log/zap"
	"strings"
	"testing"
)

func TestReconcileGitServerImage(t *testing.T) {
	logf.SetLogger(zap.New(zap.UseDevMode(true)))

	ctx := context.Background()
	req := testcontainers.ContainerRequest{
		Image:      "registry:2",
		WaitingFor: wait.ForListeningPort("5000/tcp"),
		HostConfigModifier: func(hc *container.HostConfig) {
			hc.PortBindings = nat.PortMap{
				"5001/tcp": []nat.PortBinding{
					{
						HostIP:   "localhost",
						HostPort: "5001",
					},
				},
			}
		},
	}
	registryC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
		ContainerRequest: req,
		Started:          true,
	})
	if err != nil {
		panic(err)
	}
	defer func() {
		if err := registryC.Terminate(ctx); err != nil {
			panic(err)
		}
	}()

there is even a trick to bind the needed port (e.g 5001) to the port used by the docker registry: 5000

Test output will be now

/opt/homebrew/opt/go/libexec/bin/go tool test2json -t /Users/cmoullia/Library/Caches/JetBrains/IntelliJIdea2023.2/tmp/GoLand/___image_test_go.test -test.v -test.paniconexit0 -test.run ^\QTestReconcileGitServerImage\E$
=== RUN   TestReconcileGitServerImage
2023/10/19 11:44:05 github.com/testcontainers/testcontainers-go - Connected to docker: 
  Server Version: 24.0.6
  API Version: 1.43
  Operating System: Docker Desktop
  Total Memory: 9950 MB
  Resolved Docker Host: unix:///var/run/docker.sock
  Resolved Docker Socket Path: /var/run/docker.sock
  Test SessionID: b615938a811ec0235e820caf26e0526b8c20d2c34537f103189315113b441200
  Test ProcessID: 84fb75ca-daa2-4cea-877f-95c2b4059a61
2023/10/19 11:44:05 🐳 Creating container for image docker.io/testcontainers/ryuk:0.5.1
2023/10/19 11:44:05 ✅ Container created: 7d6d992a8686
2023/10/19 11:44:05 🐳 Starting container: 7d6d992a8686
2023/10/19 11:44:05 ✅ Container started: 7d6d992a8686
2023/10/19 11:44:05 🚧 Waiting for container id 7d6d992a8686 image: docker.io/testcontainers/ryuk:0.5.1. Waiting for: &{Port:8080/tcp timeout:<nil> PollInterval:100ms}
2023/10/19 11:44:05 🐳 Creating container for image registry:2
2023/10/19 11:44:05 ✅ Container created: c0c30ecd6d3f
2023/10/19 11:44:05 🐳 Starting container: c0c30ecd6d3f
2023/10/19 11:44:05 ✅ Container started: c0c30ecd6d3f
2023/10/19 11:44:05 🚧 Waiting for container id c0c30ecd6d3f image: registry:2. Waiting for: &{Port:5000/tcp timeout:<nil> PollInterval:100ms}
    image_test.go:57: Random port: 62449
1.6977086459535549e+09	INFO	Building docker image
1.6977086459676878e+09	INFO	Docker build output	{"output": "{\"stream\":\"\\u001b[91m[WARNING]: Empty continuation line found in: RUN set -x \\u0026\\u0026   apt-get -y update                                     \\u0026\\u0026    apt-get install -y git fcgiwrap spawn-fcgi wget       \\u0026\\u0026    adduser git --home /var/lib/git                       \\u0026\\u0026    adduser nginx git                                     \\u0026\\u0026    git config --system http.receivepack true             \\u0026\\u0026    git config --system http.uploadpack true              \\u0026\\u0026    git config --system user.email \\\"[email protected]\\\"    \\u0026\\u0026    git config --system user.name \\\"Git Server\\\"            \\u0026\\u0026    ln -sf /dev/stdout /var/log/nginx/access.log          \\u0026\\u0026    ln -sf /dev/stderr /var/log/nginx/error.log\\n\\u001b[0m\"}\r\n\n{\"stream\":\"\\u001b[91m[WARNING]: Empty continuation lines will become errors in a future release.\\n\\u001b[0m\"}\r\n\n{\"stream\":\"Step 1/9 : FROM nginx:stable\"}\r\n\n{\"stream\":\"\\n\"}\r\n\n{\"stream\":\" ---\\u003e 7a57d753be2d\\n\"}\r\n\n{\"stream\":\"Step 2/9 : RUN set -x \\u0026\\u0026   apt-get -y update                                     \\u0026\\u0026    apt-get install -y git fcgiwrap spawn-fcgi wget       \\u0026\\u0026    adduser git --home /var/lib/git                       \\u0026\\u0026    adduser nginx git                                     \\u0026\\u0026    git config --system http.receivepack true             \\u0026\\u0026    git config --system http.uploadpack true              \\u0026\\u0026    git config --system user.email \\\"[email protected]\\\"    \\u0026\\u0026    git config --system user.name \\\"Git Server\\\"            \\u0026\\u0026    ln -sf /dev/stdout /var/log/nginx/access.log          \\u0026\\u0026    ln -sf /dev/stderr /var/log/nginx/error.log\"}\r\n\n{\"stream\":\"\\n\"}\r\n\n{\"stream\":\" ---\\u003e Using cache\\n\"}\r\n\n{\"stream\":\" ---\\u003e cfb91d550eec\\n\"}\r\n\n{\"stream\":\"Step 3/9 : ADD nginx.conf /etc/nginx/nginx.conf\"}\r\n\n{\"stream\":\"\\n\"}\r\n\n{\"stream\":\" ---\\u003e Using cache\\n\"}\r\n\n{\"stream\":\" ---\\u003e 2d67fe760e7f\\n\"}\r\n\n{\"stream\":\"Step 4/9 : ADD ./entrypoint.sh /usr/local/bin/entrypoint\"}\r\n\n{\"stream\":\"\\n\"}\r\n\n{\"stream\":\" ---\\u003e Using cache\\n\"}\r\n\n{\"stream\":\" ---\\u003e c8949b58893f\\n\"}\r\n\n{\"stream\":\"Step 5/9 : RUN chmod 755 /usr/local/bin/entrypoint\"}\r\n\n{\"stream\":\"\\n\"}\r\n\n{\"stream\":\" ---\\u003e Using cache\\n\"}\r\n\n{\"stream\":\" ---\\u003e 085923140d1b\\n\"}\r\n\n{\"stream\":\"Step 6/9 : ENTRYPOINT [ \\\"/usr/local/bin/entrypoint\\\" ]\"}\r\n\n{\"stream\":\"\\n\"}\r\n\n{\"stream\":\" ---\\u003e Using cache\\n\"}\r\n\n{\"stream\":\" ---\\u003e b737e7e2e951\\n\"}\r\n\n{\"stream\":\"Step 7/9 : CMD [ \\\"-start\\\" ]\"}\r\n\n{\"stream\":\"\\n\"}\r\n\n{\"stream\":\" ---\\u003e Using cache\\n\"}\r\n\n{\"stream\":\" ---\\u003e f71b921577b6\\n\"}\r\n\n{\"stream\":\"Step 8/9 : COPY srv /var/lib/initial/idpbuilder-resources\"}\r\n\n{\"stream\":\"\\n\"}\r\n\n{\"stream\":\" ---\\u003e Using cache\\n\"}\r\n\n{\"stream\":\" ---\\u003e fca00e2c4f55\\n\"}\r\n\n{\"stream\":\"Step 9/9 : LABEL idpbuilder-gitserver=testcase\"}\r\n\n{\"stream\":\"\\n\"}\r\n\n{\"stream\":\" ---\\u003e Using cache\\n\"}\r\n\n{\"stream\":\" ---\\u003e 66e45a9806ff\\n\"}\r\n\n{\"aux\":{\"ID\":\"sha256:66e45a9806ff592d302013f13d5df0bf297fffbfb021a8ff5004cf0946d99446\"}}\r\n\n{\"stream\":\"Successfully built 66e45a9806ff\\n\"}\r\n\n{\"stream\":\"Successfully tagged localhost:5001/idpbuilder--testcase-gitserver:latest\\n\"}\r\n"}
1.6977086459698071e+09	INFO	Pushing docker image	{"tag": "localhost:5001/idpbuilder--testcase-gitserver"}
1.697708646066113e+09	INFO	Image Pushed	{"digest": "sha256:ac1fc80044364bccf98031432fb32a1d8091be3466ac7442af713e0467c0f06d"}
    image_test.go:94: Removing docker image: Error response from daemon: No such image: sha256:ac1fc80044364bccf98031432fb32a1d8091be3466ac7442af713e0467c0f06d
2023/10/19 11:44:06 🐳 Terminating container: c0c30ecd6d3f
2023/10/19 11:44:06 🚫 Container terminated: c0c30ecd6d3f
--- FAIL: TestReconcileGitServerImage (0.86s)

FAIL

NOTE: Test is failing as the imageTag assumes that we pushed the image to the registry localhost:5001but as testcontainer generates random port, then test will fail: https://golang.testcontainers.org/features/networking/#exposing-container-ports-to-the-host. To be investigated

[cmoulliard]

To be investigated

Test is passing locally using as config

func TestReconcileGitServerImage(t *testing.T) {
	logf.SetLogger(zap.New(zap.UseDevMode(true)))

	ctx := context.Background()
	req := testcontainers.ContainerRequest{
		Image:        "registry:2", // registry image does expose port 5000
		WaitingFor:   wait.ForListeningPort("5000/tcp"),
		ExposedPorts: []string{"5001:5000/tcp"},
	}

[nabuskey]

We have E2E testing that actually brings up a cluster for testing. I don't think this is necessary anymore