TESTING.md

Testing Selenium Grid Helm Chart

All related testing to this helm chart will be documented in this file.

Test Traceability Matrix

Features	TC Description	Coverage	Test via
Basic Auth	Basic Auth is disabled	✓	Cluster
	Basic Auth is enabled	✓	Cluster
Auto scaling	Auto scaling with enableWithExistingKEDA is true	✓	Cluster
	Auto scaling with scalingType is job	✓	Cluster
	Auto scaling with scalingType is deployment	✓	Cluster
	Auto scaling with autoscaling.scaledOptions.minReplicaCount is 0	✓	Cluster
	Parallel tests execution against node autoscaling	✓	Cluster
Ingress	Ingress is enabled without hostname	✓	Cluster
	Ingress is enabled with hostname is set	✓	Cluster
	Hub sub-path is set with Ingress ImplementationSpecific paths	✓	Cluster
	ingress.nginx configs for NGINX ingress controller annotations	✓	Template
Distributed components	isolateComponents is enabled	✓	Cluster
	isolateComponents is disabled	✓	Cluster
Browser Nodes	Node nameOverride is set	✓	Cluster
	Sanity tests in node	✓	Cluster
	Video recorder is enabled in node	✓	Cluster
	Node extraEnvironmentVariables is set value	✓	Cluster
General	Set new image registry via global.seleniumGrid.imageRegistry	✓	Cluster
	Components are able to set .affinity	✓	Template
Tracing	Enable tracing via SE_ENABLE_TRACING	✓	Cluster
	Disable tracing via SE_ENABLE_TRACING	✓	Cluster
Node component	SE_NODE_PORT can set a port different via .port	✓	Cluster
	Extra ports can be exposed on container via .ports	✓	Cluster
	Extra ports can be exposed on Service via .service.ports	✓	Cluster
	Service type change to NodePort, specific NodePort can be set	✓	Cluster

CI Test Traceability Matrix

We have a CI pipeline to test the Helm chart with the following test cases.

Matrix	job	deployment	job_https	job_hostname	deployment_https
Cluster	Kind	Kind	Kind	Minikube	Minikube
Kubernetes version	v1.25.16	v1.26.14	v1.27.11	v1.28.7	v1.29.2
Autoscaling	x	x	x	x	x
Scaling type deployment		x			x
Scaling type job	x		x	x
Full distributed mode	x				x
Enable basic auth		x		x	x
Ingress hostname	x			x	x
Ingress K8S_PUBLIC_IP		x	x
HTTPS tls.enabled			x		x
Enable tracing	x	x		x
Enable video recorder	x	x	x	x	x
Test headless		x	x

Test Chart Template

• By using helm template command, the chart template is tested without installing it to Kubernetes cluster.
• Templates are rendered and the output as a YAML manifest file. The manifest file is then asserted with pyyaml.
• Set of values are used to render the templates located in tests/charts/templates/render.

# Back to root directory
cd ../..

# Build chart dependencies and lint
make chart_build

# Test chart template
make chart_test_template

• Build chart dependencies and lint requires Chart Testing ct. There is a config file ct.yaml to configure the chart testing.

Build & test Docker images with deploy to Kubernetes cluster

Noted: These make commands are composed and tested on Linux x86_64. Run entire commands to build and test Docker images with Helm charts in local environment.

# Back to root directory
cd ../..

# Setup Kubernetes environment
make chart_setup_env

# Build Docker images
make build

# Build and lint charts
make chart_build

# Setup Kubernetes cluster
make chart_cluster_setup

# Test Selenium Grid on Kubernetes
make chart_test_autoscaling_deployment

make chart_test_autoscaling_job

# Cleanup Kubernetes cluster
make chart_cluster_cleanup

• Setup Kubernetes environment requires Kind and Kubectl.
• Set of values are used to deploy the chart to Kubernetes cluster located in tests/charts/ci.