Test helm charts and quickstart steps in CI

[dnephin]

We don't currently have any automated tests for our helm charts or the steps in our quickstart.

We can run the quickstart against kind, which seems to run well enough in CI. I think we should setup a test suite that does some basic testing of our helm charts following the quickstart guide.

Proposal

Create a new Go module github.com/infrahq/infra/tests at ./tests in the infra repo. Using a separate module allows us to run these tests separately from our main unit and integration test suite. I think we'll always want to run these separately because they will be much slower, they will require a kube cluster, and will likely be fairly destructive to the kube cluster they run against. We want to prevent them from running accidentally, and a separate Go module is a good way to ensure that.

Add a new CI job to run this test suite. Hopefully it runs fast enough to be run on all PRs, but if not we should run it on the release-please PR, and any PR that modifies the helm/ directory. We can also add a github label to allow it to be triggered on other PRs (similar to action/fuzz). The CI job will start a kind cluster and configure it (I believe it will need metallb setup to support the connector).

Test suite

The test suite should focus more on user flows (and not on specific CLI commands or API endpoints). The first flow should probably mirror the quickstart guide as closely as possible (although it will need to target the locally built image and helm charts instead of the published ones). In the future we can add other flows, but we should keep it to a small number of popular setups.

The test suite can use a lot of the same tools we have for testing the CLI.

• https://pkg.go.dev/gotest.tools/v3/icmd for running the CLI
• https://github.com/Netflix/go-expect for navigating interactive prompts

[stale]

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.