Pass through service spec values

[willmostly]

Kubernetes supports several types of services. This change passes through arbitrary keys under service.spec, which allows specifying type-specific keys such as nodePort. The port and targetPort are autoconfigured to match the port specified in the gateway http-server configuration, and the selector is autoconfigured to match the deployment.

Suggested release note:

• Set custom service attributes. Breaking change: the service node no longer uses type and port as a subfields. Define the service.spec instead.

[nineinchnick]

I like adding these new test suites! Just a bunch of nitpicks remaining.

[nineinchnick]

Almost there, just two nitpicks about testing.

[nineinchnick]

@willmostly are you going to continue working on this?

[willmostly]

@willmostly are you going to continue working on this?

Yes just getting back into the swing of things...

[willmostly]

Tests are succeeding individually but intermittently failing when run as a group. Looks like there is some flakiness

[nineinchnick]

Almost there

[nineinchnick]

Tests are succeeding individually but intermittently failing when run as a group. Looks like there is some flakiness

I see the https test failed, but it timed out on the 8080 port, and the values have this: http-server.http.enabled: false

[willmostly]

Tests are succeeding individually but intermittently failing when run as a group. Looks like there is some flakiness

I see the https test failed, but it timed out on the 8080 port, and the values have this: http-server.http.enabled: false

The kubectl logs command is printing logs for containers in previous releases, I added --prefix=true because it is confusing for debugging

[willmostly]

connection-test has been updated to retry connections up to 20 times to prevent flaky failures. Some key observations while debugging were that

• All tests succeed when run individually
• All tests failed at various times during the first run except for the first in the sequence, complete_values
• When a connection times out, there is no corresponding entry in the gateway's http-request.log
• Only connections through the internal service time out, nodePort does not
• The nodeport test could successfully connect to 30443 and 30080 at NODE_IP, and https://trino-gateway:443, then timeout to http://trino-gateway:8080

With all this in mind, I suspect some network issues in kind

[willmostly]

The root of my test failures was DNS caching. In the course of making these improvements the service name was changed from {{ include "trino-gateway.fullname" . }} to trino-gateway. Since each test case creates a new deployment, the service was recreated with the same name, but is assigned a different IP address, and CoreDNS would sometimes return the old IP. The extended retries gave the cache a chance to update. This is fixed by making the service name settable and including the test case name in the service name to ensure it is not reused.