fix(cloud): emit ns statuses for in-cluster builds

[thsig]

What this PR does / why we need it:

We now emit a namespaceStatus event whenever one of our internal helpers for checking the status of a Kubernetes namespace is called. This is done whenever we e.g. perform a Deploy, Test or Run in that namespace, or when we perform an in-cluster Build in that namespace (e.g. via BuildKit or Kaniko).

Here, we also remove namespace statuses from the relevant result types (e.g. ServiceStatus, RunStatus and EnvironmentStatus and the associated schemas). These added unwanted noise to the interfaces and schemas.

Instead, we emit the namespaceStatus events on the plugin event broker (ctx.events) and re-emit them to garden.events in the base router.

Which issue(s) this PR fixes:

Previously, automatic environment cleanup would not get triggered for namespaces where only Builds had taken place (i.e. no Deploys, Runs or Tests). This fixes that.

Fixes #4609

Special notes for your reviewer:

I wrote a test case verifying that namespaceStatus events get emitted during BuildKit builds, but it kept hanging due to something Mutagen-related. I've verified manually that the events do get emitted (and so long as we're using the standard helpers for getting the Kubernetes namespace status in the BuildKit and Kaniko handlers, we should be good).

We should investigate this further, but I didn't want to get bogged down in debugging a test case that seemed like it might also introduce flakiness into our integ test suite.

[vvagaytsev]

@thsig this fixes #4609, right?

[thsig]

It does, yeah—forgot to mention that in the description.

[eysi09]

Had one small comment.

Otherwise I feel like we need to re-think this in general but don't want to block the fix.

But shouldn't this be handled in a consistent manner at the action or router level with events that include plugin specific information so that it extends beyond just K8s?

AEC would then be responsible for applying the clean up for different plugins (or using the CLI to do it).

[thsig]

But shouldn't this be handled in a consistent manner at the action or router level ...

In this PR, we're catching namespaceStatus events from the plugin event broker in one place in the base router. We could do it for each individual router method, but that seems unnecessarily repetitive.

... with events that include plugin specific information so that it extends beyond just K8s?

That totally makes sense—we can always change/extend the namespaceStatus events to have a detail field, for example.

In general, I understand your reservations about this PR. That said, my thinking was that the namespaceStatuses field on action results felt bolted-on and not a good pattern, and that this flow was a better fit for another "plugin event designed for Cloud" (like the old log events).

Also, both before and after this PR, we're relying on integ tests to ensure that we don't accidentally make changes that result in us no longer sending the namespaceStatus events when certain plugin handlers are called. There's currently no framework-level guarantee that any namespace statuses were generated by the plugin handler.

I do think that going forward, it would be nice to have a plugin-level mechanism to register event types for the router to listen for.

[eysi09]

Oh I don’t have reservations about the PR per se. Rather this flow in general which needs to be revisited separately imo so we can extend AEC to more plugins.

[thsig]

Should be good to go now, @vvagaytsev.

[vvagaytsev]

Thank you! I've left one more comment here. If the duplicate events make no harm, then all is good!