Ensure type exists for all monitoring configuration

[jakelandis]

#47711 and #47246 helped to validate that monitoring settings are
rejected at time of setting the monitoring settings. Else an invalid
monitoring setting can find it's way into the cluster state and result
in an exception thrown [1] on the cluster state application (there by
causing significant issues). Some additional monitoring settings have
been identified that can result in invalid cluster state that also
result in exceptions thrown on cluster state application.

All settings require a type of either http or local to be
applicable. When a setting is changed, the exporters are automatically
updated with the new settings. However, if the old or new settings lack
of a type setting an exception will be thrown (since exporters are
always of type 'http' or 'local'). Arguably we shouldn't blindly create
and destroy new exporters on each monitoring setting update, but the
lifecycle of the exporters is abit out the scope this PR is trying to
address.

This commit introduces a similar methodology to check for validity as
#47711 and #47246 but this time for ALL (including non-http) settings.
Monitoring settings are not useful unless there an exporter with a type
defined. The type is used as dependent setting, such that it must
exist to set the value. This ensures that when any monitoring settings
changes that they can only get added to cluster state if the type
exists. If the type exists (and the other validations pass) then the
exporters will get re-built and the cluster state remains valid.

Tests have been included to ensure that all dynamic monitoring settings
have the type as dependent settings.

[1]
org.elasticsearch.common.settings.SettingsException: missing exporter type for [found-user-defined] exporter
at org.elasticsearch.xpack.monitoring.exporter.Exporters.initExporters(Exporters.java:126) ~[?:?]

[jakelandis]

This is necessary since this test (and some others here) set the type via node level setting but the settings validation does not validate across node level vs. dynamic settings (they get merged after independent validation).

It also looks like we don't support validation across persistent vs. transient settings either, which makes any of these cross settings dependencies validation a bit terse. This translates to a real change in behavior w.r.t monitoring, but cross level validation is a bigger issue then monitoring. The safety here (imo) outweighs the convenience of cross type setting validation (and already exists for all HTTP type settings).

[elasticmachine]

Pinging @elastic/es-core-features (:Core/Features/Monitoring)

[danhermann]

LGTM.

As a related note, while testing this, I noticed that I could not dynamically remove an exporter defined in the elasticsearch.yml file and I thought that was possible by setting all the exporter's fields to null. This PR did not change that behavior, but if that was supposed to be possible, it broke somewhere along the way.

[jasontedor]

I don’t think we intended that to be possible. It applies to all settings defined in yml that are dynamic. When you null them out, it removes them from the cluster state, it doesn’t apply an “unset” override.