[ML] Removing counter fields from wizards

[jgowdyelastic]

Further improvements to the anomaly detection and data frame analysts wizards to ensure counter fields cannot be selected when configuring a new job.
Adds a counter flag to the new_job_caps response which is used to remove counter fields from dropdowns in the DFA and advanced AD wizards.

The outlier detection wizard still has an issue caused by the call to the _explain endpoint. This needs to be fixed in elasticsearch elastic/elasticsearch#94691

[elasticmachine]

Pinging @elastic/ml-ui (:ml)

[qn895]

Not sure if this is out of scope of this PR, but we should add this logic for the histogram fetching as well? Running into this error when using the DFA wizard:

For outlier detection, should we disable counter fields? Currenty, including them will yield a validation error.

Screen.Recording.2023-03-28.at.09.56.28.mov

[jgowdyelastic]

Not sure if this is out of scope of this PR, but we should add this logic for the histogram fetching as well? Running into this error when using the DFA wizard:

For outlier detection, should we disable counter fields? Currenty, including them will yield a validation error.

@qn895 I'm not able to reproduce these errors, the counter field never present for me?

[jgowdyelastic]

@qn895 I've pushed an update which cleans up the code a bit and removes the errors thrown when loading the histograms.
Counter fields are no longer sent to the histogram endpoint.
There is still a problem with the field selection table showing counter fields for selection, however this will hopefully be fixed by the backend change for the explain api. This table is populated by the explain api, and not from the same data which populates the histogram table above it.
IMO the explain api should mark counter fields as is_included: false so they are not selectable in the UI.

[qn895]

Tested latest changes and LGTM 🎉 Seeing counter fields in the included fields but I suppose that would be fixed when elastic/elasticsearch#94691 is addressed.

[peteharverson]

Tested latest changes and LGTM.

If / when Elasticsearch includes supported aggregations in the field caps api response (see elastic/elasticsearch#93884) we could look to allow certain aggregations to be used in the wizards for counter fields.

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 5f2f8a6

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
ml	3.4MB	3.4MB	+386.0B

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
securitySolution	432	435	+3

Total ESLint disabled count

id	before	after	diff
securitySolution	512	515	+3

History

• 💔 Build #116826 failed 05d7db3
• 💔 Build #116407 failed 739ce7b
• 💔 Build #116134 failed 13274f5
• 💔 Build #116012 failed 398882a
• 💔 Build #115939 failed 2359e00

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jgowdyelastic