[ML] Single Metric Viewer doesn't load chart if non-aggregated summmary count is used

[richcollier]

Kibana version:
v6.7

Based on a customer-discovered situation where it seems that the SMV does not properly load the chart when the following conditions are met:

• The job is created in the Adv Job Wizard
• The job has a split using either by or partition
• The summmary_count_field_name is used in the job config
• The datafeed query does NOT contain an aggregation, rather a straight query in which a field in the raw results is a summarized count. I'm not sure this is the key to why the bug exists but is a big clue.

For example, a snippet of a (somewhat contrived) job config, used on the "gallery" demo weblogs merely to reproduce the situation:

  "analysis_config": {
    "bucket_span": "1h",
    "summary_count_field_name": "status",
    "detectors": [
      {
        "detector_description": "high_count by action",
        "function": "high_count",
        "by_field_name": "action",
        "detector_index": 0
      }
    ],
    "influencers": [
      "action",
      "status"
    ]
  },

(datafeed is just a match_all query)

The result when loading the SMV is just an ever spinning Loading widget:

But, if I enable model debug ( "model_plot_config": { "enabled": true }, )

Then it works:

This is because, when the model bounds are ON, then the graph in the Single metric Viewer is built from the model plot information located in .ml-anomalies. However, if it is NOT on, then the UI tries to query the original source index to plot the graph. There must be some problem in that process.

[elasticmachine]

Pinging @elastic/ml-ui

[jkelastic]

Pinging @elastic/ml-core just following up to see if there's anything else you need

[droberts195]

This sounds uncannily similar to the problem that was solved in #31924. It’s not exactly the same problem as this one is in the results section whereas that was in the config section. But maybe there’s a clue in the way query_string malfunctioned when the default_field of an index didn’t exist on most of the documents in it.

Do we have a support diag from this customer? If not it would be good to get one as it will show the mappings and index settings for the affected indices.

[peteharverson]

The issue here is caused by the fact that a non numeric field (status) is being used as the summary_count_field_name in the job config:

When the datafeed for the job doesn't use an aggregation, the ML back-end analytics is able to use the non-numeric field by scrolling through every document and parsing the keyword field to a number. However the UI obtains the data for the charts in the Single Metric Viewer and Anomaly Explorer by using it directly in an Elasticsearch aggregation, so only numeric fields can be plotted in charts. Likewise, Kibana visualizations can only plot numeric fields (so for example. status could not be plotted in a Kibana line chart).

When a non-numeric field is used in this way, the ML UI should really be hiding the charts in the Anomaly Explorer, and preventing the detector being displayed in the Single Metric Viewer - an issue is currently open for this - #19114.

Closing issue as #19114 is already open for hiding the charts when a non-numeric field is used as the summary_count_field_name.