[Security Solution] [Detections] Error when editing duplicate of prebuilt detection rule using EQL

[MikePaquette]

Kibana version: 7.11 BC5

Elasticsearch version: 7.11 BC5

Server OS version: Elastic Cloud

Browser version: Chrome Version 88.0.4324.96

Browser OS version: macOS 10.14.6

Original install method (e.g. download page, yum, from source, etc.): Elastic Cloud Deployment

Describe the bug:
When attempting to edit an EQL-based rule (which has been duplicated from a prebuilt rule), an error occurs

Steps to reproduce:

1. Load pre-built rules (461)
2. Duplicate all 461 pre-built rules - now have 461 custom rules
3. View custom rules
4. click on one EQL-based rule to view rule details (e.g., Persistence via Microsoft Office AddIns [Duplicate])
5. Click on Edit Rule Settings
6. Get error toaster message and these details:

verification_exception
Found 1 problem
line -1:-1: Unknown index [*,-*]

Error: EsError
    at search_interceptor_EnhancedSearchInterceptor.handleSearchError (https://9047020492624fefa1c5b15963618d35.europe-west1.gcp.cloud.es.io:9243/37827/bundles/plugin/data/data.plugin.js:8:189169)
    at t.selector (https://9047020492624fefa1c5b15963618d35.europe-west1.gcp.cloud.es.io:9243/37827/bundles/plugin/dataEnhanced/dataEnhanced.plugin.js:2:35413)
    at t.error (https://9047020492624fefa1c5b15963618d35.europe-west1.gcp.cloud.es.io:9243/37827/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:429:94087)
    at t._error (https://9047020492624fefa1c5b15963618d35.europe-west1.gcp.cloud.es.io:9243/37827/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:429:134047)
    at t.error (https://9047020492624fefa1c5b15963618d35.europe-west1.gcp.cloud.es.io:9243/37827/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:21:40304)
    at t._error (https://9047020492624fefa1c5b15963618d35.europe-west1.gcp.cloud.es.io:9243/37827/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:21:40610)
    at t.error (https://9047020492624fefa1c5b15963618d35.europe-west1.gcp.cloud.es.io:9243/37827/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:21:40304)
    at t._error (https://9047020492624fefa1c5b15963618d35.europe-west1.gcp.cloud.es.io:9243/37827/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:21:40610)
    at t.error (https://9047020492624fefa1c5b15963618d35.europe-west1.gcp.cloud.es.io:9243/37827/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:21:40304)
    at t._error (https://9047020492624fefa1c5b15963618d35.europe-west1.gcp.cloud.es.io:9243/37827/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:21:40610)

Expected behavior:
No error is expected.

Screenshots (if relevant):

Errors in browser console (if relevant): None

Provide logs and/or server output (if relevant): N/A

Any additional context: Only tested on one deployment

cc: @MadameSheema have you seen this behavior?
cc: @paulewing

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[peluja1012]

@MikePaquette Thanks for reporting this. This behavior is present in previous release versions as well. It occurs because we validate EQL queries upon rule creation and rule edit. Validation takes into account the configured indices. If none of the indices exist we display the error you posted above. This error is valid because none of the indices exist and thus the rule will also fail upon rule execution. The logs-endpoint.events indices don't get created until the Elastic Endpoint package is installed. Here is a screenshot of a duplicated prebuilt rule failing execution with the same error due to missing indices:

While the observed behavior is expected, we could consider making a UX improvement such that we display the index error as first a class validation error as opposed to displaying it in an error toaster (which signals the user that there was an unhandled error).

[dontcallmesherryli]

Expected behavior, requires additional UX input to make it a better user experience.

[leegengyu]

@peluja1012 I would like to seek clarification regarding what you mentioned: If none of the indices exist we display the error you posted above. This error is valid because none of the indices exist and thus the rule will also fail upon rule execution.

In my case, I am seeing the same error (with a small variant):

The Lateral Movement via Startup Folder rule stated 3 index patterns: logs-endpoint.events.*, winlogbeat-*, and logs-windows.*. I have only got winlogbeat-* active, but not the other 2.

However, despite the case, I am still getting error for this rule (and many others, since they typically specify those 3 index patterns). Could you advise on this please?

[peluja1012]

Hi @leegengyu, could you please try executing the same EQL query using Kibana -> Dev Tools and let us know what error is reported? Also, what versions of Elasticsearch and Kibana are you running?

GET logs-endpoint.events.*,winlogbeat-*,logs-windows.*/_eql/search
{
  "query":"""
  <EQL query>
  """
}

[leegengyu]

@peluja1012 Thanks for getting back to me.

This error did not appear anymore (somehow). But I'll be sure to get back to here if it does.

Thank you once again!

[yctercero]

Addressed by #180407