Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Endpoint] Encode the index of the alert in the id response #66919

Merged

Conversation

jonathan-buttner
Copy link
Contributor

Summary

Retrieving alerts from ES is currently not working because of data streams. To reproduce this issue:

  1. Checkout master
  2. Log in to kibana
  3. Connect a live endpoint
  4. Generate an alert
  5. Attempt to view the alert in the endpoint app

This will not happen when using the resolver data generator because it uses a mapping file which forces the index to not use data streams.

You should see the following in the UI

Alert UI Failure

image

The issue is that the ingest manager leverages v2 templates and data streams for the data sent by an endpoint. The endpoint binary is configured to send event data and alerts to event-endpoint-1. Instead of creating a normal index in ES, a data stream will be created and the actual index will be something like events-endpoint-1-000001. When the alert details server code tries to retrieve the alert it uses a hard coded index value here: https://github.com/elastic/kibana/blob/master/x-pack/plugins/endpoint/common/alert_constants.ts#L19 (events-endpoint-1) and uses the ES Get api for retrieving the alert by it's ID.

This fails with the following error:

{
  "msg": "[illegal_argument_exception] The provided expression [events-endpoint-1] matches a data stream, specify the corresponding concrete indices instead.",
  "path": "/events-endpoint-1/_doc/5IPTKHIBMyjTtvMm6Xmj",
  "query": {},
  "statusCode": 400,
  "response": "{\"error\":{\"root_cause\":[{\"type\":\"illegal_argument_exception\",\"reason\":\"The provided expression [events-endpoint-1] matches a data stream, specify the corresponding concrete indices instead.\"}],\"type\":\"illegal_argument_exception\",\"reason\":\"The provided expression [events-endpoint-1] matches a data stream, specify the corresponding concrete indices instead.\"},\"status\":400}"
}

To solve this, the index will now be encoded in the response id field.

Testing

Connect a live endpoint and trigger an alert. You should be able to see the alert in resolver.

image

@jonathan-buttner jonathan-buttner added v8.0.0 release_note:skip Skip the PR/issue when compiling release notes Feature:Endpoint Elastic Endpoint feature v7.9.0 labels May 18, 2020
@jonathan-buttner jonathan-buttner requested a review from madirey May 18, 2020 18:25
@jonathan-buttner jonathan-buttner requested review from a team as code owners May 18, 2020 18:25
@elasticmachine
Copy link
Contributor

Pinging @elastic/endpoint-app-team (Feature:Endpoint)

@jonathan-buttner jonathan-buttner changed the title Encode the index of the alert in the id response [Endpoint] Encode the index of the alert in the id response May 18, 2020
@kibanamachine
Copy link
Contributor

💚 Build Succeeded

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@jonathan-buttner jonathan-buttner merged commit 1964b77 into elastic:master May 18, 2020
@jonathan-buttner jonathan-buttner deleted the fix-alerts-data-stream-get branch May 18, 2020 23:39
jonathan-buttner added a commit that referenced this pull request May 19, 2020
…66949)

* Encode the index of the alert in the id response

* Fixing tests

* Adding missed test
gmmorris added a commit to gmmorris/kibana that referenced this pull request May 19, 2020
* master: (24 commits)
  [APM] agent config 'profiling_inferred_spans_min_duration' default value is '0ms' but the min value in the field is '1ms' (elastic#66886)
  [Canvas] Fix flaky custom element functional tests (elastic#65908)
  Fix IE specific flexbox min-height issue (elastic#66555)
  [Discover] Unskip doc link functional test (elastic#66884)
  Index pattern management to Kibana platform (elastic#65026)
  Warning and link to support matrix for IE11 (elastic#66512)
  [Reporting] Consolidate Server Type Defs, move some out of Legacy (elastic#66144)
  [SIEM] [Maps] Fixes Network Map empty tooltip (elastic#66828)
  [Endpoint] Encode the index of the alert in the id response (elastic#66919)
  [services/testSubjects] reduce retry usage, add waitForEnabled (elastic#66538)
  [DOCS] Identifies cloud settings for APM (elastic#66935)
  [SIEM][CASE] Fix configuration's page user experience (elastic#66029)
  Resolver: Display node 75% view submenus (elastic#64121)
  [SIEM] Cases] Capture timeline click and open timeline in case view (elastic#66327)
  [APM] Lowercase agent names so icons work (elastic#66824)
  [dev/cli] add support for --no-cache (elastic#66837)
  [Ingest Manager] Better handling of package installation problems (elastic#66541)
  [ML] Enhances api docs for modules endpoints (elastic#66738)
  dont hide errors (elastic#66764)
  [RFC] Global search API (elastic#64284)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Endpoint Elastic Endpoint feature release_note:skip Skip the PR/issue when compiling release notes v7.9.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants