Make host card overview space aware #113983

nkhristinin · 2021-10-05T18:10:45Z

Summary

Host overview card now is using space value to make requests to this index: ml_host_risk_score_latest_${SPACE}

How to test

Be sure that the Host Overview Card is visible and module is disabled

Make the following requests in Kibana Dev Tools

Create ml_host_risk_score_latest_default index


PUT ml_host_risk_score_latest_default
{
  "mappings": {
    "properties": {
      "@timestamp": {
        "type": "date"
      },
      "host": {
        "properties": {
          "name": {
            "type": "keyword"
           }
         }
       },
      "ingest_timestamp": {
        "type": "date"
      },
      "risk": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "risk_score": {
        "type": "float"
      }
    }
  }
}

Add data to ml_host_risk_score_latest_default index


POST _bulk
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2021-02-23T23:00:00.000Z","risk_score":21,"host":{"name":"ip-10-10-10-121"},"rules":{"Unusual Linux Username":{"average_risk":21,"rule_count":2,"rule_risk":42}},"ingest_timestamp":"2021-07-27T18:02:08.319296053Z","risk":"Low"}
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2021-02-23T23:00:00.000Z","risk_score":21,"host":{"name":"ip-10-10-10-53"},"rules":{"Unusual Linux Username":{"average_risk":21,"rule_count":2,"rule_risk":42}},"ingest_timestamp":"2021-07-27T18:02:08.319484548Z","risk":"Low"}
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2020-08-21T21:00:00.000Z","risk_score":21,"host":{"name":"ip-172-31-44-235"},"rules":{"AWS Access Secret in Secrets Manager":{"average_risk":21,"rule_count":1,"rule_risk":21}},"ingest_timestamp":"2021-07-27T18:02:08.318446811Z","risk":"Low"}
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2021-08-09T00:00:00.000Z","risk_score":21,"host":{"name":"EC2AMAZ-855H8KG"},"rules":{"Unusual Windows Path Activity":{"average_risk":21,"rule_count":1,"rule_risk":21}},"ingest_timestamp":"2021-08-09T01:02:09.209326766Z","risk":"Low"}
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2021-08-11T19:00:00.000Z","risk_score":21,"host":{"name":"skynet-2.skynet.local"},"rules":{"Anomalous Windows Process Creation":{"average_risk":21,"rule_count":1,"rule_risk":21},"Unusual Windows Network Activity":{"average_risk":21,"rule_count":1,"rule_risk":21}},"ingest_timestamp":"2021-08-11T20:02:08.497398786Z","risk":"Low"}
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2021-08-11T19:00:00.000Z","risk_score":21,"host":{"name":"skynet.skynet.local"},"rules":{"Anomalous Windows Process Creation":{"average_risk":21,"rule_count":2,"rule_risk":42}},"ingest_timestamp":"2021-08-11T20:02:08.497783581Z","risk":"Low"}
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2021-08-11T19:00:00.000Z","risk_score":21,"host":{"name":"windows-native.skynet.local"},"rules":{"Anomalous Windows Process Creation":{"average_risk":21,"rule_count":11,"rule_risk":231},"Unusual Windows Network Activity":{"average_risk":21,"rule_count":3,"rule_risk":63}},"ingest_timestamp":"2021-08-11T20:02:08.497874368Z","risk":"Low"}
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2021-08-13T01:00:00.000Z","risk_score":47,"host":{"name":"linux-auditbeat-2"},"rules":{"Unusual Process Execution - Temp":{"average_risk":47,"rule_count":127,"rule_risk":5969}},"ingest_timestamp":"2021-08-13T02:02:08.431822027Z","risk":"Moderate"}
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2021-08-13T14:00:00.000Z","risk_score":47,"host":{"name":"linux-multibeat"},"rules":{"RDP (Remote Desktop Protocol) from the Internet":{"average_risk":47,"rule_count":4,"rule_risk":188}},"ingest_timestamp":"2021-08-13T15:02:08.536780945Z","risk":"Moderate"}

Check that you can see the data with Last year range

Create a new space, and go to security overview. The Host overview card should be visible and module disabled

Checklist

Delete any items that are not applicable to this PR.

Unit or functional tests were updated or added to match the most common scenarios

For maintainers

This was checked for breaking API changes and was labeled appropriately

elasticmachine · 2021-10-06T09:29:31Z

Pinging @elastic/security-solution (Team: SecuritySolution)

nkhristinin · 2021-10-06T12:47:44Z

@elasticmachine merge upstream

kibanamachine · 2021-10-06T12:47:45Z

merge conflict between base and head

rylnd

LGTM! I defer to you on making the space-aware index logic a helper method or not (cc @machadoum).

rylnd · 2021-10-06T19:16:58Z

x-pack/plugins/security_solution/cypress/integration/overview/risky_hosts_panel.spec.ts

      loginAndWaitForPage(OVERVIEW_URL);
      cy.get(
        `${OVERVIEW_RISKY_HOSTS_LINKS} ${OVERVIEW_RISKY_HOSTS_LINKS_WARNING_INNER_PANEL}`
      ).should('not.exist');
      cy.get(`${OVERVIEW_RISKY_HOSTS_VIEW_DASHBOARD_BUTTON}`).should('be.disabled');
      cy.get(`${OVERVIEW_RISKY_HOSTS_TOTAL_EVENT_COUNT}`).should('have.text', 'Showing: 1 host');
+
+      changeSpace(testSpaceName);


rylnd · 2021-10-06T19:22:44Z

x-pack/plugins/security_solution/cypress/tasks/kibana_navigation.ts

+export const changeSpace = (space: string) => {
+  cy.get(`${SPACES_BUTTON}`).click();
+  cy.get(getGoToSpaceMenuItem(space)).click();
+  cy.get(`[data-test-subj="space-avatar-${space}"]`, { timeout: 120000 });


The code as it is currently written will query the page once, and wait up to 2 minutes for a result; adding an assertion here will allow the cy.get to be retried if the assertion fails:

Suggested change

cy.get(`[data-test-subj="space-avatar-${space}"]`, { timeout: 120000 });

cy

.get(`[data-test-subj="space-avatar-${space}"]`})

.should('exist');

More to the point: without the assertion, I think there's a potential race condition between the click/page load and this subsequent cy.get.

...curity_solution/public/overview/containers/overview_risky_host_links/use_risky_host_links.ts

ecezalp

LGTM, great work 😊

nkhristinin · 2021-10-07T14:27:31Z

@elasticmachine merge upstream

nkhristinin · 2021-10-10T13:01:14Z

@elasticmachine merge upstream

kibanamachine · 2021-10-10T15:05:55Z

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	4.3MB	4.3MB	+57.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`securitySolution`	103.8KB	103.8KB	+54.0B

History

💔 Build #158956 failed 0865f7f
💔 Build #158847 failed 4e7961d
💚 Build #158810 succeeded 9dbdf66ff00901d68b9a6d91df1a0c4593784b71
💚 Build #158434 succeeded 1b40ac381c14c3f73b04737249cfcc7a68dcad90
💔 Build #158336 failed 40304d8f005c7b8a17dabf86508525f18b0f9644

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* Make host card overview space aware * Add cypress test * Move getHostRiskIndex to helpers * Fix cypress test Co-authored-by: Kibana Machine <[email protected]>

kibanamachine · 2021-10-10T18:48:37Z

💚 Backport successful

Status	Branch	Result
✅	7.x

This backport PR will be merged automatically after passing CI.

dhurley14 · 2021-10-11T20:17:38Z

x-pack/plugins/security_solution/public/types.ts

@@ -67,6 +68,7 @@ export interface StartPlugins {
  timelines: TimelinesUIStart;
  uiActions: UiActionsStart;
  ml?: MlPluginStart;
+  spaces: SpacesPluginStart;


@nkhristinin I think this should be optional. Just came across this as a conflict to my PR -> #112478

Suggested change

spaces: SpacesPluginStart;

spaces?: SpacesPluginStart;

CC: @rylnd @ecezalp just FYI since I saw you both reviewed this. The security solution has the spaces plugin listed as optional. We should not be assuming this is always present. This could have lead to some not-so-great bugs.

kibana/x-pack/plugins/security_solution/kibana.json

Lines 29 to 43 in a0b55b3

"optionalPlugins": [

"encryptedSavedObjects",

"fleet",

"ml",

"dashboard",

"newsfeed",

"security",

"spaces",

"usageCollection",

"lens",

"lists",

"home",

"telemetry",

"telemetryManagementSection"

],

Thanks for fixing it in your PR!

…od here elastic#113983

…es by `id` (#112478) * added outcome to backend routes * adds so resolved property alias_target_id to response * adds UI portion * working URL redirect on aliasMatch - todo -> update rule details page refresh button to use SO resolve. * cleanup * fix integration tests * fix jest tests * cleanup types * fix eslint.. I think vs code formatted this * WIP - undo me, working index.test.ts function * WIP - also undo me, probably * working test for aliasMatch, need to add test for outcome = conflict * add conflict callout when SO resolve yields conflict outcome * code cleanup * fix type issues * small cleanup, fix jest test after undoing changes for getFailingRuleStatus * cleanup tests * add alias_target_id to response validation too * unit test changes * update tests again * add all dependencies to useEffect and prefer useMemo * add type cast * adds integration tests for different outcomes after mocking a migrated rule leading to an aliasMatch and a migrated rule + accidental inserted rule to lead to a conflict. Also removes the outcome property if it is an exactMatch * remove unused import * fix test * functional WIP * cleanup * cleanup * finishing touches to address PR review comments * remove console.error * fix bug where spaces was not typed correctly in the plugin start method here #113983

kibanamachine · 2021-10-12T18:54:59Z

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

* Make host card overview space aware * Add cypress test * Move getHostRiskIndex to helpers * Fix cypress test Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Khristinin Nikita <[email protected]>

…ng rules by `id` (#112478) (#114683) * [Security Solution] [Platform] Utilize SO resolve api for reading rules by `id` (#112478) * added outcome to backend routes * adds so resolved property alias_target_id to response * adds UI portion * working URL redirect on aliasMatch - todo -> update rule details page refresh button to use SO resolve. * cleanup * fix integration tests * fix jest tests * cleanup types * fix eslint.. I think vs code formatted this * WIP - undo me, working index.test.ts function * WIP - also undo me, probably * working test for aliasMatch, need to add test for outcome = conflict * add conflict callout when SO resolve yields conflict outcome * code cleanup * fix type issues * small cleanup, fix jest test after undoing changes for getFailingRuleStatus * cleanup tests * add alias_target_id to response validation too * unit test changes * update tests again * add all dependencies to useEffect and prefer useMemo * add type cast * adds integration tests for different outcomes after mocking a migrated rule leading to an aliasMatch and a migrated rule + accidental inserted rule to lead to a conflict. Also removes the outcome property if it is an exactMatch * remove unused import * fix test * functional WIP * cleanup * cleanup * finishing touches to address PR review comments * remove console.error * fix bug where spaces was not typed correctly in the plugin start method here #113983 # Conflicts: # x-pack/plugins/security_solution/public/overview/containers/overview_risky_host_links/use_hosts_risk_score.ts # x-pack/plugins/security_solution/public/types.ts * fix eslint * skip 8.0 integration tests for 7.16

nkhristinin requested a review from a team as a code owner October 5, 2021 18:10

nkhristinin requested review from rylnd and ecezalp October 6, 2021 09:27

nkhristinin added Team: CTI Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v7.16.0 v8.0.0 labels Oct 6, 2021

nkhristinin added auto-backport Deprecated - use backport:version if exact versions are needed release_note:feature Makes this part of the condensed release notes release_note:skip Skip the PR/issue when compiling release notes labels Oct 6, 2021

rylnd approved these changes Oct 6, 2021

View reviewed changes

nkhristinin added 4 commits October 7, 2021 13:35

Make host card overview space aware

06bae88

Add cypress test

a3afab7

Move getHostRiskIndex to helpers

44a5758

Fix cypress test

4e7961d

nkhristinin force-pushed the space-aware branch from 9dbdf66 to 4e7961d Compare October 7, 2021 11:38

ecezalp approved these changes Oct 7, 2021

View reviewed changes

Merge branch 'master' into space-aware

0865f7f

Merge branch 'master' into space-aware

b29e545

nkhristinin merged commit 30aeb81 into elastic:master Oct 10, 2021

kibanamachine mentioned this pull request Oct 10, 2021

[7.x] Make host card overview space aware (#113983) #114416

Merged

dhurley14 reviewed Oct 11, 2021

View reviewed changes

dhurley14 added a commit to dhurley14/kibana that referenced this pull request Oct 12, 2021

fix bug where spaces was not typed correctly in the plugin start meth…

e29fc03

…od here elastic#113983

kibanamachine added backport missing Added to PRs automatically when the are determined to be missing a backport. and removed backport missing Added to PRs automatically when the are determined to be missing a backport. labels Oct 12, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make host card overview space aware #113983

Make host card overview space aware #113983

nkhristinin commented Oct 5, 2021 •

edited

Loading

elasticmachine commented Oct 6, 2021

nkhristinin commented Oct 6, 2021

kibanamachine commented Oct 6, 2021

rylnd left a comment

rylnd Oct 6, 2021

rylnd Oct 6, 2021

rylnd Oct 6, 2021

ecezalp left a comment

nkhristinin commented Oct 7, 2021

nkhristinin commented Oct 10, 2021

kibanamachine commented Oct 10, 2021

kibanamachine commented Oct 10, 2021

dhurley14 Oct 11, 2021 •

edited

Loading

dhurley14 Oct 11, 2021

nkhristinin Oct 12, 2021

kibanamachine commented Oct 12, 2021

-  cy.get(`[data-test-subj="space-avatar-${space}"]`, { timeout: 120000 });
+  cy
+    .get(`[data-test-subj="space-avatar-${space}"]`})
+    .should('exist');

	"optionalPlugins": [
	"encryptedSavedObjects",
	"fleet",
	"ml",
	"dashboard",
	"newsfeed",
	"security",
	"spaces",
	"usageCollection",
	"lens",
	"lists",
	"home",
	"telemetry",
	"telemetryManagementSection"
	],

Make host card overview space aware #113983

Make host card overview space aware #113983

Conversation

nkhristinin commented Oct 5, 2021 • edited Loading

Summary

How to test

Checklist

For maintainers

elasticmachine commented Oct 6, 2021

nkhristinin commented Oct 6, 2021

kibanamachine commented Oct 6, 2021

rylnd left a comment

Choose a reason for hiding this comment

rylnd Oct 6, 2021

Choose a reason for hiding this comment

rylnd Oct 6, 2021

Choose a reason for hiding this comment

rylnd Oct 6, 2021

Choose a reason for hiding this comment

ecezalp left a comment

Choose a reason for hiding this comment

nkhristinin commented Oct 7, 2021

nkhristinin commented Oct 10, 2021

kibanamachine commented Oct 10, 2021

💚 Build Succeeded

Metrics [docs]

Async chunks

Page load bundle

History

kibanamachine commented Oct 10, 2021

💚 Backport successful

dhurley14 Oct 11, 2021 • edited Loading

Choose a reason for hiding this comment

dhurley14 Oct 11, 2021

Choose a reason for hiding this comment

nkhristinin Oct 12, 2021

Choose a reason for hiding this comment

kibanamachine commented Oct 12, 2021

nkhristinin commented Oct 5, 2021 •

edited

Loading

dhurley14 Oct 11, 2021 •

edited

Loading