[SIEM] Client NP Cutover (elastic#64251)

* Move SIEM public/ folder to NP plugin

This is solely renames; fixes come next.

* Update relative imports in our API tests

* Fix linter errors following move to NP folder

These paths got a little shorter, so some lines could be collapsed.

* Move client dependencies to NP package.json

I'm removing the @types/js-yaml for now because I'm not sure we need it;
I'll add it back later if we do.

* Fix relative imports to other plugins

* Fix errant uses of ui/chrome

* Remove legacy plugin shim

* Move feature registration into plugin

This previously had to be part of legacy bootstrapping due to an order
of operations issue.

* Disconnect legacy plugin

The index file should now be redundant with what's in the plugin:

* app registration
* feature registration

* Move public gitattributes

* Remove references to legacy embeddables

We can now use the NP API. Maps embeddable will not work here until
their work is merged, but this should prevent us from importing legacy
code and thus breaking the build.

* Add our frontend dependencies to kibana.json

These are all required for now, because that's how they're typed. If
they _should_ be optional (and I think several should), we need to
update the type and handle the null case within the app.

* Replace use of ui/new_platform mocks in embeddable utils

* Fix remaining jest tests

* Replace build-breaking ui/new_platform mocks with equivalents in core
proper
* Remove unnecessary mocks of ui/new_platform

* Remove references to legacy SIEM folder

* I left the reference in CODEOWNERS in case someone tries to sneak
something back
* I left the .gitignore reference for the same reason

* Fix mocks of relative paths

These were not caught by typescript and were causing test failures.

* Export our client plugin contracts

They're empty for now.

* Move from deprecated appmount API

The new one dropped a param we weren't using.

* Add missing mock causing test failures

* Don't re-export core types from our plugin

Import them from core where we need them, instead

* Move Actions UI registry outside of mount

This is already imported, there's no benefit (and potential timing
issues) with doing this inside the mount.

* Add security's setup contract to our StartServices

This doesn't change what's used, only how we're typing it. The types are now a
little more truthful as:

* our StartPlugins don't include setup contracts
* our StartServices includes everything we use at Start time, including
the one setup plugin.

* Add order and icon back to the sidebar link

* Replace plugin class properties with constants

These are shared, and should be consistent.

* Enable our UI on NP

* Add missed plugin dependencies

We're not using their contracts, but we are importing code from them.

* Revert use of constant in translation

Can't do that, whoops

* i18n our feature catalogue entry

* Remove unnecessary array from single element

* Remove unused keys

These were the legacy translations used... well, I don't know where they
were used.

* Ignore circular dependencies in external plugins

* Normalize exclusions

* Add undeclared dependencies to kibana.json

We import our maps embeddable from maps, and we pass inspector to the
embeddable. I just missed these in my audit. This was causing errors in
the map embeddable.

* Await our call to setLayerList

This is an async call that we need to complete before we can render.

* Reduce siem plugin size

When we load our initial plugin (before our app is loaded), were were
implicitly importing all of kibana_react with this import. While a
global module prevents this from affecting our bundle size currently,
that could change in the future. Since we only need a reference to our
class, we just import that instead.

.eslintrc.js

@@ -562,7 +562,7 @@ module.exports = {
      */
     {
       // front end typescript and javascript files only
-      files: ['x-pack/legacy/plugins/siem/public/**/*.{js,ts,tsx}'],
+      files: ['x-pack/plugins/siem/public/**/*.{js,ts,tsx}'],
       rules: {
         'import/no-nodejs-modules': 'error',
         'no-restricted-imports': [
@@ -611,7 +611,7 @@ module.exports = {
     // {
     //   // will introduced after the other warns are fixed
     //   // typescript and javascript for front end react performance
-    //   files: ['x-pack/legacy/plugins/siem/public/**/!(*.test).{js,ts,tsx}'],
+    //   files: ['x-pack/plugins/siem/public/**/!(*.test).{js,ts,tsx}'],
     //   plugins: ['react-perf'],
     //   rules: {
     //     // 'react-perf/jsx-no-new-object-as-prop': 'error',

Jenkinsfile

@@ -42,7 +42,7 @@ kibanaPipeline(timeoutMinutes: 135, checkPrChanges: true) {
             'xpack-ciGroup10': kibanaPipeline.xpackCiGroupProcess(10),
             'xpack-accessibility': kibanaPipeline.functionalTestProcess('xpack-accessibility', './test/scripts/jenkins_xpack_accessibility.sh'),
             'xpack-siemCypress': { processNumber ->
-              whenChanged(['x-pack/plugins/siem/', 'x-pack/legacy/plugins/siem/', 'x-pack/test/siem_cypress/']) {
+              whenChanged(['x-pack/plugins/siem/', 'x-pack/test/siem_cypress/']) {
                 kibanaPipeline.functionalTestProcess('xpack-siemCypress', './test/scripts/jenkins_siem_cypress.sh')(processNumber)
               }
             },

x-pack/.i18nrc.json

@@ -36,7 +36,7 @@
     "xpack.searchProfiler": "plugins/searchprofiler",
     "xpack.security": ["legacy/plugins/security", "plugins/security"],
     "xpack.server": "legacy/server",
-    "xpack.siem": ["plugins/siem", "legacy/plugins/siem"],
+    "xpack.siem": "plugins/siem",
     "xpack.snapshotRestore": "plugins/snapshot_restore",
     "xpack.spaces": ["legacy/plugins/spaces", "plugins/spaces"],
     "xpack.taskManager": "legacy/plugins/task_manager",

x-pack/index.js

@@ -17,7 +17,6 @@ import { spaces } from './legacy/plugins/spaces';
 import { canvas } from './legacy/plugins/canvas';
 import { infra } from './legacy/plugins/infra';
 import { taskManager } from './legacy/plugins/task_manager';
-import { siem } from './legacy/plugins/siem';
 import { remoteClusters } from './legacy/plugins/remote_clusters';
 import { upgradeAssistant } from './legacy/plugins/upgrade_assistant';
 import { uptime } from './legacy/plugins/uptime';
@@ -42,7 +41,6 @@ module.exports = function(kibana) {
     indexManagement(kibana),
     infra(kibana),
     taskManager(kibana),
-    siem(kibana),
     remoteClusters(kibana),
     upgradeAssistant(kibana),
     uptime(kibana),

x-pack/plugins/siem/.gitattributes

@@ -1,4 +1,6 @@
 # Auto-collapse generated files in GitHub
 # https://help.github.com/en/articles/customizing-how-changed-files-appear-on-github
 x-pack/plugins/siem/server/graphql/types.ts linguist-generated=true
+x-pack/plugins/siem/public/graphql/types.ts linguist-generated=true
+x-pack/plugins/siem/public/graphql/introspection.json linguist-generated=true
 

x-pack/plugins/siem/common/constants.ts

@@ -6,6 +6,8 @@
 
 export const APP_ID = 'siem';
 export const APP_NAME = 'SIEM';
+export const APP_ICON = 'securityAnalyticsApp';
+export const APP_PATH = `/app/${APP_ID}`;
 export const DEFAULT_BYTES_FORMAT = 'format:bytes:defaultPattern';
 export const DEFAULT_DATE_FORMAT = 'dateFormat';
 export const DEFAULT_DATE_FORMAT_TZ = 'dateFormat:tz';

x-pack/plugins/siem/kibana.json

@@ -3,8 +3,23 @@
   "version": "8.0.0",
   "kibanaVersion": "kibana",
   "configPath": ["xpack", "siem"],
-  "requiredPlugins": ["actions", "alerting", "features", "licensing"],
-  "optionalPlugins": ["encryptedSavedObjects", "ml", "security", "spaces"],
+  "requiredPlugins": [
+    "actions",
+    "alerting",
+    "data",
+    "embeddable",
+    "esUiShared",
+    "features",
+    "home",
+    "inspector",
+    "kibanaUtils",
+    "licensing",
+    "maps",
+    "triggers_actions_ui",
+    "uiActions",
+    "usageCollection"
+  ],
+  "optionalPlugins": ["encryptedSavedObjects", "ml", "newsfeed", "security", "spaces"],
   "server": true,
-  "ui": false
+  "ui": true
 }

x-pack/plugins/siem/package.json

@@ -5,7 +5,7 @@
   "private": true,
   "license": "Elastic-License",
   "scripts": {
-    "extract-mitre-attacks": "node scripts/extract_tactics_techniques_mitre.js && node ../../../scripts/eslint ../../legacy/plugins/siem/public/pages/detection_engine/mitre/mitre_tactics_techniques.ts --fix",
+    "extract-mitre-attacks": "node scripts/extract_tactics_techniques_mitre.js && node ../../../scripts/eslint ./public/pages/detection_engine/mitre/mitre_tactics_techniques.ts --fix",
     "build-graphql-types": "node scripts/generate_types_from_graphql.js",
     "cypress:open": "cypress open --config-file ./cypress/cypress.json",
     "cypress:run": "cypress run --spec ./cypress/integration/**/*.spec.ts --config-file ./cypress/cypress.json --reporter ../../node_modules/cypress-multi-reporters --reporter-options configFile=./cypress/reporter_config.json; status=$?; ../../node_modules/.bin/mochawesome-merge --reportDir ../../../target/kibana-siem/cypress/results > ../../../target/kibana-siem/cypress/results/output.json; ../../../node_modules/.bin/marge ../../../target/kibana-siem/cypress/results/output.json --reportDir ../../../target/kibana-siem/cypress/results; mkdir -p ../../../target/junit && cp ../../../target/kibana-siem/cypress/results/*.xml ../../../target/junit/ && exit $status;",
@@ -15,6 +15,7 @@
     "@types/lodash": "^4.14.110"
   },
   "dependencies": {
-    "lodash": "^4.17.15"
+    "lodash": "^4.17.15",
+    "react-markdown": "^4.0.6"
   }
 }