Merge branch 'main' of github.com:elastic/kibana into rule-details-ru…

…leId

.buildkite/pipelines/pull_request/osquery_cypress.yml

@@ -0,0 +1,11 @@
+steps:
+  - command: .buildkite/scripts/steps/functional/osquery_cypress.sh
+    label: 'Osquery Cypress Tests'
+    agents:
+      queue: ci-group-6
+    depends_on: build
+    timeout_in_minutes: 120
+    retry:
+      automatic:
+        - exit_status: '*'
+          limit: 1

.buildkite/scripts/pipelines/pull_request/pipeline.js

@@ -86,6 +86,16 @@ const uploadPipeline = (pipelineContent) => {
       pipeline.push(getPipeline('.buildkite/pipelines/pull_request/fleet_cypress.yml'));
     }
 
+    if (
+      (await doAnyChangesMatch([
+        /^x-pack\/plugins\/osquery/,
+        /^x-pack\/test\/osquery_cypress/,
+      ])) ||
+      process.env.GITHUB_PR_LABELS.includes('ci:all-cypress-suites')
+    ) {
+      pipeline.push(getPipeline('.buildkite/pipelines/pull_request/osquery_cypress.yml'));
+    }
+
     if (await doAnyChangesMatch([/^x-pack\/plugins\/uptime/])) {
       pipeline.push(getPipeline('.buildkite/pipelines/pull_request/uptime.yml'));
     }

.buildkite/scripts/steps/functional/osquery_cypress.sh

@@ -0,0 +1,20 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+source .buildkite/scripts/common/util.sh
+
+.buildkite/scripts/bootstrap.sh
+.buildkite/scripts/download_build_artifacts.sh
+
+export JOB=kibana-osquery-cypress
+
+echo "--- Osquery Cypress tests"
+
+cd "$XPACK_DIR"
+
+checks-reporter-with-killswitch "Osquery Cypress Tests" \
+ node scripts/functional_tests \
+   --debug --bail \
+   --kibana-install-dir "$KIBANA_BUILD_LOCATION" \
+   --config test/osquery_cypress/cli_config.ts

.github/CODEOWNERS

@@ -22,7 +22,6 @@
 
 # Vis Editors
 /x-pack/plugins/lens/ @elastic/kibana-vis-editors
-/src/plugins/advanced_settings/ @elastic/kibana-vis-editors
 /src/plugins/charts/ @elastic/kibana-vis-editors
 /src/plugins/vis_default_editor/ @elastic/kibana-vis-editors
 /src/plugins/vis_types/metric/ @elastic/kibana-vis-editors
@@ -263,6 +262,7 @@
 /src/plugins/home/server/*.ts @elastic/kibana-core
 /src/plugins/home/server/services/ @elastic/kibana-core
 /src/plugins/kibana_overview/ @elastic/kibana-core
+/src/plugins/advanced_settings/ @elastic/kibana-core
 /x-pack/plugins/global_search_bar/ @elastic/kibana-core
 #CC# /src/core/server/csp/ @elastic/kibana-core
 #CC# /src/plugins/saved_objects/ @elastic/kibana-core

dev_docs/key_concepts/performance.mdx

@@ -3,11 +3,13 @@ id: kibDevPerformance
 slug: /kibana-dev-docs/key-concepts/performance
 title: Performance
 summary: Performance tips for Kibana development.
-date: 2021-09-02
+date: 2021-12-03
 tags: ['kibana', 'onboarding', 'dev', 'performance']
 ---
 
-## Keep Kibana fast
+## Client-side considerations
+
+### Lazy load code
 
 _tl;dr_: Load as much code lazily as possible. Everyone loves snappy
 applications with a responsive UI and hates spinners. Users deserve the
@@ -105,3 +107,15 @@ Many OSS tools allow you to analyze the generated stats file:
   Webpack authors
 - [webpack-visualizer](https://chrisbateman.github.io/webpack-visualizer/)
 - [webpack-bundle-analyzer](https://github.com/webpack-contrib/webpack-bundle-analyzer)
+
+## Server-side considerations
+
+### Don't block the event loop
+
+[Node.js is single threaded](https://nodejs.dev/learn/introduction-to-nodejs) which means a single CPU-intensive server-side, synchronous operation will block any other functionality waiting to execute on the Kibana server. The affects background tasks, like alerts, and search sessions, as well as search requests and page loads.
+
+**When writing code that will run on the server, [don't block the event loop](https://nodejs.org/en/docs/guides/dont-block-the-event-loop/)**. Instead consider:
+
+- Writing async code. For example, leverage [setImmediate](https://nodejs.dev/learn/understanding-setimmediate) inside for loops.
+- Executing logic on the client instead. This may not be a good option if you require a lot of data going back and forth between the server and the client, as that can also slow down the user's experience, especially over slower bandwidth internet connections.
+- Worker threads are also an option if the code doesn't rely on stateful Kibana services. If you are interested in using worker threads, please reach out to a tech-lead before doing so. We will likely want to implement a worker threads pool to ensure worker threads cooperate appropriately.

dev_docs/tutorials/saved_objects.mdx

@@ -252,6 +252,8 @@ Having said that, if a document is encountered that is not in the expected shape
 fail an upgrade than to silently ignore a corrupt document which can cause unexpected behaviour at some future point in time. When such a scenario is encountered,
 the error should be verbose and informative so that the corrupt document can be corrected, if possible.
 
+**WARNING:** Do not attempt to change the `migrationVersion`, `id`, or `type` fields within a migration function, this is not supported.
+
 ### Testing Migrations
 
 Bugs in a migration function cause downtime for our users and therefore have a very high impact. Follow the <DocLink id="kibDevTutorialTestingPlugins" section="saved-objects-migrations" text="Saved Object migrations section in the plugin testing guide"/>.

docs/developer/architecture/core/saved-objects-service.asciidoc

@@ -259,6 +259,9 @@ upgrade. In most scenarios, it is better to fail an upgrade than to silently
 ignore a corrupt document which can cause unexpected behaviour at some future
 point in time.
 
+WARNING: Do not attempt to change the `migrationVersion`, `id`, or `type` fields
+within a migration function, this is not supported.
+
 It is critical that you have extensive tests to ensure that migrations behave
 as expected with all possible input documents. Given how simple it is to test
 all the branch conditions in a migration function and the high impact of a bug

docs/development/core/public/kibana-plugin-core-public.scopedhistory._constructor_.md

@@ -9,13 +9,13 @@ Constructs a new instance of the `ScopedHistory` class
 <b>Signature:</b>
 
 ```typescript
-constructor(parentHistory: History, basePath: string);
+constructor(parentHistory: History<HistoryLocationState>, basePath: string);
 ```
 
 ## Parameters
 
 |  Parameter | Type | Description |
 |  --- | --- | --- |
-|  parentHistory | History |  |
+|  parentHistory | History&lt;HistoryLocationState&gt; |  |
 |  basePath | string |  |
 

docs/development/core/public/kibana-plugin-core-public.scopedhistory.block.md

@@ -9,5 +9,5 @@ Add a block prompt requesting user confirmation when navigating away from the cu
 <b>Signature:</b>
 
 ```typescript
-block: (prompt?: string | boolean | History.TransitionPromptHook<HistoryLocationState> | undefined) => UnregisterCallback;
+block: (prompt?: string | boolean | TransitionPromptHook<HistoryLocationState> | undefined) => UnregisterCallback;
 ```

docs/development/core/public/kibana-plugin-core-public.scopedhistory.createsubhistory.md

@@ -9,5 +9,5 @@ Creates a `ScopedHistory` for a subpath of this `ScopedHistory`<!-- -->. Useful
 <b>Signature:</b>
 
 ```typescript
-createSubHistory: <SubHistoryLocationState = unknown>(basePath: string) => ScopedHistory<SubHistoryLocationState>;
+createSubHistory: (basePath: string) => ScopedHistory<HistoryLocationState>;
 ```

docs/development/core/public/kibana-plugin-core-public.scopedhistory.md

@@ -28,9 +28,9 @@ export declare class ScopedHistory<HistoryLocationState = unknown> implements Hi
 |  Property | Modifiers | Type | Description |
 |  --- | --- | --- | --- |
 |  [action](./kibana-plugin-core-public.scopedhistory.action.md) |  | Action | The last action dispatched on the history stack. |
-|  [block](./kibana-plugin-core-public.scopedhistory.block.md) |  | (prompt?: string \| boolean \| History.TransitionPromptHook&lt;HistoryLocationState&gt; \| undefined) =&gt; UnregisterCallback | Add a block prompt requesting user confirmation when navigating away from the current page. |
+|  [block](./kibana-plugin-core-public.scopedhistory.block.md) |  | (prompt?: string \| boolean \| TransitionPromptHook&lt;HistoryLocationState&gt; \| undefined) =&gt; UnregisterCallback | Add a block prompt requesting user confirmation when navigating away from the current page. |
 |  [createHref](./kibana-plugin-core-public.scopedhistory.createhref.md) |  | (location: LocationDescriptorObject&lt;HistoryLocationState&gt;, { prependBasePath }?: { prependBasePath?: boolean \| undefined; }) =&gt; Href | Creates an href (string) to the location. If <code>prependBasePath</code> is true (default), it will prepend the location's path with the scoped history basePath. |
-|  [createSubHistory](./kibana-plugin-core-public.scopedhistory.createsubhistory.md) |  | &lt;SubHistoryLocationState = unknown&gt;(basePath: string) =&gt; ScopedHistory&lt;SubHistoryLocationState&gt; | Creates a <code>ScopedHistory</code> for a subpath of this <code>ScopedHistory</code>. Useful for applications that may have sub-apps that do not need access to the containing application's history. |
+|  [createSubHistory](./kibana-plugin-core-public.scopedhistory.createsubhistory.md) |  | (basePath: string) =&gt; ScopedHistory&lt;HistoryLocationState&gt; | Creates a <code>ScopedHistory</code> for a subpath of this <code>ScopedHistory</code>. Useful for applications that may have sub-apps that do not need access to the containing application's history. |
 |  [go](./kibana-plugin-core-public.scopedhistory.go.md) |  | (n: number) =&gt; void | Send the user forward or backwards in the history stack. |
 |  [goBack](./kibana-plugin-core-public.scopedhistory.goback.md) |  | () =&gt; void | Send the user one location back in the history stack. Equivalent to calling [ScopedHistory.go(-1)](./kibana-plugin-core-public.scopedhistory.go.md)<!-- -->. If no more entries are available backwards, this is a no-op. |
 |  [goForward](./kibana-plugin-core-public.scopedhistory.goforward.md) |  | () =&gt; void | Send the user one location forward in the history stack. Equivalent to calling [ScopedHistory.go(1)](./kibana-plugin-core-public.scopedhistory.go.md)<!-- -->. If no more entries are available forwards, this is a no-op. |

docs/development/core/server/kibana-plugin-core-server.kibanaresponsefactory.md

@@ -12,14 +12,32 @@ Set of helpers used to create `KibanaResponse` to form HTTP response on an incom
 kibanaResponseFactory: {
     custom: <T extends string | Record<string, any> | Error | Buffer | Stream | {
         message: string | Error;
-        attributes?: Record<string, any> | undefined;
+        attributes?: ResponseErrorAttributes | undefined;
     } | undefined>(options: CustomHttpResponseOptions<T>) => KibanaResponse<T>;
-    badRequest: (options?: ErrorHttpResponseOptions) => KibanaResponse<ResponseError>;
-    unauthorized: (options?: ErrorHttpResponseOptions) => KibanaResponse<ResponseError>;
-    forbidden: (options?: ErrorHttpResponseOptions) => KibanaResponse<ResponseError>;
-    notFound: (options?: ErrorHttpResponseOptions) => KibanaResponse<ResponseError>;
-    conflict: (options?: ErrorHttpResponseOptions) => KibanaResponse<ResponseError>;
-    customError: (options: CustomHttpResponseOptions<ResponseError>) => KibanaResponse<ResponseError>;
+    badRequest: (options?: ErrorHttpResponseOptions) => KibanaResponse<string | Error | {
+        message: string | Error;
+        attributes?: ResponseErrorAttributes | undefined;
+    }>;
+    unauthorized: (options?: ErrorHttpResponseOptions) => KibanaResponse<string | Error | {
+        message: string | Error;
+        attributes?: ResponseErrorAttributes | undefined;
+    }>;
+    forbidden: (options?: ErrorHttpResponseOptions) => KibanaResponse<string | Error | {
+        message: string | Error;
+        attributes?: ResponseErrorAttributes | undefined;
+    }>;
+    notFound: (options?: ErrorHttpResponseOptions) => KibanaResponse<string | Error | {
+        message: string | Error;
+        attributes?: ResponseErrorAttributes | undefined;
+    }>;
+    conflict: (options?: ErrorHttpResponseOptions) => KibanaResponse<string | Error | {
+        message: string | Error;
+        attributes?: ResponseErrorAttributes | undefined;
+    }>;
+    customError: (options: CustomHttpResponseOptions<ResponseError>) => KibanaResponse<string | Error | {
+        message: string | Error;
+        attributes?: ResponseErrorAttributes | undefined;
+    }>;
     redirected: (options: RedirectResponseOptions) => KibanaResponse<string | Record<string, any> | Buffer | Stream>;
     ok: (options?: HttpResponseOptions) => KibanaResponse<string | Record<string, any> | Buffer | Stream>;
     accepted: (options?: HttpResponseOptions) => KibanaResponse<string | Record<string, any> | Buffer | Stream>;

docs/settings/alert-action-settings.asciidoc

@@ -126,7 +126,7 @@ into a single string.  This configuration can be used for environments where
 the files cannot be made available.
 
 `xpack.actions.enabledActionTypes` {ess-icon}::
-A list of action types that are enabled. It defaults to `[*]`, enabling all types. The names for built-in {kib} action types are prefixed with a `.` and include: `.server-log`, `.slack`, `.email`, `.index`, `.pagerduty`, and `.webhook`. An empty list `[]` will disable all action types.
+A list of action types that are enabled. It defaults to `[*]`, enabling all types. The names for built-in {kib} action types are prefixed with a `.` and include: `.email`, `.index`, `.jira`, `.pagerduty`, `.resilient`, `.server-log`, `.servicenow`, .`servicenow-itom`, `.servicenow-sir`, `.slack`, `.swimlane`, `.teams`, and `.webhook`. An empty list `[]` will disable all action types.
 +
 Disabled action types will not appear as an option when creating new connectors, but existing connectors and actions of that type will remain in {kib} and will not function.
 

docs/setup/docker.asciidoc

@@ -14,10 +14,13 @@ https://github.com/elastic/dockerfiles/tree/{branch}/kibana[GitHub].
 These images contain both free and subscription features.
 <<managing-licenses,Start a 30-day trial>> to try out all of the features.
 
-[float]
+[discrete]
 [[run-kibana-on-docker-for-dev]]
 === Run {kib} on Docker for development
 
+. Start an {es} container for development or testing:
++
+--
 ifeval::["{release-state}"=="unreleased"]
 
 NOTE: No Docker images are currently available for {kib} {version}.
@@ -26,14 +29,16 @@ endif::[]
 
 ifeval::["{release-state}"!="unreleased"]
 
-. Start an {es} container for development or testing:
-+
 [source,sh,subs="attributes"]
 ----
 docker network create elastic
 docker pull {es-docker-image}
 docker run --name es-node01 --net elastic -p 9200:9200 -p 9300:9300 -t {es-docker-image}
 ----
+
+endif::[]
+
+--
 +
 When you start {es} for the first time, the following security configuration 
 occurs automatically:
@@ -51,30 +56,26 @@ and enrollment token.
 . Copy the generated password and enrollment token and save them in a secure 
 location. These values are shown only when you start {es} for the first time.
 You'll use these to enroll {kib} with your {es} cluster and log in.
+
+. In a new terminal session, start {kib} and connect it to your {es} container:
 +
-[NOTE]
-====
-If you need to reset the password for the `elastic` user or other
-built-in users, run the {ref}/reset-password.html[`elasticsearch-reset-password`]
-tool. To generate new enrollment tokens for {kib} or {es} nodes, run the
-{ref}/create-enrollment-token.html[`elasticsearch-create-enrollment-token`] tool.
-These tools are available in the {es} `bin` directory of the Docker container.
+--
+ifeval::["{release-state}"=="unreleased"]
 
-For example:
+NOTE: No Docker images are currently available for {kib} {version}.
 
-[source,sh]
-----
-docker exec -it es-node01 /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic
-----
-====
+endif::[]
+
+ifeval::["{release-state}"!="unreleased"]
 
-. In a new terminal session, start {kib} and connect it to your {es} container:
-+
 [source,sh,subs="attributes"]
 ----
 docker pull {docker-image}
 docker run --name kib-01 --net elastic -p 5601:5601 {docker-image}
 ----
+
+endif::[]
+--
 +
 When you start {kib}, a unique link is output to your terminal.
 
@@ -86,7 +87,32 @@ When you start {kib}, a unique link is output to your terminal.
   .. Log in to {kib} as the `elastic` user with the password that was generated
   when you started {es}.
 
-[float]
+[[docker-generate]]
+[discrete]
+=== Generate passwords and enrollment tokens
+If you need to reset the password for the `elastic` user or other
+built-in users, run the {ref}/reset-password.html[`elasticsearch-reset-password`]
+tool. This tool is available in the {es} `bin` directory of the Docker container.
+
+For example, to reset the password for the `elastic` user:
+
+[source,sh]
+----
+docker exec -it es-node01 /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic
+----
+
+If you need to generate new enrollment tokens for {kib} or {es} nodes, run the
+{ref}/create-enrollment-token.html[`elasticsearch-create-enrollment-token`] tool.
+This tool is available in the {es} `bin` directory of the Docker container.
+
+For example, to generate a new enrollment token for {kib}:
+
+[source,sh]
+----
+docker exec -it es-node01 /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana
+----
+
+[discrete]
 === Remove Docker containers
 
 To remove the containers and their network, run:
@@ -98,8 +124,7 @@ docker rm es-node01
 docker rm kib-01
 ----
 
-endif::[]
-[float]
+[discrete]
 [[configuring-kibana-docker]]
 === Configure Kibana on Docker
 
@@ -108,7 +133,7 @@ conventional approach is to provide a `kibana.yml` file as described in
 {kibana-ref}/settings.html[Configuring Kibana], but it's also possible to use
 environment variables to define settings.
 
-[float]
+[discrete]
 [[bind-mount-config]]
 ==== Bind-mounted configuration
 
@@ -135,7 +160,7 @@ docker run -it --rm -v full_path_to/config:/usr/share/kibana/config -v full_path
 docker run -it --rm -v full_path_to/config:/usr/share/kibana/config -v full_path_to/data:/usr/share/kibana/data {docker-image} bin/kibana-keystore add test_keystore_setting
 ----
 
-[float]
+[discrete]
 [[environment-variable-config]]
 ==== Environment variable configuration
 
@@ -179,7 +204,7 @@ services:
 Since environment variables are translated to CLI arguments, they take
 precedence over settings configured in `kibana.yml`.
 
-[float]
+[discrete]
 [[docker-defaults]]
 ==== Docker defaults
 The following settings have different default values when using the Docker

docs/user/introduction.asciidoc

@@ -233,7 +233,7 @@ To get the most from the search feature, follow these tips:
 |Search by type
 |`type:dashboard`
 
-Available types: `application`, `canvas-workpad`, `dashboard`, `index-pattern`, `lens`, `maps`, `query`, `search`, `visualization`
+Available types: `application`, `canvas-workpad`, `dashboard`, `data-view`, `lens`, `maps`, `query`, `search`, `visualization`
 
 |Search by tag
 |`tag:mytagname` +

jest.config.integration.js

@@ -6,8 +6,22 @@
  * Side Public License, v 1.
  */
 
+const Fs = require('fs');
+const Path = require('path');
+
 module.exports = {
   preset: '@kbn/test/jest_integration',
   rootDir: '.',
-  roots: ['<rootDir>/src', '<rootDir>/packages'],
+  roots: [
+    '<rootDir>/src',
+    '<rootDir>/packages',
+    ...Fs.readdirSync(Path.resolve(__dirname, 'x-pack')).flatMap((name) => {
+      // create roots for all x-pack/* dirs except for test
+      if (name !== 'test' && Fs.statSync(Path.resolve(__dirname, 'x-pack', name)).isDirectory()) {
+        return [`<rootDir>/x-pack/${name}`];
+      }
+
+      return [];
+    }),
+  ],
 };