merge with main

.buildkite/pipelines/flaky_tests/runner.js

@@ -116,7 +116,7 @@ steps.push({
   label: 'Build Kibana Distribution and Plugins',
   agents: { queue: 'c2-8' },
   key: 'build',
-  if: "build.env('BUILD_ID_FOR_ARTIFACTS') == null || build.env('BUILD_ID_FOR_ARTIFACTS') == ''",
+  if: "build.env('KIBANA_BUILD_ID') == null || build.env('KIBANA_BUILD_ID') == ''",
 });
 
 for (const testSuite of testSuites) {

dev_docs/contributing/best_practices.mdx

@@ -9,9 +9,9 @@ tags: ['kibana', 'onboarding', 'dev', 'architecture']
 
 ## General
 
-First things first, be sure to review our <DocLink id="kibDevPrinciples" text="development principles"/> and check out all the available
-platform <DocLink id="kibBuildingBlocks" text="building blocks"/> that can simplify plugin development.
-
+Be sure to follow our <DocLink id="kibDevPrinciples" text="development principles"/>
+and <DocLink id="kibStandards" text="standards and guidelines"/>.
+ 
 ## Documentation
 
 Documentation best practices can be found <DocLink id="kibDocumentation" text="here"/>.
@@ -54,11 +54,27 @@ We use es-lint rules when possible, but please review our [styleguide](https://g
 
 Es-lint overrides on a per-plugin level are discouraged.
 
-## Plugin best practices
+## Using the SavedObjectClient
 
-Don't export <DocLink id="kibPlatformIntro" section="public-plugin-api" text="public APIs"/> without reason. Make your public APIs as small as possible. You will have to maintain them, and consider backward compatibility when making changes.
+The <DocLink id="kibCoreSavedObjectsPluginApi" section="def-public.SavedObjectsClient" text="SavedObjectClient" /> should always be used for reading and writing saved objects that you manage. For saved objects managed by other plugins, their plugin APIs should be used instead.
+
+Good:
+```
+const dataView = dataViewStartContract.get(dataViewId);
+```
+
+Bad:
+```
+const dataView = savedObjectsClient.get(dataViewId) as DataView;
+```
+
+## Resusable react components
 
-Add `README.md` to all your plugins and services and include contact information.
+Use [EUI](https://elastic.github.io/eui) for all your basic UI components to create a consistent UI experience. We also have generic UI components offered from the <DocLink id="kibKibanaReactPluginApi" text="kibana_react" /> plugin and the <DocLink id="kibSharedUXPluginApi" text="shared_ux" /> plugin.
+
+## Don't export code that doesn't need to be public
+
+Don't export <DocLink id="kibPlatformIntro" section="public-plugin-api" text="public APIs"/> without reason. Make your public APIs as small as possible. You will have to maintain them, and consider backward compatibility when making changes.
 
 ## Re-inventing the wheel
 
@@ -120,6 +136,77 @@ There are some exceptions where a separate repo makes sense. However, they are e
 
 It may be tempting to get caught up in the dream of writing the next package which is published to npm and downloaded millions of times a week. Knowing the quality of developers that are working on Kibana, this is a real possibility. However, knowing which packages will see mass adoption is impossible to predict. Instead of jumping directly to writing code in a separate repo and accepting all of the complications that come along with it, prefer keeping code inside the Kibana repo. A [Kibana package](https://github.com/elastic/kibana/tree/main/packages) can be used to publish a package to npm, while still keeping the code inside the Kibana repo. Move code to an external repo only when there is a good reason, for example to enable external contributions.
 
+## Licensing
+
+<DocCallOut title="Internal only">
+
+Has there been a discussion about which license this feature should be available under? Open up a license issue in [https://github.com/elastic/dev](https://github.com/elastic/dev) if you are unsure.
+
+</DocCallOut>
+
+## Testing scenarios
+
+Every PR submitted should be accompanied by tests. Read through the <DocLink id="kibDevTutorialTestingPlugins" text="testing plugins tutorial" /> for how to test.
+
+### Browser coverage
+
+Refer to the list of browsers and OS Kibana supports https://www.elastic.co/support/matrix
+
+Does the feature work efficiently on the below listed browsers
+  - chrome
+  - Firefox
+  - Safari
+  - IE11
+
+### Upgrade Scenarios
+  - Migration scenarios- Does the feature affect old indices, saved objects ?
+  - Has the feature been tested with Kibana aliases
+  - Read/Write privileges of the indices before and after the upgrade?
+
+### Test coverage
+  - Does the feature have sufficient unit test coverage? (does it handle storeinSessions?)
+  - Does the feature have sufficient Functional UI  test coverage?
+  - Does the feature have sufficient Rest API coverage test coverage?
+  - Does the feature have sufficient Integration test coverage?
+
+### Environment configurations
+
+- Kibana should be fully [cross cluster search](https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-cross-cluster-search.html) compatible (aside from admin UIs which only work on the local cluster).
+- How does your plugin behave when optional dependencies are disabled?
+- How does your app behave under anonymous access, or with security disabled?
+- Make sure to test your PR in a cloud environment. Read about the <DocLink id="kibDevTutorialCI" section="labels" text="ci:deploy cloud" /> label which makes this very easy.
+
+
+## Backward compatibility
+
+Any time you change state that is part of a Saved Object you will have to write a <DocLink id="kibDevDocsSavedObjectsIntro" section="migrations-and-backward-compatibility" text="migration" />.
+
+Never store state from another plugin in your Saved Objects or URLs unless it implements the <DocLink id="kibDevDocsPersistableStateIntro" text="persistable state interface"/>. Remember to check for migrations when deserializing that state.
+
+If you expose state and you wish to allow other plugins to persist you must ensure it implements the <DocLink id="kibDevDocsPersistableStateIntro" text="persistable state interface"/>. This is very common for `by value` entities, like visualizations that exist on a dashboard but are not part of the visualization library. If you make a breaking change to this state you must remember to register a migration for it.
+
+Saved objects exported from past Kibana versions should always continue to work. Bookmarked URLs should also always work. Check out <DocLink id="kibDevKeyConceptsNavigation" section="specifying-state" text="URL Locators" /> to learn about migrating state in URLs.
+
+## Avoid these common mistakes
+
+### Treating Kibana's filesystem as durable storage
+
+Plugins should rarely, if ever, access Kibana's filesystem directly. Kibana instances are commonly ephemeral and anything written to the filesystem will potentially
+not be there on restart.
+
+### Storing state in server memory
+
+There are generally multiple instances of Kibana all hosted behind a round-robin load-balancer. As a result, storing state in server memory is risky as there is no
+guarantee that a single end-user's HTTP requests will be served by the same Kibana instance.
+
+### Using WebSockets
+
+Kibana has a number of platform services that don't work with WebSockets, for example authentication and authorization. If your use-case would benefit substantially
+from websockets, talk to the Kibana Core team about adding support. Do not hack around this limitation, everytime that someone has, it's created so many problems
+it's been eventually removed.
+
+
+
 ## Security best practices
 
 When writing code for Kibana, be sure to follow these best practices to avoid common vulnerabilities. Refer to the included Open Web

docs/management/cases/add-connectors.asciidoc

@@ -0,0 +1,56 @@
+[[add-case-connectors]]
+== Add connectors
+
+preview::[]
+
+You can add connectors to cases to push information to these external incident
+management systems:
+
+* IBM Resilient
+* Jira
+* ServiceNow ITSM
+* ServiceNow SecOps
+* {swimlane}
+
+NOTE: To create connectors and send cases to external systems, you must have the
+appropriate {kib} feature privileges. Refer to <<setup-cases>>.
+
+[discrete]
+[[create-case-connectors]]
+== Create connectors
+
+You can create connectors in *Management > {stack-manage-app} > {rules-ui}*, as
+described in <<action-types>>. Alternatively, you can create them in
+*Management > {stack-manage-app} > Cases*:
+
+. Click *Edit external connection*.
++
+[role="screenshot"]
+image::images/cases-connectors.png[]
+
+. From the *Incident management system* list, select *Add new connector*.
+
+. Select an external incident management system.
+
+. Enter your required settings. Refer to <<resilient-action-type>>,
+<<jira-action-type>>, <<servicenow-action-type>>, <<servicenow-sir-action-type>>,
+or <<swimlane-action-type>> for connector configuration details.
+
+. Click *Save*.
+
+[discrete]
+[[edit-case-connector-settings]]
+== Edit connector settings
+
+You can create additional connectors, update existing connectors, change
+the default connector, and change case closure options.
+
+. Go to *Management > {stack-manage-app} > Cases*, click *Edit external connection*.
+
+. To change whether cases are automatically closed after they are sent to an
+external system, update the case closure options.
+
+. To change the default connector for new cases, select the connector from the
+*Incident management system* list.
+
+. To update a connector, click *Update <connector name>* and edit the connector fields as required.

docs/management/cases/cases.asciidoc

@@ -5,7 +5,8 @@ preview::[]
 
 Cases are used to open and track issues directly in {kib}. All cases list
 the original reporter and all the users who contribute to a case (_participants_).
-You can also send cases to third party systems by configuring external connectors.
+You can also send cases to external incident management systems by configuring
+connectors.
 
 [role="screenshot"]
 image::images/cases.png[Cases page]
@@ -17,4 +18,5 @@ You also cannot attach alerts from the {observability} or {security-app} to
 cases in *{stack-manage-app}*.
 
 * <<setup-cases>>
-* <<manage-cases>>
+* <<manage-cases>>
+* <<add-case-connectors>>

docs/management/cases/index.asciidoc

@@ -1,4 +1,4 @@
 include::cases.asciidoc[]
 include::setup-cases.asciidoc[leveloffset=+1]
 include::manage-cases.asciidoc[leveloffset=+1]
-//=== Configure external connectors
+include::add-connectors.asciidoc[leveloffset=+1]

docs/management/cases/manage-cases.asciidoc

@@ -16,9 +16,8 @@ TIP: In the `Description` area, you can use
 https://www.markdownguide.org/cheat-sheet[Markdown] syntax to create formatted
 text.
 
-. For *External incident management system*, select a connector. If you've
-previously added one, that connector displays as the default selection.
-Otherwise, the default setting is `No connector selected`.
+. For *External incident management system*, select a connector. For more
+information, refer to <<add-case-connectors>>.
 
 . After you've completed all of the required fields, click *Create case*.
 

docs/setup/configuring-reporting.asciidoc

@@ -6,7 +6,16 @@
 <titleabbrev>Configure reporting</titleabbrev>
 ++++
 
-To enable users to manually and automatically generate reports, install the reporting packages, grant users access to the {report-features}, and secure the reporting endpoints.
+For security, you grant users access to the {report-features} and secure the reporting endpoints
+with TLS/SSL encryption. Additionally, you can install graphical packages into the operating system
+to enable the {kib} server to have screenshotting capabilities.
+
+* <<install-reporting-packages>>
+* <<grant-user-access>>
+* <<reporting-roles-user-api>>
+* <<grant-user-access-basic>>
+* <<grant-user-access-external-provider>>
+* <<securing-reporting>>
 
 [float]
 [[install-reporting-packages]]
@@ -32,7 +41,7 @@ If you are using Ubuntu/Debian systems, install the following packages:
 * `libfontconfig1`
 * `libnss3`
 
-If the system is missing dependencies, *Reporting* fails in a non-deterministic way. {kib} runs a self-test at server startup, and
+If the system is missing dependencies, a screenshot report job may fail in a non-deterministic way. {kib} runs a self-test at server startup, and
 if it encounters errors, logs them in the Console. The error message does not include
 information about why Chromium failed to run. The most common error message is `Error: connect ECONNREFUSED`, which indicates
 that {kib} could not connect to the Chromium process.
@@ -53,7 +62,7 @@ xpack.reporting.roles.enabled: false
 +
 NOTE: If you use the default settings, you can still create a custom role that grants reporting privileges. The default role is `reporting_user`. This behavior is being deprecated and does not allow application-level access controls for {report-features}, and does not allow API keys or authentication tokens to authorize report generation. Refer to <<reporting-advanced-settings, reporting security settings>> for information and caveats about the deprecated access control features.
 
-. Create the reporting role. 
+. Create the reporting role.
 
 .. Open the main menu, then click *Stack Management*.
 
@@ -77,14 +86,13 @@ For more information, refer to {ref}/security-privileges.html[Security privilege
 
 .. Click *Customize*, then click *Analytics*.
 
-.. Next each application listed, click *All* or click *Read*. You will need to enable the *Customize sub-feature
-privileges* checkbox to grant reporting privileges if you select *Read*.
+.. For each application, select *All*, or to customize the privileges, select *Read* and *Customize sub-feature privileges*.
 +
-If you’ve followed the example above, you should end up on a screen defining your customized privileges that looks like this:
+NOTE: If you have a Basic license, sub-feature privileges are unavailable. For details, check out <<grant-user-access-basic>>.
 [role="screenshot"]
-image::user/reporting/images/kibana-privileges-with-reporting.png["Kibana privileges with Reporting options"]
+image::user/reporting/images/kibana-privileges-with-reporting.png["Kibana privileges with Reporting options, Gold or higher license"]
 +
-NOTE: If *Reporting* options for application features are not available, contact your administrator, or <<reporting-advanced-settings,check that xpack.reporting.roles.enabled is set to false in kibana.yml>>. 
+NOTE: If the *Reporting* options for application features are unavailable, and the cluster license is higher than Basic, contact your administrator, or <<reporting-advanced-settings,check that `xpack.reporting.roles.enabled` is set to `false` in kibana.yml>>.
 
 .. Click *Add {kib} privilege*.
 
@@ -94,7 +102,7 @@ NOTE: If *Reporting* options for application features are not available, contact
 
 .. Open the main menu, then click *Stack Management*.
 
-.. Click *Users*, then click the user you want to assign the reporting role to. 
+.. Click *Users*, then click the user you want to assign the reporting role to.
 
 .. From the *Roles* dropdown, select *custom_reporting_user*.
 
@@ -105,29 +113,43 @@ Granting the privilege to generate reports also grants the user the privilege to
 [float]
 [[reporting-roles-user-api]]
 ==== Grant access with the role API
-With <<grant-user-access, {kib} application privileges>> enabled in Reporting, you can also use the {ref}/security-api-put-role.html[role API] to grant access to the {report-features}. Grant custom reporting roles to users in combination with other roles that grant read access to the data in {es}, and at least read access in the applications where users can generate reports.
+With <<grant-user-access, {kib} application privileges>> enabled in Reporting, you can also use the {ref}/security-api-put-role.html[role API] to grant access to the {report-features}, using *All* privileges, or sub-feature privileges.
 
-[source, sh]
+NOTE: If you have a Basic license, sub-feature privileges are unavailable. For details, check out the API command to grant *All* privileges in <<grant-user-access-basic>>.
+
+Grant users custom Reporting roles, other roles that grant read access to the data in {es}, and at least read access in the applications where users can generate reports.
+
+[source, json]
 ---------------------------------------------------------------
-POST /_security/role/custom_reporting_user
+PUT localhost:5601/api/security/role/custom_reporting_user
 {
-  metadata: {},
-  elasticsearch: { cluster: [], indices: [], run_as: [] },
-  kibana: [
+  "elasticsearch": { "cluster": [], "indices": [], "run_as": [] },
+  "kibana": [
     {
-      base: [],
-      feature: {
-        dashboard: [
-          'generate_report', <1>
-          'download_csv_report' <2>
+      "base": [],
+      "feature": {
+        "dashboard": [
+          "minimal_read",
+          "generate_report", <1>
+          "download_csv_report" <2>
+        ],
+        "discover": [
+          "minimal_read",
+          "generate_report" <3>
+        ],
+        "canvas": [
+          "minimal_read",
+          "generate_report" <4>
         ],
-        discover: ['generate_report'], <3>
-        canvas: ['generate_report'], <4>
-        visualize: ['generate_report'], <5>
+        "visualize": [
+          "minimal_read",
+          "generate_report" <5>
+        ]
       },
-      spaces: ['*'],
+      "spaces": [ "*" ]
     }
-  ]
+  ],
+  "metadata": {} // optional
 }
 ---------------------------------------------------------------
 // CONSOLE
@@ -139,6 +161,41 @@ POST /_security/role/custom_reporting_user
 <5> Grants access to generate PNG and PDF reports in *Visualize Library*.
 
 [float]
+[[grant-user-access-basic]]
+=== Grant users access with a Basic license
+
+With a Basic license, you can grant users access with custom roles to {report-features} with <<kibana-privileges, {kib} application privileges>>. However, with a Basic license, sub-feature privileges are unavailable. <<grant-user-access,Create a role>>, then select *All* privileges for the applications where users can create reports.
+
+[role="screenshot"]
+image::user/reporting/images/kibana-privileges-with-reporting-basic.png["Kibana privileges with Reporting options, Basic license"]
+
+With a Basic license, sub-feature application privileges are unavailable, but you can use the {ref}/security-api-put-role.html[role API] to grant access to CSV {report-features}:
+
+[source, sh]
+---------------------------------------------------------------
+PUT localhost:5601/api/security/role/custom_reporting_user
+{
+  "elasticsearch": { "cluster": [], "indices": [], "run_as": [] },
+  "kibana": [
+    {
+      "base": [],
+      "feature": {
+        "dashboard": [ "all" ], <1>
+        "discover": [ "all" ], <2>
+      },
+      "spaces": [ "*" ]
+    }
+  ],
+  "metadata": {} // optional
+}
+---------------------------------------------------------------
+// CONSOLE
+
+<1> Grants access to generate CSV reports from saved searches in *Discover*.
+<2> Grants access to download CSV reports from saved search panels in *Dashboard*.
+
+[float]
+[[grant-user-access-external-provider]]
 ==== Grant access using an external provider
 
 If you are using an external identity provider, such as LDAP or Active Directory, you can assign roles to individual users or groups of users. Role mappings are configured in {ref}/mapping-roles.html[`config/role_mapping.yml`].