Merge branch 'master' into feature/observability-exploratory-view-per…

…centile-breakdowns

docs/management/connectors/action-types/email.asciidoc

@@ -5,7 +5,7 @@
 <titleabbrev>Email</titleabbrev>
 ++++
 
-The email connector uses the SMTP protocol to send mail messages, using an integration of https://nodemailer.com/[Nodemailer]. Email message text is sent as both plain text and html text.
+The email connector uses the SMTP protocol to send mail messages, using an integration of https://nodemailer.com/[Nodemailer]. An exception is Microsoft Exchange, which uses HTTP protocol for sending emails, https://docs.microsoft.com/en-us/graph/api/user-sendmail[Send mail]. Email message text is sent as both plain text and html text.
 
 NOTE: For emails to have a footer with a link back to {kib}, set the <<server-publicBaseUrl, `server.publicBaseUrl`>> configuration setting.
 
@@ -17,9 +17,13 @@ Email connectors have the following configuration properties.
 
 Name::      The name of the connector. The name is used to identify a  connector in the management UI connector listing, or in the connector list when configuring an action.
 Sender::    The from address for all emails sent with this connector. This can be specified in `user@host-name` format or as `"human name <user@host-name>"` format. See the https://nodemailer.com/message/addresses/[Nodemailer address documentation] for more information.
+Service::   The name of the email service. If `service` is one of Nodemailer's https://nodemailer.com/smtp/well-known/[well-known email service providers], the `host`, `port`, and `secure` properties are defined with the default values and disabled for modification. If `service` is `MS Exchange Server`, the `host`, `port`, and `secure` properties are ignored and `tenantId`, `clientId`, `clientSecret` are required instead. If `service` is `other`, the `host` and `port` properties must be defined.
 Host::      Host name of the service provider. If you are using the <<action-settings, `xpack.actions.allowedHosts`>> setting, make sure this hostname is added to the allowed hosts.
 Port::      The port to connect to on the service provider.
 Secure::    If true, the connection will use TLS when connecting to the service provider. Refer to the https://nodemailer.com/smtp/#tls-options[Nodemailer TLS documentation] for more information.  If not true, the connection will initially connect over TCP, then attempt to switch to TLS via the SMTP STARTTLS command.
+Tenant ID:: The directory tenant that the application plans to operate against, in GUID format.
+Client ID:: The application ID that is assigned to your app, in GUID format. You can find this information in the portal where you registered your app.
+Client Secret:: The client secret that you generated for your app in the app registration portal. The client secret must be URL-encoded before being sent. The Basic auth pattern of providing credentials in the Authorization header, per https://datatracker.ietf.org/doc/html/rfc6749#section-2.3.1[RFC 6749], is also supported.
 Require authentication:: If true, a username and password for login type authentication must be provided.
 Username::  Username for login type authentication.
 Password::  Password for login type authentication.
@@ -40,6 +44,7 @@ Use the <<action-settings, Action configuration settings>> to customize connecto
    name: preconfigured-email-connector-type
    actionTypeId: .email
    config:
+     service: other
      from: [email protected]
      host: validhostname
      port: 8080
@@ -51,17 +56,20 @@ Use the <<action-settings, Action configuration settings>> to customize connecto
 
 Config defines information for the connector type.
 
-`service`:: The name of the email service. If `service` is `elastic_cloud` (for Elastic Cloud notifications) or one of Nodemailer's https://nodemailer.com/smtp/well-known/[well-known email service providers], `host`, `port`, and `secure` properties are ignored. If `service` is `other`, `host` and `port` properties must be defined. For more information on the `gmail` service value, see the https://nodemailer.com/usage/using-gmail/[Nodemailer Gmail documentation].
+`service`:: The name of the email service. If `service` is `elastic_cloud` (for Elastic Cloud notifications) or one of Nodemailer's https://nodemailer.com/smtp/well-known/[well-known email service providers], the `host`, `port`, and `secure` properties are ignored. If `service` is `other`,  the `host` and `port` properties must be defined. For more information on the `gmail` service value, refer to https://nodemailer.com/usage/using-gmail/[Nodemailer Gmail documentation]. If `service` is `exchange_server`, the `tenantId`, `clientId`, `clientSecret` properties are required instead of `host` and `port`.
 `from`:: An email address that corresponds to *Sender*.
 `host`:: A string that corresponds to *Host*.
 `port`:: A number that corresponds to *Port*.
 `secure`:: A boolean that corresponds to *Secure*.
 `hasAuth`:: A boolean that corresponds to *Requires authentication*. If `true`, this connector will require values for `user` and `password` inside the secrets configuration. Defaults to `true`.
+`tenantId`:: A GUID format value that corresponds to *Tenant ID*, which is a part of OAuth 2.0 Client Credentials Authentication. 
+`clientId`:: A GUID format value that corresponds to *Client ID*, which is a part of OAuth 2.0 Client Credentials Authentication.
 
 Secrets defines sensitive information for the connector type.
 
 `user`:: A string that corresponds to *Username*. Required if `hasAuth` is set to `true`.
 `password`:: A string that corresponds to *Password*. Should be stored in the <<creating-keystore, {kib} keystore>>. Required if `hasAuth` is set to `true`.
+`clientSecret`:: A string that corresponds to *Client Secret*. Should be stored in the <<creating-keystore, {kib} keystore>>. Required if `service` is set to `exchange_server`, which uses OAuth 2.0 Client Credentials Authentication.
 
 [float]
 [[define-email-ui]]
@@ -91,15 +99,15 @@ Message::       The message text of the email. Markdown format is supported.
 [[configuring-email]]
 ==== Configuring email accounts for well-known services
 
-The email connector can send email using many popular SMTP email services.
+The email connector can send email using many popular SMTP email services and the Microsoft Exchange Graph API.
 
 For more information about configuring the email connector to work with different email systems, refer to:
 
 * <<elasticcloud>>
 * <<gmail>>
 * <<outlook>>
-* <<exchange>>
 * <<amazon-ses>>
+* <<exchange>>
 
 For other email servers, you can check the list of well-known services that Nodemailer supports in the JSON file https://github.com/nodemailer/nodemailer/blob/master/lib/well-known/services.json[well-known/services.json].  The properties of the objects in those files &mdash; `host`, `port`, and `secure` &mdash; correspond to the same email connector configuration properties.  A missing `secure` property in the "well-known/services.json" file is considered `false`.  Typically, `port: 465` uses `secure: true`, and `port: 25` and `port: 587` use `secure: false`.
 
@@ -114,6 +122,9 @@ Use the following connector settings to send email from Elastic Cloud:
 Sender::
 `[email protected]`
 
+Service::
+`elastic_cloud`
+
 Host::
 `dockerhost`
 
@@ -136,9 +147,11 @@ https://mail.google.com[Gmail] SMTP service:
 [source,text]
 --------------------------------------------------
   config:
-    host: smtp.gmail.com
-    port: 465
-    secure: true
+    service: gmail
+    // `host`, `port` and `secure` have the following default values and do not need to set: 
+    // host: smtp.gmail.com
+    // port: 465
+    // secure: true
   secrets:
     user: <username>
     password: <password>
@@ -164,9 +177,11 @@ https://www.outlook.com/[Outlook.com] SMTP service:
 [source,text]
 --------------------------------------------------
 config:
-    host: smtp.office365.com
-    port: 587
-    secure: false
+    service: outlook365
+    // `host`, `port` and `secure` have the following default values and do not need to set: 
+    // host: smtp.office365.com
+    // port: 587
+    // secure: false
 secrets:
     user: <email.address>
     password: <password>
@@ -189,9 +204,11 @@ http://aws.amazon.com/ses[Amazon Simple Email Service] (SES) SMTP service:
 [source,text]
 --------------------------------------------------
 config:
-    host: email-smtp.us-east-1.amazonaws.com <1>
-    port: 465
-    secure: true
+    service: ses
+    // `host`, `port` and `secure` have the following default values and do not need to set: 
+    // host: email-smtp.us-east-1.amazonaws.com <1>
+    // port: 465
+    // secure: true
 secrets:
     user: <username>
     password: <password>
@@ -207,15 +224,15 @@ NOTE:   You must use your Amazon SES SMTP credentials to send email through
         at AWS.
 
 [float]
-[[exchange]]
-==== Sending email from Microsoft Exchange
+[[exchange-basic-auth]]
+==== Sending email from Microsoft Exchange with Basic Authentication
 
-Use the following email connector configuration to send email from Microsoft
-Exchange:
+deprecated:[This Microsoft Exchange configuration is deprecated in 7.16.0, and will be removed later, because Microsoft is deprecating https://docs.microsoft.com/en-us/lifecycle/announcements/exchange-online-basic-auth-deprecated [Basic Authentication]:
 
 [source,text]
 --------------------------------------------------
 config:
+    service: other
     host: <your exchange server>
     port: 465
     secure: true
@@ -229,3 +246,64 @@ secrets:
 <2> Many organizations support use of your email address as your username.
     Check with your system administrator if you receive
     authentication-related failures.
+
+To prepare for the removal of Basic Auth, you must update all existing Microsoft Exchange connectors with the new configuration based on the https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-client-creds-grant-flow[OAuth 2.0 Client Credentials Authentication].
+
+[float]
+[[exchange]]
+==== Sending email from Microsoft Exchange with OAuth 2.0
+
+Before you create an email connector for Microsoft Exchange, you must create and register the client integration application on the https://go.microsoft.com/fwlink/?linkid=2083908[Azure portal]:
+
+[role="screenshot"]
+image::management/connectors/images/exchange-register-app.png[Register client application for MS Exchange]
+
+Next, open *Manage > API permissions*, and then define the permissions for the registered application to send emails. Refer to the https://docs.microsoft.com/en-us/graph/api/user-sendmail?view=graph-rest-1.0&tabs=http#permissions[documentation] for the Microsoft Graph API.
+[role="screenshot"]
+image::management/connectors/images/exchange-api-permissions.png[MS Exchange API permissions]
+
+Add the "Mail.Send" permission for Microsoft Graph.  The permission appears in the list with the status "Not granted for <your Azure active directory>":
+[role="screenshot"]
+image::management/connectors/images/exchange-not-granted.png[MS Exchange "Mail.Send" not granted]
+
+Click *Grant admin consent for <your Azure active directory>*.
+[role="screenshot"]
+image::management/connectors/images/exchange-grant-confirm.png[MS Exchange grant confirmation]
+
+Confirm that the status for the "Mail.Send" permission is now granted.
+[role="screenshot"]
+image::management/connectors/images/exchange-granted.png[MS Exchange grant confirmation]
+
+[float]
+[[exchange-client-secret]]
+===== Configure Microsoft Exchange Client secret
+To configure the Client secret , open *Manage > Certificates & secrets*.
+[role="screenshot"]
+image::management/connectors/images/exchange-secrets.png[MS Exchange secrets configuration]
+
+Add a new client secret, then copy the value and put it to the proper field in the Microsoft Exchange email connector.
+
+[float]
+[[exchange-client-tenant-id]]
+===== Configure Microsoft Exchange Client ID and Tenant ID
+To find the application Client ID,  open the *Overview* page.
+[role="screenshot"]
+image::management/connectors/images/exchange-client-tenant.png[MS Exchange Client ID and Tenant ID configuration]
+
+Copy and paste this values to the proper fields in the Microsoft Exchange email connector.
+
+Use the following email connector configuration to send email from Microsoft Exchange:
+[source,text]
+--------------------------------------------------
+config:
+    service: exchange_server
+    clientId: <The Application (client) ID> <1>
+    tenantId: <The directory tenant ID, in GUID format.>
+    from: <email address of service account> <2>
+secrets:
+    clientSecret: <URL-encoded string>
+--------------------------------------------------
+<1> This application information is on the https://go.microsoft.com/fwlink/?linkid=2083908[Azure portal – App registrations].
+<2> Some organizations configure Exchange to validate that the `from` field is a
+    valid local email account.
+

packages/kbn-monaco/src/esql/index.ts

@@ -6,7 +6,7 @@
  * Side Public License, v 1.
  */
 
-import { LangModule as LangModuleType } from '../types';
+import { LangModuleType } from '../types';
 import { ID } from './constants';
 import { lexerRules } from './lexer_rules';
 

packages/kbn-monaco/src/helpers.ts

@@ -6,7 +6,7 @@
  * Side Public License, v 1.
  */
 import { monaco } from './monaco_imports';
-import { LangModule as LangModuleType } from './types';
+import { LangModuleType } from './types';
 
 function registerLanguage(language: LangModuleType) {
   const { ID, lexerRules, languageConfiguration } = language;

packages/kbn-monaco/src/index.ts

@@ -11,14 +11,11 @@ import './register_globals';
 
 export { monaco } from './monaco_imports';
 export { XJsonLang } from './xjson';
-export { PainlessLang, PainlessContext, PainlessAutocompleteField } from './painless';
+export * from './painless';
 /* eslint-disable-next-line @kbn/eslint/module_migration */
 import * as BarePluginApi from 'monaco-editor/esm/vs/editor/editor.api';
 
 import { registerLanguage } from './helpers';
-import {
-  LangModule as LangModuleType,
-  CompleteLangModule as CompleteLangModuleType,
-} from './types';
 
-export { BarePluginApi, registerLanguage, LangModuleType, CompleteLangModuleType };
+export { BarePluginApi, registerLanguage };
+export * from './types';

packages/kbn-monaco/src/painless/index.ts

@@ -9,7 +9,7 @@
 import { ID } from './constants';
 import { lexerRules, languageConfiguration } from './lexer_rules';
 import { getSuggestionProvider, getSyntaxErrors } from './language';
-import { CompleteLangModule as CompleteLangModuleType } from '../types';
+import { CompleteLangModuleType } from '../types';
 
 export const PainlessLang: CompleteLangModuleType = {
   ID,
@@ -19,4 +19,4 @@ export const PainlessLang: CompleteLangModuleType = {
   getSyntaxErrors,
 };
 
-export { PainlessContext, PainlessAutocompleteField } from './types';
+export * from './types';

packages/kbn-monaco/src/types.ts

@@ -7,15 +7,15 @@
  */
 import { monaco } from './monaco_imports';
 
-export interface LangModule {
+export interface LangModuleType {
   ID: string;
   lexerRules: monaco.languages.IMonarchLanguage;
   languageConfiguration?: monaco.languages.LanguageConfiguration;
   getSuggestionProvider?: Function;
   getSyntaxErrors?: Function;
 }
 
-export interface CompleteLangModule extends LangModule {
+export interface CompleteLangModuleType extends LangModuleType {
   languageConfiguration: monaco.languages.LanguageConfiguration;
   getSuggestionProvider: Function;
   getSyntaxErrors: Function;

packages/kbn-monaco/src/xjson/index.ts

@@ -12,6 +12,6 @@
 import './language';
 import { ID } from './constants';
 import { lexerRules, languageConfiguration } from './lexer_rules';
-import { LangModule as LangModuleType } from '../types';
+import { LangModuleType } from '../types';
 
 export const XJsonLang: LangModuleType = { ID, lexerRules, languageConfiguration };

src/core/public/doc_links/doc_links_service.ts

@@ -308,6 +308,8 @@ export class DocLinksService {
           apmRules: `${KIBANA_DOCS}apm-alerts.html`,
           emailAction: `${KIBANA_DOCS}email-action-type.html`,
           emailActionConfig: `${KIBANA_DOCS}email-action-type.html`,
+          emailExchangeClientSecretConfig: `${KIBANA_DOCS}email-action-type.html#exchange-client-secret`,
+          emailExchangeClientIdConfig: `${KIBANA_DOCS}email-action-type.html#exchange-client-tenant-id`,
           generalSettings: `${KIBANA_DOCS}alert-action-settings-kb.html#general-alert-action-settings`,
           indexAction: `${KIBANA_DOCS}index-action-type.html`,
           esQuery: `${KIBANA_DOCS}rule-type-es-query.html`,

src/plugins/vis_types/timeseries/public/application/components/lib/index_pattern_select/switch_mode_popover.tsx

@@ -57,7 +57,7 @@ export const SwitchModePopover = ({ onModeChange, useKibanaIndices }: PopoverPro
     allowStringIndicesLabel = (
       <FormattedMessage
         id="visTypeTimeseries.indexPatternSelect.switchModePopover.enableAllowStringIndices"
-        defaultMessage="To search by Elasticsearch indices enable {allowStringIndices} setting."
+        defaultMessage="To query Elasticsearch indices, you must enable the {allowStringIndices} setting."
         values={{
           allowStringIndices: canEditAdvancedSettings ? (
             <EuiLink color="accent" onClick={handleAllowStringIndicesLinkClick}>

src/plugins/vis_types/xy/public/editor/components/options/metrics_axes/index.tsx

@@ -6,7 +6,7 @@
  * Side Public License, v 1.
  */
 
-import React, { useState, useEffect, useCallback, useMemo } from 'react';
+import React, { useState, useEffect, useCallback } from 'react';
 import { cloneDeep, get } from 'lodash';
 
 import { EuiSpacer } from '@elastic/eui';
@@ -19,14 +19,14 @@ import { SeriesPanel } from './series_panel';
 import { CategoryAxisPanel } from './category_axis_panel';
 import { ValueAxesPanel } from './value_axes_panel';
 import {
-  makeSerie,
   isAxisHorizontal,
   countNextAxisNumber,
   getUpdatedAxisName,
   mapPositionOpposite,
   mapPosition,
   mapPositionOpposingOpposite,
 } from './utils';
+import { getSeriesParams } from '../../../../utils/get_series_params';
 
 export type SetParamByIndex = <P extends keyof ValueAxis, O extends keyof SeriesParam>(
   axesName: 'valueAxes' | 'seriesParams',
@@ -273,40 +273,19 @@ function MetricsAxisOptions(props: ValidationVisOptionsProps<VisParams>) {
   );
 
   const schemaName = vis.type.schemas.metrics[0].name;
-  const metrics = useMemo(() => {
-    return aggs.bySchemaName(schemaName);
-  }, [schemaName, aggs]);
-
   const firstValueAxesId = stateParams.valueAxes[0].id;
 
   useEffect(() => {
-    const updatedSeries = metrics.map((agg) => {
-      const params = stateParams.seriesParams.find((param) => param.data.id === agg.id);
-      const label = agg.makeLabel();
-
-      // update labels for existing params or create new one
-      if (params) {
-        return {
-          ...params,
-          data: {
-            ...params.data,
-            label,
-          },
-        };
-      } else {
-        const series = makeSerie(
-          agg.id,
-          label,
-          firstValueAxesId,
-          stateParams.seriesParams[stateParams.seriesParams.length - 1]
-        );
-        return series;
-      }
-    });
+    const updatedSeries = getSeriesParams(
+      aggs,
+      stateParams.seriesParams,
+      schemaName,
+      firstValueAxesId
+    );
 
-    setValue('seriesParams', updatedSeries);
+    if (updatedSeries) setValue('seriesParams', updatedSeries);
     updateAxisTitle(updatedSeries);
-  }, [metrics, firstValueAxesId, setValue, stateParams.seriesParams, updateAxisTitle]);
+  }, [firstValueAxesId, setValue, stateParams.seriesParams, updateAxisTitle, aggs, schemaName]);
 
   return isTabSelected ? (
     <>