Merge branch 'feature/ingest-node-pipelines' of github.com:elastic/ki…

…bana into pipeline-editor-part-mvp-2

* 'feature/ingest-node-pipelines' of github.com:elastic/kibana: (34 commits)
  [Ingest Node Pipelines] Clone Pipeline (elastic#64049)
  Move authz lib out of snapshot restore (elastic#63947)
  Migrate vis_type_table to kibana/new platform  (elastic#63105)
  Enable include/exclude in Terms agg for numeric fields (elastic#59425)
  follow conventions for saved object definitions (elastic#63571)
  [Docs]Adds saved object key setting to load balancing kib instances (elastic#63935)
  kbn/config-schema: Consider maybe properties as optional keys in ObjectType (elastic#63838)
  Fix page layouts, clean up unused code (elastic#63992)
  [SIEM] Adds recursive exact key checks for validation and formatter
  [Maps] Migrate remaining maps client files to NP (except routi… (elastic#63859)
  [Maps] Do not fetch geo_shape from docvalues (elastic#63997)
  Vega doc changes (elastic#63889)
  [Metrics UI] Reorganize file layout for Metrics UI (elastic#60049)
  Add sub-menus to Resolver node (for 75% zoom) (elastic#63476)
  [FTR] Add test suite metrics tracking/output (elastic#62515)
  [Ingest pipelines] Delete pipeline (elastic#63635)
  [ML] Fixing single metric viewer page padding (elastic#63839)
  [Discover] Allow user to generate a report after saving a modified saved search (elastic#63623)
  [Reporting] Config flag to escape formula CSV values (elastic#63645)
  [Metrics UI] Remove remaining field filtering (elastic#63398)
  ...

.github/CODEOWNERS

@@ -13,6 +13,7 @@
 /src/legacy/core_plugins/kibana/public/dev_tools/ @elastic/kibana-app
 /src/legacy/core_plugins/vis_type_vislib/ @elastic/kibana-app
 /src/plugins/vis_type_xy/ @elastic/kibana-app
+/src/plugins/vis_type_table/ @elastic/kibana-app
 /src/plugins/kibana_legacy/ @elastic/kibana-app
 /src/plugins/vis_type_timelion/ @elastic/kibana-app
 /src/plugins/dashboard/ @elastic/kibana-app

.i18nrc.json

@@ -48,7 +48,7 @@
     "visDefaultEditor": "src/plugins/vis_default_editor",
     "visTypeMarkdown": "src/plugins/vis_type_markdown",
     "visTypeMetric": "src/plugins/vis_type_metric",
-    "visTypeTable": "src/legacy/core_plugins/vis_type_table",
+    "visTypeTable": "src/plugins/vis_type_table",
     "visTypeTagCloud": "src/legacy/core_plugins/vis_type_tagcloud",
     "visTypeTimeseries": ["src/legacy/core_plugins/vis_type_timeseries", "src/plugins/vis_type_timeseries"],
     "visTypeVega": "src/legacy/core_plugins/vis_type_vega",

CONTRIBUTING.md

@@ -22,6 +22,7 @@ A high level overview of our contributing guidelines.
     - [Setting Up SSL](#setting-up-ssl)
   - [Linting](#linting)
   - [Internationalization](#internationalization)
+  - [Localization](#localization)
   - [Testing and Building](#testing-and-building)
     - [Debugging server code](#debugging-server-code)
     - [Instrumenting with Elastic APM](#instrumenting-with-elastic-apm)
@@ -408,6 +409,11 @@ ReactDOM.render(
 There are a number of tools created to support internationalization in Kibana that would allow one to validate internationalized labels,
 extract them to a `JSON` file or integrate translations back to Kibana. To know more, please read corresponding [readme](src/dev/i18n/README.md) file.
 
+### Localization
+
+We cannot support accepting contributions to the translations from any source other than the translators we have engaged to do the work.
+We are still to develop a proper process to accept any contributed translations. We certainly appreciate that people care enough about the localization effort to want to help improve the quality. We aim to build out a more comprehensive localization process for the future and will notify you once contributions can be supported, but for the time being, we are not able to incorporate suggestions.
+
 ### Testing and Building
 
 To ensure that your changes will not break other functionality, please run the test suite and build process before submitting your Pull Request.

docs/setup/production.asciidoc

@@ -133,7 +133,8 @@ server.port
 Settings that must be the same:
 --------
 xpack.security.encryptionKey //decrypting session cookies
-xpack.reporting.encryptionKey //decrypting reports stored in Elasticsearch
+xpack.reporting.encryptionKey //decrypting reports
+xpack.encryptedSavedObjects.encryptionKey // decrypting saved objects
 --------
 
 Separate configuration files can be used from the command line by using the `-c` flag:

docs/user/alerting/action-types/pagerduty.asciidoc

@@ -4,19 +4,142 @@
 
 The PagerDuty action type uses the https://v2.developer.pagerduty.com/docs/events-api-v2[v2 Events API] to trigger, acknowledge, and resolve PagerDuty alerts.
 
+* <<pagerduty-benefits, PagerDuty and Elastic integration benefits>>
+* <<pagerduty-connector-configuration, Connector configuration>>
+* <<pagerduty-action-configuration, Action configuration>>
+
+[float]
+[[pagerduty-benefits]]
+=== PagerDuty + Elastic integration benefits
+
+By integrating PagerDuty with alerts, you can:
+
+* Route your alerts to the right PagerDuty responder within your team, based on your structure, escalation policies, and workflows.
+* Automatically generate incidents of different types and severity based on each alert’s context.
+* Tailor the incident data to match your needs by easily passing the alerting context from Kibana to PagerDuty.
+
+[float]
+[[pagerduty-how-it-works]]
+==== How it works
+
+{kib} allows you to create alerts to notify you of a significant move
+in your dataset.
+You can create alerts for all your Observability, Security, and Elastic Stack use cases.
+Alerts will trigger a new incident on the corresponding PagerDuty service.
+
+[float]
+==== Requirements
+
+In the `kibana.yml` configuration file, you must add the <<general-alert-action-settings, saved objects encryption setting>>.
+This is required to encrypt parameters that must be secured, for example PagerDuty’s integration key.
+
+If you have security enabled:
+
+* You must have
+application privileges to access Metrics, APM, Uptime, or SIEM.
+* If you are using a self-managed deployment with security, you must have
+Transport Security Layer (TLS) enabled for communication <<configuring-tls-kib-es, between Elasticsearch and Kibana>>.
+Alerts uses API keys to secure background alert checks and actions,
+and API keys require {ref}/configuring-tls.html#tls-http[TLS on the HTTP interface].
+
+Although not a requirement, to harden the integrations security you might want to
+review the <<action-settings, Actions settings>> that are available to you.
+
+[float]
+[[pagerduty-support]]
+==== Support
+If you need help with this integration, get in touch with the {kib} team by visiting
+https://support.elastic.co[support.elastic.co] or by using the *Ask Elastic* option in the {kib} Help menu.
+You can also select the {kib} category at https://discuss.elastic.co/[discuss.elastic.co].
+
+[float]
+[[pagerduty-integration-walkthrough]]
+==== Integration with PagerDuty walkthrough
+
+[float]
+[[pagerduty-in-pagerduty]]
+===== In PagerDuty
+
+. From the *Configuration* menu, select *Services*.
+. Add an integration to a service:
++
+* If you are adding your integration to an existing service,
+click the name of the service you want to add the integration to.
+Then, select the *Integrations* tab and click the *New Integration* button.
+* If you are creating a new service for your integration,
+go to
+https://support.pagerduty.com/docs/services-and-integrations#section-configuring-services-and-integrations[Configuring Services and Integrations]
+and follow the steps outlined in the *Create a New Service* section, selecting *Elastic* as the *Integration Type* in step 4.
+Continue with the <<pagerduty-in-elastic, In Elastic>> section once you have finished these steps.
+
+. Enter an *Integration Name* in the format Elastic-service-name (for example, Elastic-Alerting or Kibana-APM-Alerting)
+and select Elastic from the *Integration Type* menu.
+. Click *Add Integration* to save your new integration.
++
+You will be redirected to the *Integrations* tab for your service. An Integration Key is generated on this screen.
++
+[role="screenshot"]
+image::user/alerting/images/pagerduty-integration.png[PagerDuty Integrations tab]
+
+. Save this key, as you will use it when you configure the integration with Elastic in the next section.
+
+[float]
+[[pagerduty-in-elastic]]
+===== In Elastic
+
+. Create a PagerDuty Connector in Kibana.  You can:
++
+* Create a connector as part of creating an alert by selecting PagerDuty in the *Actions*
+section of the alert configuration and selecting *Add new*.
+* Alternatively, create a connector by navigating to *Management* from the {kib} navbar and selecting
+*Alerts and Actions*. Then, select the *Connectors* tab, click the *Create connector* button, and select the PagerDuty option.
+
+. Configure the connector by giving it a name and optionally entering the API URL and Routing Key, or using the defaults.
++
+See <<pagerduty-in-pagerduty, In PagerDuty>> for how to obtain the endpoint and key information from PagerDuty and
+<<pagerduty-connector-configuration, Connector configuration>> for more details.
+
+. Save the Connector.
+
+. Create an alert using *Management > Alerts and Actions* or the application of your choice.
+
+. Set up an action using your PagerDuty connector, by determining:
++
+* The action’s type: Trigger, Resolve, or Acknowledge.
+* The event’s severity: Info, warning, error, or critical.
+* An array of different fields, including the timestamp, group, class, component, and your dedup key.
+Depending on your custom needs, assign them variables from the alerting context.
+To see the available context variables, click on the *Add alert variable* icon next
+to each corresponding field. For more details on these parameters, see the
+<<pagerduty-action-configuration, Actions Configuration>> and the PagerDuty
+https://v2.developer.pagerduty.com/v2/docs/send-an-event-events-api-v2[API v2 documentation].
+
+
+[float]
+[[pagerduty-uninstall]]
+==== How to uninstall
+To remove a PagerDuty connector from an alert, simply remove it
+from the *Actions* section of that alert, using the remove (x) icon.
+This will disable the integration for the particular alert.
+
+To delete the connector entirely, go to *Management > Alerts and Actions*.
+Select the *Connectors* tab, and then click on the delete icon.
+This is an irreversible action and impacts all alerts that use this connector.
+
+
 [float]
 [[pagerduty-connector-configuration]]
-==== Connector configuration
+=== Connector configuration
 
 PagerDuty connectors have the following configuration properties:
 
 Name::      The name of the connector. The name is used to identify a  connector in the management UI connector listing, or in the connector list when configuring an action.
-API URL::   An optional PagerDuty event URL. Defaults to `https://events.pagerduty.com/v2/enqueue`. If you are using the <<action-settings, `xpack.actions.whitelistedHosts`>> setting, make sure the hostname is whitelisted. 
+API URL::   An optional PagerDuty event URL. Defaults to `https://events.pagerduty.com/v2/enqueue`. If you are using the <<action-settings, `xpack.actions.whitelistedHosts`>> setting, make sure the hostname is whitelisted.
 Routing Key::   A 32 character PagerDuty Integration Key for an integration on a service or on a global ruleset.
 
 [float]
 [[pagerduty-action-configuration]]
-==== Action configuration
+=== Action configuration
 
 PagerDuty actions have the following properties:
 
@@ -26,8 +149,8 @@ Dedup Key::     All actions sharing this key will be associated with the same Pa
 Timestamp::     An *optional* https://v2.developer.pagerduty.com/v2/docs/types#datetime[ISO-8601 format date-time], indicating the time the event was detected or generated.
 Component::     An *optional* value indicating the component of the source machine that is responsible for the event, for example `mysql` or `eth0`.
 Group::         An *optional* value indicating the logical grouping of components of a service, for example `app-stack`.
-Source::        An *optional* value indicating the affected system, preferably a hostname or fully qualified domain name. Defaults to the {kib} saved object id of the action. 
+Source::        An *optional* value indicating the affected system, preferably a hostname or fully qualified domain name. Defaults to the {kib} saved object id of the action.
 Summary::       An *optional* text summary of the event, defaults to `No summary provided`. The maximum length is 1024 characters.
 Class::         An *optional* value indicating the class/type of the event, for example `ping failure` or `cpu load`.
 
-For more details on these properties, see https://v2.developer.pagerduty.com/v2/docs/send-an-event-events-api-v2[PagerDuty v2 event parameters].
+For more details on these properties, see https://v2.developer.pagerduty.com/v2/docs/send-an-event-events-api-v2[PagerDuty v2 event parameters].