docs: finish duplicating apm overview

docs/apm-components.asciidoc

@@ -0,0 +1,64 @@
+[[apm-components]]
+=== Components and documentation
+
+Elastic APM consists of four components: *APM agents*, *Elastic Agent*, *Elasticsearch*, and *Kibana*.
+
+image::./images/apm-architecture.png[Architecture of Elastic APM]
+
+[float]
+==== APM Agents
+
+APM agents are open source libraries written in the same language as your service.
+You may only need one, or you might use all of them.
+You install them into your service as you would install any other library.
+They instrument your code and collect performance data and errors at runtime.
+This data is buffered for a short period and sent on to APM Server.
+
+Each agent has its own documentation:
+
+* {apm-go-ref-v}/introduction.html[Go agent]
+* {apm-ios-ref-v}/intro.html[iOS agent]
+* {apm-java-ref-v}/intro.html[Java agent]
+* {apm-dotnet-ref-v}/intro.html[.NET agent]
+* {apm-node-ref-v}/intro.html[Node.js agent]
+* {apm-php-ref-v}/intro.html[PHP agent]
+* {apm-py-ref-v}/getting-started.html[Python agent]
+* {apm-ruby-ref-v}/introduction.html[Ruby agent]
+* {apm-rum-ref-v}/intro.html[JavaScript Real User Monitoring (RUM) agent]
+
+[float]
+==== Elastic Agent
+
+Elastic Agent is a single, unified way to add monitoring for logs, metrics, traces, and other types of data to each host.
+A single agent makes it easier and faster to deploy monitoring across your infrastructure.
+The agent’s single, unified policy makes it easier to add integrations for new data sources.
+
+The APM integration within Elastic Agent receives performance data from your APM agents,
+validates and processes it, and then transforms the data into {es} documents.
+Removing this logic from APM agents help keeps them light, prevents certain security risks,
+and improves compatibility across the Elastic Stack.
+
+[float]
+==== Elasticsearch
+
+{ref}/index.html[Elasticsearch] is a highly scalable free and open full-text search and analytics engine.
+It allows you to store, search, and analyze large volumes of data quickly and in near real time.
+Elasticsearch is used to store APM performance metrics and make use of its aggregations.
+
+[float]
+==== Kibana APM app
+
+{kibana-ref}/index.html[Kibana] is a free and open analytics and visualization platform designed to work with Elasticsearch.
+You use Kibana to search, view, and interact with data stored in Elasticsearch.
+
+Since application performance monitoring is all about visualizing data and detecting bottlenecks,
+it's crucial you understand how to use the {kibana-ref}/xpack-apm.html[APM app] in Kibana.
+The following sections will help you get started:
+
+* {apm-app-ref}/apm-ui.html[Set up]
+* {apm-app-ref}/apm-getting-started.html[Get started]
+* {apm-app-ref}/apm-how-to.html[How-to guides]
+
+APM also has built-in integrations with Machine learning. To learn more about this feature,
+or the anomaly detection feature that's built on top of it,
+refer to {kibana-ref}/machine-learning-integration.html[Machine learning integration].

docs/apm-distributed-tracing.asciidoc

@@ -101,9 +101,9 @@ with each agent's API.
 
 Sending services must add the `traceparent` header to outgoing requests.
 
-// --
-// include::../../tab-widgets/distributed-trace-send-widget.asciidoc[]
-// --
+--
+include::./tab-widgets/distributed-trace-send-widget.asciidoc[]
+--
 
 [float]
 [[distributed-tracing-incoming]]
@@ -112,9 +112,9 @@ Sending services must add the `traceparent` header to outgoing requests.
 Receiving services must parse the incoming `traceparent` header,
 and start a new transaction or span as a child of the received context.
 
-// --
-// include::../../tab-widgets/distributed-trace-receive-widget.asciidoc[]
-// --
+--
+include::./tab-widgets/distributed-trace-receive-widget.asciidoc[]
+--
 
 [float]
 [[distributed-tracing-rum]]

docs/apm-overview.asciidoc

@@ -22,5 +22,6 @@ like JVM metrics in the Java Agent, and Go runtime metrics in the Go Agent.
 [float]
 === Give Elastic APM a try
 
-// Learn more about the <<components,components>> that make up Elastic APM,
+Learn more about the <<apm-components,components>> that make up Elastic APM
+// ,
 // or jump right into the <<install-and-run,quick start guide>>.

docs/apm-rum.asciidoc

@@ -0,0 +1,12 @@
+[[apm-rum]]
+=== Real User Monitoring (RUM)
+Real User Monitoring captures user interaction with clients such as web browsers.
+The {apm-rum-ref-v}[JavaScript Agent] is Elastic’s RUM Agent.
+// To use it you need to {apm-server-ref-v}/configuration-rum.html[enable RUM support] in the APM Server.
+
+Unlike Elastic APM backend agents which monitor requests and responses,
+the RUM JavaScript agent monitors the real user experience and interaction within your client-side application.
+The RUM JavaScript agent is also framework-agnostic, which means it can be used with any frontend JavaScript application.
+
+You will be able to measure metrics such as "Time to First Byte", `domInteractive`,
+and `domComplete` which helps you discover performance issues within your client-side application as well as issues that relate to the latency of your server-side application.

docs/cross-cluster-search.asciidoc

@@ -0,0 +1,42 @@
+[[cross-cluster-search]]
+=== Cross-cluster search
+
+Elastic APM utilizes Elasticsearch's cross-cluster search functionality.
+Cross-cluster search lets you run a single search request against one or more
+{ref}/modules-remote-clusters.html[remote clusters] --
+making it easy to search APM data across multiple sources.
+This means you can also have deployments per data type, making sizing and scaling more predictable,
+and allowing for better performance while managing multiple observability use cases.
+
+[float]
+[[set-up-cross-cluster-search]]
+==== Set up cross-cluster search
+
+*Step 1. Set up remote clusters.*
+
+If you're using the Hosted Elasticsearch Service, see {cloud}/ec-enable-ccs.html[Enable cross-cluster search].
+
+You can add remote clusters directly in Kibana, under *Management* > *Elasticsearch* > *Remote clusters*.
+All you need is a name for the remote cluster and the seed node(s).
+Remember the names of your remote clusters, you'll need them in step two.
+See {ref}/ccr-getting-started.html[managing remote clusters] for detailed information on the setup process.
+
+Alternatively, you can {ref}/modules-remote-clusters.html#configuring-remote-clusters[configure remote clusters]
+in Elasticsearch's `elasticsearch.yml` file.
+
+*Step 2. Edit the default APM app index pattern.*
+
+The APM app index pattern determines from which clusters and indices to display data.
+Index patterns follow this convention: `<cluster-names>:<pattern>`.
+The default value is `apm-*`, which displays data from any index beginning with `apm-`.
+
+To display data from the local cluster, and all remote clusters,
+you'll need to update the index pattern to: `*:apm-*,apm-*`.
+You can also specify certain clusters to display data from, for example, `obs-1:apm-*,obs-2:apm-*`.
+
+There are two ways to edit the default index pattern:
+
+* In the APM app -- Navigate to *APM* > *Settings* > *Indices*, and change all `xpack.apm.indices.*` values to
+include the new index pattern, e.g., `*:apm-*,apm-*`.
+* In `kibana.yml` -- All of the {kibana-ref}/apm-settings-kb.html[`xpack.apm.indices.*`] configuration values must
+include the new index pattern, e.g., `*:apm-*,apm-*`.

docs/features.asciidoc

@@ -7,25 +7,22 @@
 
 * <<apm-data-security>>
 * <<apm-distributed-tracing>>
-// * <<rum>>
-// * <<trace-sampling>>
-// * <<opentracing>>
-// * <<open-telemetry-elastic>>
-// * <<observability-integrations>>
-// * <<apm-cross-cluster-search>>
+* <<apm-rum>>
+* <<sampling>>
+* <<open-telemetry>>
+* <<log-correlation>>
+* <<cross-cluster-search>>
 
 include::./apm-data-security.asciidoc[]
 
 include::./apm-distributed-tracing.asciidoc[]
 
-// include::./rum.asciidoc[]
+include::./apm-rum.asciidoc[]
 
-// include::./trace-sampling.asciidoc[]
+include::./sampling.asciidoc[]
 
-// include::./opentracing.asciidoc[]
+include::./open-telemetry.asciidoc[]
 
-// include::./opentelemetry-elastic.asciidoc[]
+include::./log-correlation.asciidoc[]
 
-// include::./obs-integrations.asciidoc[]
-
-// include::./cross-cluster-search.asciidoc[]
+include::./cross-cluster-search.asciidoc[]

docs/integrations-index.asciidoc

@@ -9,6 +9,8 @@ include::{asciidoc-dir}/../../shared/attributes.asciidoc[]
 
 include::apm-overview.asciidoc[]
 
+include::apm-components.asciidoc[]
+
 == Quick start
 
 Include quick start file from obs-docs repo

docs/log-correlation.asciidoc

@@ -0,0 +1,176 @@
+[[log-correlation]]
+=== Logging integration
+
+Many applications use logging frameworks to help record, format, and append an application's logs.
+Elastic APM now offers a way to make your application logs even more useful,
+by integrating with the most popular logging frameworks in their respective languages.
+This means you can easily inject trace information into your logs,
+allowing you to explore logs in the {observability-guide}/monitor-logs.html[Logs app],
+then jump straight into the corresponding APM traces -- all while preserving the trace context.
+
+To get started:
+
+. Enable log correlation
+. Add APM identifiers to your logs
+. Ingest your logs into Elasticsearch
+
+[float]
+==== Enable Log correlation
+
+Some Agents require you to first enable log correlation in the Agent.
+This is done with a configuration variable, and is different for each Agent.
+See the relevant https://www.elastic.co/guide/en/apm/agent/index.html[Agent documentation] for further information.
+
+// Not enough of the Agent docs are ready yet.
+// Commenting these out and will replace when ready.
+// * *Java*: {apm-java-ref-v}/config-logging.html#config-enable-log-correlation[`enable_log_correlation`]
+// * *.NET*: {apm-dotnet-ref-v}/[]
+// * *Node.js*: {apm-node-ref-v}/[]
+// * *Python*: {apm-py-ref-v}/[]
+// * *Ruby*: {apm-ruby-ref-v}/[]
+// * *Rum*: {apm-rum-ref-v}/[]
+
+[float]
+==== Add APM identifiers to your logs
+
+Once log correlation is enabled,
+you must ensure your logs contain APM identifiers.
+In some supported frameworks, this is already done for you.
+In other scenarios, like for unstructured logs,
+you'll need to add APM identifiers to your logs in any easy to parse manner.
+
+The identifiers we're interested in are: {ecs-ref}/ecs-tracing.html[`trace.id`] and
+{ecs-ref}/ecs-tracing.html[`transaction.id`]. Certain Agents also support the `span.id` field.
+
+This process for adding these fields will differ based the Agent you're using, the logging framework,
+and the type and structure of your logs.
+
+See the relevant https://www.elastic.co/guide/en/apm/agent/index.html[Agent documentation] to learn more.
+
+// Not enough of the Agent docs have been backported yet.
+// Commenting these out and will replace when ready.
+// * *Go*: {apm-go-ref-v}/supported-tech.html#supported-tech-logging[Logging frameworks]
+// * *Java*: {apm-java-ref-v}/[] NOT merged yet https://github.com/elastic/apm-agent-java/pull/854
+// * *.NET*: {apm-dotnet-ref-v}/[]
+// * *Node.js*: {apm-node-ref-v}/[]
+// * *Python*: {apm-py-ref-v}/[]
+// * *Ruby*: {apm-ruby-ref-v}/[] Not backported yet https://www.elastic.co/guide/en/apm/agent/ruby/master/log-correlation.html
+// * *Rum*: {apm-rum-ref-v}/[]
+
+[float]
+==== Ingest your logs into Elasticsearch
+
+Once your logs contain the appropriate identifiers (fields), you need to ingest them into Elasticsearch.
+Luckily, we've got a tool for that -- Filebeat is Elastic's log shipper.
+The {filebeat-ref}/filebeat-installation-configuration.html[Filebeat quick start]
+guide will walk you through the setup process.
+
+Because logging frameworks and formats vary greatly between different programming languages,
+there is no one-size-fits-all approach for ingesting your logs into Elasticsearch.
+The following tips should hopefully get you going in the right direction:
+
+**Download Filebeat**
+
+There are many ways to download and get started with Filebeat.
+Read the {filebeat-ref}/filebeat-installation-configuration.html[Filebeat quick start] guide to determine which is best for you.
+
+**Configure Filebeat**
+
+Modify the {filebeat-ref}/configuring-howto-filebeat.html[`filebeat.yml`] configuration file to your needs.
+Here are some recommendations:
+
+* Set `filebeat.inputs` to point to the source of your logs
+* Point Filebeat to the same Elastic Stack that is receiving your APM data
+  * If you're using Elastic cloud, set `cloud.id` and `cloud.auth`.
+  * If your using a manual setup, use `output.elasticsearch.hosts`.
+
+[source,yml]
+----
+filebeat.inputs:
+- type: log <1>
+  paths: <2>
+    - /var/log/*.log
+cloud.id: "staging:dXMtZWFzdC0xLmF3cy5mb3VuZC5pbyRjZWMNjN2Q3YTllOTYyNTc0Mw==" <3>
+cloud.auth: "elastic:YOUR_PASSWORD" <4>
+----
+<1> Configures the `log` input
+<2> Path(s) that must be crawled to fetch the log lines
+<3> Used to resolve the Elasticsearch and Kibana URLs for Elastic Cloud
+<4> Authorization token for Elastic Cloud
+
+**JSON logs**
+
+For JSON logs you can use the {filebeat-ref}/filebeat-input-log.html[`log` input] to read lines from log files.
+Here's what a sample configuration might look like:
+
+[source,yml]
+----
+filebeat.inputs:
+  json.keys_under_root: true <1>
+  json.add_error_key: true <2>
+  json.message_key: message <3>
+----
+<1> `true` copies JSON keys to the top level in the output document
+<2> Tells Filebeat to add an `error.message` and `error.type: json` key in case of JSON unmarshalling errors
+<3> Specifies the JSON key on which to apply line filtering and multiline settings
+
+**Parsing unstructured logs**
+
+Consider the following log that is decorated with the `transaction.id` and `trace.id` fields:
+
+[source,log]
+----
+2019-09-18 21:29:49,525 - django.server - ERROR - "GET / HTTP/1.1" 500 27 | elasticapm transaction.id=fcfbbe447b9b6b5a trace.id=f965f4cc5b59bdc62ae349004eece70c span.id=None
+----
+
+All that's needed now is an {filebeat-ref}/configuring-ingest-node.html[ingest node processor] to pre-process your logs and
+extract these structured fields before they are indexed in Elasticsearch.
+To do this, you'd need to create a pipeline that uses Elasticsearch's {ref}/grok-processor.html[Grok Processor].
+Here's an example:
+
+[source, json]
+----
+PUT _ingest/pipeline/log-correlation
+{
+  "description": "Parses the log correlation IDs out of the raw plain-text log",
+  "processors": [
+    {
+      "grok": {
+        "field": "message", <1>
+        "patterns": ["%{GREEDYDATA:message} | elasticapm transaction.id=%{DATA:transaction.id} trace.id=%{DATA:trace.id} span.id=%{DATA:span.id}"] <2>
+      }
+    }
+  ]
+}
+----
+<1> The field to use for grok expression parsing
+<2> An ordered list of grok expression to match and extract named captures with:
+`%{DATA:transaction.id}` captures the value of `transaction.id`,
+`%{DATA:trace.id}` captures the value or `trace.id`, and
+`%{DATA:span.id}` captures the value of `span.id`.
+
+NOTE: Depending on how you've added APM data to your logs,
+you may need to tweak this grok pattern in order to work for your setup.
+In addition, it's possible to extract more structure out of your logs.
+Make sure to follow the {ecs-ref}/ecs-field-reference.html[Elastic Common Schema]
+when defining which fields you are storing in Elasticsearch.
+
+Then, configure Filebeat to use the processor in `filebeat.yml`:
+
+[source, json]
+----
+output.elasticsearch:
+  pipeline: "log-correlation"
+----
+
+If your logs contain messages that span multiple lines of text (common in Java stack traces),
+you'll also need to configure {filebeat-ref}/multiline-examples.html[multiline settings].
+
+The following example shows how to configure Filebeat to handle a multiline message where the first line of the message begins with a bracket ([).
+
+[source,yml]
+----
+multiline.pattern: '^\['
+multiline.negate: true
+multiline.match: after
+----