Initial readme commit with some stub articles (#127420)

Co-authored-by: Kibana Machine <[email protected]>

docs/developer/plugin-list.asciidoc

@@ -523,8 +523,8 @@ newly created modules as well.
 Elastic.
 
 
-|{kib-repo}blob/{branch}/x-pack/plugins/monitoring[monitoring]
-|WARNING: Missing README.
+|{kib-repo}blob/{branch}/x-pack/plugins/monitoring/readme.md[monitoring]
+|This plugin provides the Stack Monitoring kibana application.
 
 
 |{kib-repo}blob/{branch}/x-pack/plugins/monitoring_collection/README.md[monitoringCollection]

x-pack/plugins/monitoring/dev_docs/how_to/cloud_setup.md

@@ -0,0 +1,50 @@
+First sign up on https://cloud.elastic.co/ and create a deployment in any convenient region, possibly one close to you.
+
+> **Elasticians**: Please use your work email address when signing up to avoid trial expiration. Also review the (internal) [Cloud First Testing](https://docs.elastic.dev/dev/guides/cloud-first-testing) documentation for additional features available to you.
+
+Once the deployment is created, enable logging and monitoring as covered in the Elasticsearch Service documentation under [Enable logging and monitoring](https://www.elastic.co/guide/en/cloud/current/ec-enable-logging-and-monitoring.html#ec-enable-logging-and-monitoring-steps).
+
+For testing purposes, shipping data to the same deployment you just created is fine.
+
+![Elasticsearch Service Console showing Logs and Metrics being configured to ship data to "this deployment"](../images/ec_logs_and_metrics_configuration.png)
+
+Once the plan is done you can open Stack Monitoring in the deployment's kibana.
+
+To connect a locally running instance of kibana to the cloud cluster, you'll need to create a user for it. You can do this via the UI, but here's a curl example for copy-pasting.
+
+First, set your endpoint and password as shell variables:
+
+```shell
+ELASTICSEARCH_ENDPOINT='<<<elasticsearch endpoint shown on cloud.elastic.co>>>'
+ELASTIC_PASSWORD='<<<elastic password displayed during deployment creation>>>'
+```
+
+Then create a `kibana_dev` user with the same password. `kibana_system` is already in use by the kibana launched by the elasticsearch service:
+
+```shell
+curl -X PUT ${ELASTICSEARCH_ENDPOINT}/_security/user/kibana_dev \
+  -H "Content-Type: application/json" \
+  -u "elastic:${ELASTIC_PASSWORD}" \
+  -d @- <<JSON
+    { "password": "${ELASTIC_PASSWORD}", "roles": [ "kibana_system" ] }
+JSON
+```
+
+Then create a kibana configuration for the deployment:
+
+```shell
+cat > config/kibana.cloud.yml <<YAML
+elasticsearch.hosts: ${ELASTICSEARCH_ENDPOINT}
+elasticsearch.username: kibana_dev
+elasticsearch.password: ${ELASTIC_PASSWORD}
+elasticsearch.ignoreVersionMismatch: true
+YAML
+```
+
+And start kibana with that config:
+
+```shell
+yarn start --config config/kibana.cloud.yml
+```
+
+Note that your local kibana will run data migrations and probably render the cloud created kibana unusable after your local kibana starts up.

x-pack/plugins/monitoring/dev_docs/how_to/local_setup.md

@@ -0,0 +1,80 @@
+# Basic setups
+
+## Yarn and internal collection
+
+For the simplest Elasticsearch & Kibana stack monitoring setup from a kibana clone, using [internal collection](../reference/terminology.md#internal-collection), first start elasticsearch with monitoring and a local [exporter](../reference/terminology.md#exporter) enabled.
+
+```shell
+yarn es snapshot --license trial \
+  -E xpack.monitoring.collection.enabled=true \ 
+  -E xpack.monitoring.exporters.id0.type=local
+```
+
+Then start kibana:
+
+```shell
+yarn start
+```
+
+Open kibana and navigate to "Stack Monitoring" (sidebar, homepage, or search bar). You should see a page like this.
+
+![Stack Monitoring overview page with Elasticsearch and Kibana panels using internal collection](../images/ek_internal_collection_overview.png)
+
+This is definitely the simplest way to get some data to explore, but internal collection is a deprecated collection mode, so next we'll use metricbeat collection.
+
+## Yarn and metricbeat collection
+
+To set up stack monitoring with [metricbeat collection](../reference/terminology.md#metricbeat-collection), first start elasticsearch with a trial license.
+
+```shell
+yarn es snapshot --license trial
+```
+
+Next, we'll need to give kibana a fixed base url so metricbeat can query it. So add this to your `kibana.dev.yml` file:
+
+```yml
+server.basePath: '/ftw'
+```
+
+Then start kibana:
+
+```shell
+yarn start
+```
+
+Next start metricbeat. Any method of [installing metricbeat](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-installation-configuration.html) works fine. We'll use docker since it is a good common point regardless of your development OS.
+
+```shell
+docker run --name metricbeat \
+  --pull always --rm \
+  --hostname=metricbeat \
+  --publish=5066:5066 \
+  --volume="$(pwd)/x-pack/plugins/monitoring/dev_docs/reference/metricbeat.yarn.yml:/usr/share/metricbeat/metricbeat.yml:ro" \
+  docker.elastic.co/beats/metricbeat:master-SNAPSHOT
+```
+
+## Filebeat for logs
+
+Regardless of the metrics collection method, logs will get collected using filebeat.
+
+Similar to metricbeat, any method of [installing filebeat](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation-configuration.html) works fine. We'll use docker again here as a good common point.
+
+```shell
+docker run --name filebeat \
+  --pull always --rm \
+  --hostname=filebeat \
+  --publish=5067:5067 \
+  --volume="$(pwd)/.es:/es:ro" \
+  --volume="$(pwd)/x-pack/plugins/monitoring/dev_docs/reference/filebeat.yarn.yml:/usr/share/filebeat/filebeat.yml:ro" \
+  docker.elastic.co/beats/filebeat:master-SNAPSHOT
+```
+
+# Complete docker setup
+
+We also maintain an internal docker-compose setup for running a full stack with monitoring enabled for all components.
+
+See (internal) https://github.com/elastic/observability-dev/tree/main/tools/docker-testing-cluster for more details.
+
+# Running more stack components from source
+
+See (internal) https://github.com/elastic/observability-dev/blob/main/docs/monitoring/monitoring-simulation-notes.md for details.

x-pack/plugins/monitoring/dev_docs/how_to/testing.md

@@ -0,0 +1,10 @@
+The Stack Monitoring plugin uses standard Kibana testing constructs.
+
+See the [Kibana Testing guide](https://www.elastic.co/guide/en/kibana/master/development-tests.html) for details on how to run the various test suites.
+
+We mainly use:
+1. Jest unit tests - located in sibling files to the source code 
+2. [api_integration tests](../../../../test/api_integration/apis/monitoring)
+3. [functional tests](../../../../test/functional/apps/monitoring)
+
+The functional and api integration tests are both under a 'Monitoring' description, so you can use `--grep Monitoring` to run only our tests.

x-pack/plugins/monitoring/dev_docs/reference/filebeat.yarn.yml

@@ -0,0 +1,36 @@
+# For collecting logs from Elasticearch launched from a kibana clone on the docker host
+
+http.enabled: true
+http.host: "0.0.0.0"
+http.port: 5067
+
+filebeat.modules:
+  - module: elasticsearch
+    server:
+      enabled: true
+      var.paths:
+        - /es/8.*/logs/*.log
+        - /es/8.*/logs/*_server.json
+    gc:
+      var.paths:
+        - /es/8.*/logs/gc.log.[0-9]*
+        - /es/8.*/logs/gc.log
+    audit:
+      var.paths:
+        - /es/8.*/logs/*_access.log
+        - /es/8.*/logs/*_audit.json
+    slowlog:
+      var.paths:
+        - /es/8.*/logs/*_index_search_slowlog.log
+        - /es/8.*/logs/*_index_indexing_slowlog.log
+        - /es/8.*/logs/*_index_search_slowlog.json
+        - /es/8.*/logs/*_index_indexing_slowlog.json
+    deprecation:
+      var.paths:
+        - /es/8.*/logs/*_deprecation.log
+        - /es/8.*/logs/*_deprecation.json
+
+output.elasticsearch:
+  hosts: [ "host.docker.internal:9200" ]
+  username: "elastic"
+  password: "changeme"

x-pack/plugins/monitoring/dev_docs/reference/metricbeat.yarn.yml

@@ -0,0 +1,36 @@
+# For collecting metrics from Elasticearch and Kibana launched from a kibana clone on the docker host
+
+http.enabled: true
+http.host: "0.0.0.0"
+
+metricbeat.modules:
+  - module: elasticsearch
+    xpack.enabled: true
+    period: 10s
+    hosts:
+      - "host.docker.internal:9200"
+    username: "elastic"
+    password: "changeme"
+
+  - module: kibana
+    xpack.enabled: true
+    basepath: "/ftw"
+    period: 10s
+    hosts: [ "host.docker.internal:5601" ]
+    username: "elastic"
+    password: "changeme"
+
+  - module: beat
+    xpack.enabled: true
+    period: 10s
+    hosts:
+      # metricbeat
+      - "http://host.docker.internal:5066"
+      # filebeat
+      - "http://host.docker.internal:5067"
+
+
+output.elasticsearch:
+  hosts: [ "host.docker.internal:9200" ]
+  username: "elastic"
+  password: "changeme"

x-pack/plugins/monitoring/dev_docs/reference/terminology.md

@@ -0,0 +1,9 @@
+# Terminology
+
+#### Internal collection
+
+The process of collecting monitoring data handled by the stack components themselves. Each component is responsible for sending documents to elasticsearch directly.
+
+#### Metricbeat collection
+
+The process of collecting monitoring data using metricbeat. Each component exposes an endpoint that metricbeat queries using a module for that component. Metricbeat then sends the data to elasticsearch for all monitored components. 

x-pack/plugins/monitoring/readme.md

@@ -0,0 +1,22 @@
+# Documentation for Stack Monitoring developers
+
+This plugin provides the Stack Monitoring kibana application.
+
+## Getting started
+- [Local setup](dev_docs/how_to/local_setup.md)
+- [Cloud setup](dev_docs/how_to/cloud_setup.md)
+- [Testing](dev_docs/how_to/testing.md)
+
+## Concepts
+- [Architectural Overview](dev_docs/reference/architectural_overview.md) (WIP)
+- [Terminology](dev_docs/reference/terminology.md) (WIP)
+- [Data Collection modes](dev_docs/reference/data_collection_modes.md) (WIP)
+- [Rules and Alerts](dev_docs/reference/rules_alerts.md)
+
+## Tooling
+- [Debugging logging](dev_docs/how_to/debug_logging.md) (WIP)
+- [APM tracing](dev_docs/how_to/apm_tracing.md) (WIP)
+
+## Troubleshooting
+- [Diagnostic queries](dev_docs/runbook/diagnostic_queries.md) (WIP)
+- [CPU metrics](dev_docs/runbook/cpu_metrics.md) (WIP)