Skip to content

Commit

Permalink
Replace ecs-metadata with awsecscontainermetrics
Browse files Browse the repository at this point in the history
Update the docs to deprecate ecs-metadata and replace it with the `awsecscontainermetrics`.

Corresponding change on the Splunk Collector: signalfx/splunk-otel-collector#5541
  • Loading branch information
pjanotti authored Oct 23, 2024
1 parent 1b39708 commit c30284f
Showing 1 changed file with 27 additions and 42 deletions.
69 changes: 27 additions & 42 deletions gdi/opentelemetry/deployments/deployments-ecs-ec2.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ Deploy the Collector with Amazon ECS EC2
********************************************************

.. meta::
:description: Deploy the Splunk Observability Cloud OpenTelemetry Collector as a Daemon service in an Amazon ECS EC2 cluster.
:description: Deploy the Splunk Observability Cloud OpenTelemetry Collector as a Sidecar in an Amazon ECS EC2 cluster.

Use the guided setup to deploy the Collector as a Daemon service in an Amazon ECS EC2 cluster. The guided setup provides a JSON task definition for the Collector.
Use the guided setup to deploy the Collector as a Sidecar in an Amazon ECS EC2 cluster. The guided setup provides a JSON task definition for the Collector.

Choose one of the following Collector configuration options:

Expand All @@ -28,60 +28,26 @@ Getting started

The following sections describe how to create a task definition and launch the Collector. A task definition is required to run Docker containers in Amazon ECS. After creating the task definition, you need to launch the Collector.

Create a task definition
Add the Collector as a Sidecar
---------------------------------
.. note::

Knowledge of Amazon ECS using launch type EC2 is assumed. See :new-page:`Getting started with the classic console using Amazon EC2 <https://docs.aws.amazon.com/AmazonECS/latest/developerguide/getting-started-ecs-ec2.html>` for further reading.

Creating the task definition requires using release v0.34.1 or newer (which corresponds to image tag 0.34.1 and newer) of the Collector. See the :new-page:`image repository <https://quay.io/repository/signalfx/splunk-otel-collector?tab=tags>` to download the latest image.

To create the task definition:
Open the ECS task definition to which the Collector Sidecar is going to be added:

1. Locate the task definition for the Collector from the :new-page:`repository <https://github.com/signalfx/splunk-otel-collector/blob/main/deployments/ecs/ec2/splunk-otel-collector.json>`.
2. Replace ``MY_SPLUNK_ACCESS_TOKEN`` and ``MY_SPLUNK_REALM`` with valid values. You should pin the image version to a specific version instead of ``latest`` to avoid upgrade issues.
3. Create a task definition of EC2 launch type. See :new-page:`Creating a task definition using the new console <https://docs.aws.amazon.com/AmazonECS/latest/developerguide/create-task-definition.html>` for the instructions. The supplied task definition is a minimal definition. See :new-page:`Task definition parameters <https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html>` for additional configuration options.
2. Merge the definitions of the Collector with the existing ECS task definition.
3. Replace ``MY_SPLUNK_ACCESS_TOKEN`` and ``MY_SPLUNK_REALM`` with valid values. You can pin the image version to a specific version instead of ``latest`` if you want to avoid automatic upgrades.

The Collector is configured to use the default configuration file ``/etc/otel/collector/ecs_ec2_config.yaml``. The Collector image Dockerfile is available at :new-page:`Dockerfile <https://github.com/signalfx/splunk-otel-collector/blob/main/cmd/otelcol/Dockerfile>` and the contents of the default configuration file can be seen at :new-page:`ECS EC2 configuration <https://github.com/signalfx/splunk-otel-collector/blob/main/cmd/otelcol/config/collector/ecs_ec2_config.yaml>`.

.. note::

You do not need the ``smartagent/ecs-metadata`` metrics receiver in the default configuration file if all you want is tracing. You can take the default configuration, remove the receiver, then use the configuration in a custom configuration following the directions in :ref:`ecs-ec2-custom-config`.

The configured network mode for the task is ``host``. This means that task metadata endpoint version 2 used by the ``smartagent/ecs-metadata`` receiver is not activated by default. See :new-page:`task metadata endpoint <https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-metadata-endpoint.html>` to determine if task metadata endpoint version 3 is activated by default for your task. If this version is activated, then add the following to the environment list in the task definition:

.. code-block:: none
{
"name": "ECS_TASK_METADATA_ENDPOINT",
"value": "${ECS_CONTAINER_METADATA_URI}/task"
},
{
"name": "ECS_TASK_STATS_ENDPOINT",
"value": "${ECS_CONTAINER_METADATA_URI}/task/stats"
}
You do not need the ``awsecscontainermetrics`` receiver in the default configuration file if all you want is tracing. You can take the default configuration, remove the receiver, then use the configuration in a custom configuration following the directions in :ref:`ecs-ec2-custom-config`.

Assign a stringified array of metrics you want excluded to environment variable ``METRICS_TO_EXCLUDE``. You can set the memory limit for the ``memory_limiter`` processor using environment variable ``SPLUNK_MEMORY_LIMIT_MIB``. The default memory limit is 512 MiB.

Launch the Collector
=============================
The Collector is designed to be run as a Daemon service in an EC2 ECS cluster. To create a Collector service from the Amazon ECS console:

#. Go to your cluster in the console.
#. Select :guilabel:`Services`.
#. Select :guilabel:`Create`.
#. Select the following options:
#. Launch Type: EC2
#. Task Definition (Family): splunk-otel-collector
#. Task Definition (Revision): 1 (or whatever the latest is in your case)
#. Service Name: splunk-otel-collector
#. Service type: DAEMON
#. Leave everything else at default.
#. Select :guilabel:`Next step`.
#. Leave everything on this next page at their defaults and select :guilabel:`Next step`.
#. Leave everything on this next page at their defaults and select :guilabel:`Next step`.
#. Select :guilabel:`Create Service` to deploy the Collector onto each node in the ECS cluster. You should see infrastructure and docker metrics flowing soon.

.. _ecs-ec2-custom-config:

Use a custom configuration
Expand All @@ -94,7 +60,7 @@ Alternatively, you can specify the custom configuration YAML directly using the

``ecs_observer`` configuration
--------------------------------
Use extension Amazon Elastic Container Service Observer (``ecs_observer``) in your custom configuration to discover metrics targets in running tasks, filtered by service names, task definitions, and container labels. ``ecs_observer`` is currently limited to Prometheus targets and requires the read-only permissions below. You can add the permissions to the task role by adding them to a customer-managed policy that is attached to the task role.
Use extension Amazon Elastic Container Service Observer (``ecs_observer``) in your custom configuration to discover metrics targets in running tasks, filtered by service names, task definitions, and container labels. ``ecs_observer`` is currently limited to Prometheus targets and requires the read-only permissions below. The Collector should be configured to run as an ECS Daemon. You can add the permissions to the task role by adding them to a customer-managed policy that is attached to the task role.

.. code-block:: yaml
Expand Down Expand Up @@ -147,6 +113,25 @@ The results are written to ``/etc/ecs_sd_targets.yaml``. The ``prometheus`` rece
.. _aws-parameter-store:

Launch the Collector as a Daemon
-------------------------------
To launch the Collector from the Amazon ECS console:

#. Go to your cluster in the console.
#. Select :guilabel:`Services`.
#. Select :guilabel:`Create`.
#. Select the following options:
#. Launch Type: EC2
#. Task Definition (Family): splunk-otel-collector
#. Task Definition (Revision): 1 (or whatever the latest is in your case)
#. Service Name: splunk-otel-collector
#. Service type: DAEMON
#. Leave everything else at default.
#. Select :guilabel:`Next step`.
#. Leave everything on this next page at their defaults and select :guilabel:`Next step`.
#. Leave everything on this next page at their defaults and select :guilabel:`Next step`.
#. Select :guilabel:`Create Service` to deploy the Collector onto each node in the ECS cluster. You should see infrastructure and docker metrics flowing soon.

Use the AWS Parameter Store
----------------------------

Expand Down

0 comments on commit c30284f

Please sign in to comment.