README.md

AWS Kinesis Data Firehose Receiver

Status
Stability	alpha: metrics, logs
Distributions	contrib
Issues
Code Owners	@Aneurysm9

Receiver for ingesting AWS Kinesis Data Firehose delivery stream messages and parsing the records received based on the configured record type.

Configuration

Example:

receivers:
  awsfirehose:
    endpoint: 0.0.0.0:4433
    record_type: cwmetrics
    access_key: "some_access_key"
    tls:
      cert_file: server.crt
      key_file: server.key

The configuration includes the Opentelemetry collector's server confighttp, which allows for a variety of settings. Only the most relevant ones will be discussed here, but all are available. The AWS Kinesis Data Firehose Delivery Streams currently only support HTTPS endpoints using port 443. This can be potentially circumvented using a Load Balancer.

endpoint:

The address:port to bind the listener to.

default: localhost:4433

See our security best practices doc to understand how to set the endpoint in different environments.

tls:

See documentation for more details.

A cert_file and key_file are required.

record_type:

The type of record being received from the delivery stream. Each unmarshaler handles a specific type, so the field allows the receiver to use the correct one.

default: cwmetrics

See the Record Types section for all available options.

access_key (Optional):

The access key to be checked on each request received. This can be set when creating or updating the delivery stream. See documentation for details.

Record Types

cwmetrics

The record type for the CloudWatch metric stream. Expects the format for the records to be JSON. See documentation for details.

cwlogs

The record type for the CloudWatch log stream. Expects the format for the records to be JSON. For example:

{
  "messageType": "DATA_MESSAGE",
  "owner": "111122223333",
  "logGroup": "my-log-group",
  "logStream": "my-log-stream",
  "subscriptionFilters": ["my-subscription-filter"],
  "logEvents": [
    {
      "id": "123",
      "timestamp": 1725544035523,
      "message": "My log message."
    }
  ]
}

otlp_v1

The OTLP v1 format as produced by CloudWatch metric streams. See documentation for details.