New component: Envoy access log receiver

[zirain]

The purpose and use-cases of the new component

This receiver is design to receive logs from Envoy ALS.

Example configuration for the component

receivers:
  envoyaccesslog:
    protocols:
      grpc:
  envoyaccesslog/withendpoint:
    protocols:
      grpc:
        endpoint: 0.0.0.0:8080

Telemetry data types supported

logs

Code Owner(s)

No response

Sponsor (optional)

No response

Additional context

Both Envoy Gateway and Istio support expose access log with ALS, this would help them use Otel-collector as unify backend for collecting logs, metrics and tracing.

[evan-bradley]

Thanks for the details @zirain.

From what I can tell, Envoy and Istio both also support exporting access logs through OTLP. Where possible, we usually advise implementing OTLP directly since it reduces the requirement for maintaining support for a separate format in the application and Collector receiver. Could you comment on the difference in support for or usage of ALS and OTLP in each?

[zirain]

ALS is used widely in the past(before OTLP become an standard), and I alos believe is the future, and I think we alreay support most of the features in OTLP(if there's something missed, plan to support it once end users report).

In other words, it'd better to support ALS in OTel-collector to support better adoption of OTLP, with this users can migrate between ALS and OTLP smoothly.

[evan-bradley]

it'd better to support ALS in OTel-collector to support better adoption of OTLP, with this users can migrate between ALS and OTLP smoothly.

Just so I can understand a little better, when would users emply a Collector for this? I would, with no prior knowledge, assume that users can configure Envoy/Istio to export OTLP nearly as easily as they can update their ALS export configuration to point to a Collector endpoint. For users who want to ingest OTLP, but still export ALS from Envoy/Istio, I would assume they fall into one of a few buckets:

1. Don't want to update their Envoy/Istio configurations, but can update a given ALS log ingest endpoint to actually be a Collector.
2. Are running versions of Envoy/Istio that don't support OTLP
3. Want to take advantage of functionality in ALS not available with OTLP
4. Can send logs to multiple ALS endpoints inside Envoy/Istio, but can't send both ALS and OTLP at the same time

Do any of these situations apply, or are there other scenarios where users exporting ALS logs would want a Collector to convert those to OTLP? I just want to make sure I can explain when users should opt for a Collector with this receiver instead of using native Envoy/Istio OTLP exporting.

[zirain]

You're absolutely right.

In the case of the EG project, I initially insisted on introducing only OTLP, but users kept asking for ALS support.

By introducing this, end users also benefit from other collector components(e.g. built-in processors, OTTL) instread of doing everything themself in their custom backend.

[evan-bradley]

If users want to continue to use ALS, does that mean we should also include an ALS exporter in the Collector?

[zirain]

I think a receiver is enough for now, let us see what will happen in the future.

[evan-bradley]

Could you go a bit more in detail about the backends users are sending to? I'm still not totally clear on the entire picture.

It sounds like currently users are sending logs over ALS from their EG instances to a custom backend. If we introduce a Collector in the middle, what format will the Collector export? Do their custom backends support OTLP?

[zirain]

I have a demo project show what backends users need to do, and another one in EG project.

I want to use OTel-collector for them both.

[evan-bradley]

From an offline discussion with @zirain: A common setup for Envoy Gateway users is to have: EG --> ALS converter --> ELK/Loki. The ALS converter assumes a similar role to the Collector and converts ALS to a format that can be sent to the backend datastore. The goal with this receiver is that the Collector can replace these converters and users can take advantage of the rest of the Collector's functionality.

I'll add the "Accepted Component" label and act as the sponsor for this component.

[zirain]

@evan-bradley thanks for act as sponsor this component, will start my work next week.