Skip to content

Commit

Permalink
[azure-eventhub] Make the azure-eventhub input v1 tracer opt-in (#41932)
Browse files Browse the repository at this point in the history
Make the `azure-eventhub` input v1 internal logs tracer opt-in. To enable internal (legacy) Event Hub SDK logs, now you need to set the following environment variable:

```sh
BEATS_AZURE_EVENTHUB_INPUT_TRACING_ENABLED: true
```

Before this change, the tracer was enabled by default.

A few customers reported higher than anticipated error logs in edge cases. These error logs are an essential signal to troubleshoot the root cause of several issues. However, we should give the option to completely opt out from the internal logs tracing when needed.
  • Loading branch information
zmoog authored Dec 6, 2024
1 parent 583d345 commit 3b5823f
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 2 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -358,6 +358,7 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Added support for retry configuration in GCS input. {issue}11580[11580] {pull}41862[41862]
- Improve S3 polling mode states registry when using list prefix option. {pull}41869[41869]
- AWS S3 input registry cleanup for untracked s3 objects. {pull}41694[41694]
- The environment variable `BEATS_AZURE_EVENTHUB_INPUT_TRACING_ENABLED: true` enables internal logs tracer for the azure-eventhub input. {issue}41931[41931] {pull}41932[41932]

*Auditbeat*

Expand Down
5 changes: 4 additions & 1 deletion x-pack/filebeat/docs/inputs/input-azure-eventhub.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,10 @@ Users can make use of the `azure-eventhub` input in order to read messages from
The azure-eventhub input implementation is based on the the event processor host (EPH is intended to be run across multiple processes and machines while load balancing message consumers more on this here https://github.com/Azure/azure-event-hubs-go#event-processor-host, https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-event-processor-host).
State such as leases on partitions and checkpoints in the event stream are shared between receivers using an Azure Storage container. For this reason, as a prerequisite to using this input, users will have to create or use an existing storage account.


Users can enable internal logs tracing for this input by setting the environment
variable `BEATS_AZURE_EVENTHUB_INPUT_TRACING_ENABLED: true`. When enabled,
this input will log additional information to the logs. Additional information
includes partition ownership, blob lease information, and other internal state.


Example configuration:
Expand Down
8 changes: 7 additions & 1 deletion x-pack/filebeat/input/azureeventhub/tracer.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,20 @@ package azureeventhub

import (
"context"
"os"

"github.com/devigned/tab"

"github.com/elastic/elastic-agent-libs/logp"
)

func init() {
tab.Register(new(logsOnlyTracer))
// Register the logs tracer only if the environment variable is
// set to avoid the overhead of the tracer in environments where
// it's not needed.
if os.Getenv("BEATS_AZURE_EVENTHUB_INPUT_TRACING_ENABLED") == "true" {
tab.Register(new(logsOnlyTracer))
}
}

// logsOnlyTracer manages the creation of the required
Expand Down

0 comments on commit 3b5823f

Please sign in to comment.