[extension/bearertokenauth] Bearer token auth extension does not pick up the updated token from file

[saikrishna397]

Component(s)

extension/bearertokenauth

What happened?

Description

Bearer token auth extension is not picking up the updated token and instead it fetches the token once and that's re-used forever.
The issue is related to using bearer token auth for http clients and works fine for gRPC.

Steps to Reproduce

1. Use bearer token auth extension that gets the token from a file.

config:
  extensions:
    bearertokenauth:
      filename: "/etc/secrets/prometheus/<FILE_NAME>"
  exporters:
    prometheusremotewrite:
      auth: 
        authenticator: bearertokenauth
      endpoint: ""

2. Check if the bearer token auth picks up the token and sends it along with the request.
3. Update the contents of the token and verify if the new token is updated in the request.

Expected Result

Request from the bearer token auth should have the updated bearer token.

Actual Result

Request from the bearer token auth has the old token.
When I used a valid token and tried to send data to a prometheus remote write EP, it failed with the following error:

Exporting failed. The error is not retryable. Dropping data.  {"kind": "exporter", "data_type": "metrics", "name": "prometheusremotewrite", "error │
│ ": "Permanent error: Permanent error: remote write returned HTTP status 401 Unauthorized; err = %!w(<nil>): {\"Error\":{\"Code\":\"TokenExpired\",\"Message\":\"IDX10223: Lifetime validation failed. The token is expired. Val │
│ idTo: 'System.DateTime', Current time: 'System.DateTime'.\"}}", "dropped_items": 29}

Collector version

0.66.0

Environment information

Environment

OS: macOS Monterey

OpenTelemetry Collector configuration

config:
    extensions:
      health_check:
      pprof:
      memory_ballast: {}
      bearertokenauth:
        filename: "/etc/secrets/prometheus/TOKEN"
    receivers:
      otlp:
        protocols:
          grpc:
            endpoint: 0.0.0.0:4317
      kubeletstats:
        collection_interval: 30s
        auth_type: "serviceAccount"
        endpoint: "https://${NODE_NAME}:10250"
        insecure_skip_verify: true
      prometheus:
        config:
          scrape_configs:
          - job_name: "collector-metrics"
            scrape_interval: 15s
            static_configs:
            - targets: ['localhost:8888']
          - job_name: "otel-collector"
            scrape_interval: 15s
            kubernetes_sd_configs:
            - role: pod
              selectors:
              - role: pod
                field: "spec.nodeName=$NODE_NAME"
            relabel_configs:
            # scrape pods annotated with "prometheus.io/scrape: true"
            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
              regex: "true"
              action: keep
            # read the port from "prometheus.io/port: <port>" annotation and update scraping address accordingly
            - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
              action: replace
              target_label: __address__
              regex: ([^:]+)(?::\d+)?;(\d+)
              # escaped $1:$2
              replacement: $$1:$$2
            - source_labels: [__meta_kubernetes_namespace]
              action: replace
              target_label: kubernetes_namespace
            - source_labels: [__meta_kubernetes_pod_name]
              action: replace
              target_label: kubernetes_pod_name
    processors:
      batch:
      memory_limiter:
        check_interval: 1s
        limit_percentage: 80
        spike_limit_percentage: 30
      resourcedetection:
        detectors:
        - env
        timeout: 2s
        override: false
    exporters:
      prometheusremotewrite:
        auth: 
          authenticator: bearertokenauth
        endpoint: "<AZURE_PROMETHEUS_ENDPOINT>"
        resource_to_telemetry_conversion:
          enabled: true
    service:
      extensions: [health_check, pprof, bearertokenauth]
      pipelines:
        metrics:
          receivers: [prometheus, otlp, kubeletstats]
          processors: [memory_limiter, batch, resourcedetection]
          exporters: [prometheusremotewrite]

Log output

Exporting failed. The error is not retryable. Dropping data.  {"kind": "exporter", "data_type": "metrics", "name": "prometheusremotewrite", "error │
│ ": "Permanent error: Permanent error: remote write returned HTTP status 401 Unauthorized; err = %!w(<nil>): {\"Error\":{\"Code\":\"TokenExpired\",\"Message\":\"IDX10223: Lifetime validation failed. The token is expired. Val │
│ idTo: 'System.DateTime', Current time: 'System.DateTime'.\"}}", "dropped_items": 29}

Additional context

N/A

[frzifus]

/label extension/bearertokenauth

[jpkrohling]

Good catch. I see why:

opentelemetry-collector-contrib/extension/bearertokenauthextension/bearertokenauth.go

Lines 178 to 199 in 669a5b7

	func (b *BearerTokenAuth) RoundTripper(base http.RoundTripper) (http.RoundTripper, error) {
	return &BearerAuthRoundTripper{
	baseTransport: base,
	bearerToken: b.bearerToken(),
	}, nil
	}
	// BearerAuthRoundTripper intercepts and adds Bearer token Authorization headers to each http request.
	type BearerAuthRoundTripper struct {
	baseTransport http.RoundTripper
	bearerToken string
	}
	// RoundTrip modifies the original request and adds Bearer token Authorization headers.
	func (interceptor *BearerAuthRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
	req2 := req.Clone(req.Context())
	if req2.Header == nil {
	req2.Header = make(http.Header)
	}
	req2.Header.Set("Authorization", interceptor.bearerToken)
	return interceptor.baseTransport.RoundTrip(req2)
	}

We should instead be calling b.bearerToken() within the RoundTrip function, to get the updated token. How do you feel about sending in a PR?

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• extension/bearertokenauth: @jpkrohling @pavankrish123

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[jpkrohling]

I'm working on this.

[saikrishna397]

I'm working on this.

This is fixed in #17052. Closing this issue.