[Bug]: Can't use ES_USE_ALIASES with spark-dependencies in jaeger operator

[ksponomarev]

What happened?

Hello
I can't enable ES_USE_ALIASES with

apiVersion: jaegertracing.io/v1
kind: Jaeger

for dependencies
I think I need it here

Steps to reproduce

1. Install jaeger-operator
2. Install jaeger
3. Set use-aliases: true
4. Run spark-dependencies job

Expected behavior

Spark-dependencies use aliases

Relevant log output

22/08/03 11:29:23 INFO ElasticsearchDependenciesJob: Running Dependencies job for 2022-08-03T00:00Z, reading from dtv-kaas-jaeger-span-read index, result storing to dtv-kaas-jaeger-dependencies-write

Screenshot

No response

Additional context

No response

Jaeger backend version

No response

SDK

No response

Pipeline

Application->OTEL Collector->Jaeger->Elasticsearch

Stogage backend

Elasticsearch

Operating system

Linux

Deployment model

Kubernetes

Deployment configs

apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
  name: jaeger
spec:
  strategy: production
  imagePullSecrets:
      - name: artifactory
  storage:
    type: elasticsearch
    dependencies:
      image: docker.io/jaegertracing/spark-dependencies:latest
      successfulJobsHistoryLimit: 1
      backoffLimit: 2
      schedule: "55 23 * * *"
      javaOpts: "-Djavax.net.ssl.trustStore=trust.store -Djavax.net.ssl.trustStorePassword=1111111111"
      resources:
        requests:
          memory: 4096Mi
        limits:
          memory: 4096Mi
    esRollover:
      image: docker.io/jaegertracing/jaeger-es-rollover:1.36.0
      conditions: "{\"max_age\": \"14d\"}"
      readTTL: 168h
      imagePullPolicy: Always
      imagePullSecrets:
        - name: artifactory
      backoffLimit: 2
      successfulJobsHistoryLimit: 1
      schedule: "55 22 * * *"
      resources:
        requests:
          cpu: 200m
          memory: 128Mi
    esIndexCleaner:
      image: docker.io/jaegertracing/jaeger-es-index-cleaner:1.36.0
      enabled: true
      backoffLimit: 2
      numberOfDays: 1
      successfulJobsHistoryLimit: 1
      schedule: "55 21 * * *"
      resources:
        requests:
          cpu: 200m
          memory: 128Mi
    options:
      es:
        server-urls: https://elsticsearch:9200
        tls:
          enabled: true
        username: admin
        password: pass
        index-prefix: dtv-kaas
        index-rollover-frequency-services: day
        index-rollover-frequency-spans: day
        use-aliases: true
        create-index-templates: false
  agent:
    image: docker.io/jaegertracing/jaeger-agent:1.36.0
    imagePullPolicy: Always
    resources:
      limits:
        cpu: 100m
        memory: 128Mi
    imagePullSecrets:
        - name: artifactory
    options:
      log-level: info
  ingress:
    enabled: true
    annotations:
      cert-manager.io/cluster-issuer: vaultpki-clusterissuer
    tls:
      - hosts:
        - jaeger.example.com
        secretName: jaeger.example.com
    hosts:
      - jaeger.example.com
  collector:
    maxReplicas: 5
    minReplicas: 1
    image: docker.io/jaegertracing/jaeger-collector:1.36.0
    imagePullPolicy: Always
    options:
      es:
        use-aliases: true
    resources:
      limits:
        cpu: 100m
        memory: 128Mi
  query:
    replicas: 3
    image: docker.io/jaegertracing/jaeger-query:1.36.0
    tracingEnabled: true
    imagePullPolicy: Always
    resources:
      limits:
        cpu: 200m
        memory: 256Mi
    options:
      prometheus:
          server-url: http://prometheus-stack-prometheus.prometheus-stack:9090
      es:
        use-aliases: true
    metricsStorage:
      type: prometheus
  ui:
      options:
        dependencies:
          menuEnabled: true

[avdeevi]

I have same problem.
jaegertracing/spark-dependencies ignore environment variable ES_USE_ALIASES=true

[ksponomarev]

I fixed this with rebuilding container with ENV ES_USE_ALIASES=true
But it's not cool

[brentmjohnson]

not sure if the operator eventually reconciles resources and overwrites this, but you can also patch the cronjob as a workaround like the following:

kubectl patch CronJob/jaeger-spark-dependencies -n observability --type json -p='[
  {"op": "add", "path": "/spec/jobTemplate/spec/template/spec/containers/0/env/-", "value": { "name": "ES_USE_ALIASES", "value": "true" }}
]'

[frzifus]

Seems the ES_USE_ALIAS option is simply ignored. Compare it with ES_INDEX_PREFIX.

    options:
      es:
        use-aliases: true
        index-prefix: dtv-kaas

Here the v1.37 source.

jaeger-operator/pkg/cronjob/spark_dependencies.go

Lines 167 to 183 in 58f9155

	case v1.JaegerESStorage:
	vars := []corev1.EnvVar{
	{Name: "ES_NODES", Value: sFlagsMap["es.server-urls"]},
	{Name: "ES_INDEX_PREFIX", Value: sFlagsMap["es.index-prefix"]},
	{Name: "ES_INDEX_DATE_SEPARATOR", Value: sFlagsMap["es.index-date-separator"]},
	{Name: "ES_USERNAME", Value: sFlagsMap["es.username"]},
	{Name: "ES_PASSWORD", Value: sFlagsMap["es.password"]},
	{Name: "ES_TIME_RANGE", Value: s.Dependencies.ElasticsearchTimeRange},
	}
	if s.Dependencies.ElasticsearchNodesWanOnly != nil {
	vars = append(vars, corev1.EnvVar{Name: "ES_NODES_WAN_ONLY", Value: strconv.FormatBool(*s.Dependencies.ElasticsearchNodesWanOnly)})
	}
	if s.Dependencies.ElasticsearchClientNodeOnly != nil {
	vars = append(vars, corev1.EnvVar{Name: "ES_CLIENT_NODE_ONLY", Value: strconv.FormatBool(*s.Dependencies.ElasticsearchClientNodeOnly)})
	}
	return vars
	default:

---

Would someone of you like to work on a patch? :)

[frzifus]

@miyunari, are you interested in working on this?

[miyunari]

yes I would be happy to work on this :)

[yakimba]

We are facing the same issue without jaeger-operator.
We have installed spark-dependencies as cronjob in kubernetes. But spark with env ES_USE_ALIASES: true tries to use index jaeger-span-dd-mm-yyyy, but we don't have such indexes. We only have indexes with names like jaeger-span-00000XX.
As a result spark always tolds that 0 dependencies found

[oncipriani]

We are facing the same issue without jaeger-operator. We have installed spark-dependencies as cronjob in kubernetes. But spark with env ES_USE_ALIASES: true tries to use index jaeger-span-dd-mm-yyyy, but we don't have such indexes. We only have indexes with names like jaeger-span-00000XX. As a result spark always tolds that 0 dependencies found

Exactly the same issue here. Installed using the Helm Chart.