Argument order gets changed causing a collector re-deployment

[jkandasa]

Affected services: Jaeger-collector, Jaeger-query

Issue 1:
When we deploy jaeger services by the operator, immediate redeploy is happening
Issue 2:
Jaeger services get redeployed by the operator in a random interval (approx: 10 minutes)

CR:

apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
  name: jaegerqe
spec:
  ingress:
    security: none
  strategy: production
  collector:
    replicas: 4
    image: jaegertracing/jaeger-collector:1.11
    options:
      log-level: debug
      metrics-backend: expvar
      collector:
        num-workers: 50
        queue-size: 20000
      es:
        bulk:
          size: 5000000
          workers: 10
          flush-interval: 200ms
  query:
    replicas: 1
    image: jaegertracing/jaeger-query:1.11
    options:
      log-level: debug
      metrics-backend: expvar
      query:
        port: 16686
  agent:
    strategy: sidecar
    image: jaegertracing/jaeger-agent:1.11
    options:
      log-level: debug
      metrics-backend: expvar
  storage:
    type: elasticsearch
    esIndexCleaner:
      enabled: false
    sparkDependencies:
      enabled: false
    options:
      es:
        server-urls: http://elasticsearch:9200

logs:

• jaeger-operator.log
• watch-pods.log
• query.log
• collector.log

Similar issue: #307

[jpkrohling]

There are indeed two revisions for the jaegerqe-collector:

$ kubectl get deployments -o yaml jaegerqe-collector | grep deployment.kubernetes.io/revision
    deployment.kubernetes.io/revision: "2"

This is the first revision:

deployments "jaegerqe-collector" with revision #1
Pod Template:
  Labels:	app=jaeger
	app.kubernetes.io/component=collector
	app.kubernetes.io/instance=jaegerqe
	app.kubernetes.io/managed-by=jaeger-operator
	app.kubernetes.io/name=jaegerqe-collector
	app.kubernetes.io/part-of=jaeger
	pod-template-hash=730370291
  Annotations:	prometheus.io/port=14268
	prometheus.io/scrape=true
	sidecar.istio.io/inject=false
  Service Account:	jaegerqe
  Containers:
   jaeger-collector:
    Image:	jaegertracing/jaeger-collector:1.11
    Ports:	9411/TCP, 14267/TCP, 14268/TCP
    Host Ports:	0/TCP, 0/TCP, 0/TCP
    Args:
      --collector.num-workers=50
      --collector.queue-size=20000
      --es.bulk.flush-interval=200ms
      --es.bulk.size=5000000
      --es.bulk.workers=10
      --log-level=debug
      --metrics-backend=expvar
      --es.server-urls=http://elasticsearch:9200
      --sampling.strategies-file=/etc/jaeger/sampling/sampling.json
    Readiness:	http-get http://:14269/ delay=1s timeout=1s period=10s #success=1 #failure=3
    Environment:
      SPAN_STORAGE_TYPE:	elasticsearch
      COLLECTOR_ZIPKIN_HTTP_PORT:	9411
    Mounts:
      /etc/jaeger/sampling from jaegerqe-sampling-configuration-volume (ro)
  Volumes:
   jaegerqe-sampling-configuration-volume:
    Type:	ConfigMap (a volume populated by a ConfigMap)
    Name:	jaegerqe-sampling-configuration
    Optional:	false

And this is the second revision:

deployments "jaegerqe-collector" with revision #2
Pod Template:
  Labels:	app=jaeger
	app.kubernetes.io/component=collector
	app.kubernetes.io/instance=jaegerqe
	app.kubernetes.io/managed-by=jaeger-operator
	app.kubernetes.io/name=jaegerqe-collector
	app.kubernetes.io/part-of=jaeger
	pod-template-hash=3132922151
  Annotations:	prometheus.io/port=14268
	prometheus.io/scrape=true
	sidecar.istio.io/inject=false
  Service Account:	jaegerqe
  Containers:
   jaeger-collector:
    Image:	jaegertracing/jaeger-collector:1.11
    Ports:	9411/TCP, 14267/TCP, 14268/TCP
    Host Ports:	0/TCP, 0/TCP, 0/TCP
    Args:
      --es.bulk.workers=10
      --log-level=debug
      --metrics-backend=expvar
      --collector.num-workers=50
      --collector.queue-size=20000
      --es.bulk.flush-interval=200ms
      --es.bulk.size=5000000
      --es.server-urls=http://elasticsearch:9200
      --sampling.strategies-file=/etc/jaeger/sampling/sampling.json
    Readiness:	http-get http://:14269/ delay=1s timeout=1s period=10s #success=1 #failure=3
    Environment:
      SPAN_STORAGE_TYPE:	elasticsearch
      COLLECTOR_ZIPKIN_HTTP_PORT:	9411
    Mounts:
      /etc/jaeger/sampling from jaegerqe-sampling-configuration-volume (ro)
  Volumes:
   jaegerqe-sampling-configuration-volume:
    Type:	ConfigMap (a volume populated by a ConfigMap)
    Name:	jaegerqe-sampling-configuration
    Optional:	false

They are pretty much the same, except for the order of the arguments. @objectiser, @pavolloffay are we sorting them somewhere, or is OpenShift doing this sort?

[objectiser]

@jpkrohling Not that I am aware of - the only manipulation that is being done, which might impact order, is to extract the storage relevant options. Perhaps before adding to the spec, we need to sort them just to make sure?