Jeager All in One with Opensearch Backend

[osmanghoni1]

Hello, I am trying to get a Jaeger All in One Container running with an Opensearch storage backend running together using docker.

This is my docker-compose file.

version: '3'
services:
  opensearch-node:
    image: opensearchproject/opensearch:latest 
    container_name: opensearch-node
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node1
      - discovery.type=single-node
    ports:
      - "9200:9200"
      - "9600:9600"
    networks:
      - opensearch-net

  jaeger:
    container_name: jeager-opensearch
    image: jaegertracing/all-in-one:1.44.0
    ports:
      - "16686:16686"
      - "4317:4317"
      - "4318:4318"
      - "5775:5775"
      - "5778:5778"
      - "14250:14250"
      - "14268:14268"
      - "6831:6831"
      - "6832:6832"
    environment:
      SPAN_STORAGE_TYPE: "opensearch"
      ES_SERVER_URLS: https://opensearch-node:9200
      ES_USERNAME: username
      ES_PASSWORD: password
      ES_TLS_SKIP_HOST_VERIFY: "true"
      ES_TLS_ENABLED: "true"
      ES_TAGS_AS_FIELDS_ALL: "true"
    networks:
      - opensearch-net

networks:
  opensearch-net:

Unfortunately the Jaeger container looks to be having issues connecting to Opensearch

Here is the error I get
jeager-opensearch | {"level":"fatal","ts":1683580577.938551,"caller":"./main.go:109","msg":"Failed to init storage factory","error":"failed to create primary Elasticsearch client: health check timeout: Head \"https://opensearch-node:9200\": dial tcp 172.18.0.2:9200: connect: connection refused: no Elasticsearch node available","stacktrace":"main.main.func1\n\t./main.go:109\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/[email protected]/command.go:916\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/[email protected]/command.go:1044\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/[email protected]/command.go:968\nmain.main\n\t./main.go:240\nruntime.main\n\truntime/proc.go:250"} jeager-opensearch exited with code 1

I have tried disabling the security plugin in opensearch and disabling tls in jeager but get the same output.

I am able to run the following from a jeager all in one using in memory storage on the same network as opensearch

/ # curl -X GET "https://opensearch-node:9200" -ku username:password

{
  "name" : "opensearch-node1",
  "cluster_name" : "opensearch-cluster",
  "cluster_uuid" : "eQL3wCeMQm6aCtxf2M-WrA",
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.6.0",
    "build_type" : "tar",
    "build_hash" : "7203a5af21a8a009aece1474446b437a3c674db6",
    "build_date" : "2023-02-24T18:57:04.388618985Z",
    "build_snapshot" : false,
    "lucene_version" : "9.5.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

Any help and guidance is appreciated.

[osmanghoni1]

I was able to get this working with the following docker-compose.yml

version: '3'
services:
  opensearch-node:
    image: opensearchproject/opensearch:latest 
    container_name: opensearch-node
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node
      - discovery.type=single-node
    ports:
      - "9200:9200"
      - "9600:9600"
    networks:
      - opensearch-net

  jaeger:
    container_name: jeager-opensearch
    image: jaegertracing/all-in-one:1.44.0
    ports:
      - "16685:16685"
      - "16686:16686"
      - "16687:16687"
      - "4317:4317"
      - "4318:4318"
      - "5775:5775"
      - "5778:5778"
      - "14250:14250"
      - "14268:14268"
      - "14269:14269"
      - "6831:6831"
      - "6832:6832"
    environment:
      SPAN_STORAGE_TYPE: "opensearch"
      ES_SERVER_URLS: https://opensearch-node:9200
      ES_USERNAME: admin
      ES_PASSWORD: admin
      ES_TLS_SKIP_HOST_VERIFY: "true"
      ES_TLS_ENABLED: "true"
      ES_TAGS_AS_FIELDS_ALL: "true"
      ES_INDEX_PREFIX: jeager-dev
    depends_on:
      - opensearch-node
    restart: on-failure
    networks:
      - opensearch-net

networks:
  opensearch-net:

One important thing was to allow the opensearch container to run and finish setting up before the jaeger container starts up.

[Faisal-developer]

@osmanghoni1 I am getting following error after using you .yml file
{"level":"info","ts":1690828133.087098,"caller":"[email protected]/server.go:935","msg":"[core][Server #1] grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams received bogus greeting from client: \"\\x16\\x03\\x01\\x02\\x00\\x01\\x00\\x01\\xfc\\x03\\x03\\xf7\\xcdK\\f\\xce%\\xb5Ӂ\\x01\\x1bY\\x1d\""","system":"grpc","grpc_log":true}
{"level":"info","ts":1690828133.0890238,"caller":"[email protected]/server.go:935","msg":"[core][Server #1] grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams received bogus greeting from client: \"\\x16\\x03\\x01\\x02\\x00\\x01\\x00\\x01\\xfc\\x03\\x03K\\xbd<\\x93\\xbd)$\\xb25\\xb54\\xbe\\xad\""","system":"grpc","grpc_log":true}
{"level":"info","ts":1690828137.0771334,"caller":"[email protected]/server.go:935","msg":"[core][Server #1] grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams received bogus greeting from client: \"GET / HTTP/1.1\\r\\nHost: lo\""","system":"grpc","grpc_log":true}
{"level":"info","ts":1690828138.1196744,"caller":"[email protected]/server.go:935","msg":"[core][Server #1] grpc: Server.Serve failed to create ServerTransport: connection error: desc = "transport: http2Server.HandleStreams received bogus greeting from client: \"GET / HTTP/1.1\\r\\nHost: lo\""","system":"grpc","grpc_log":true}

[rudolphm]

I tried using the docker-compose.yml file you put here but the jaeger-opensearch container cannot connect to opensearch with this error in the logs:

2023-09-29 12:14:42 {"level":"fatal","ts":1695982482.1504517,"caller":"./main.go:109","msg":"Failed to init storage factory","error":"failed to create primary Elasticsearch client: health check timeout: Head \"https://opensearch-node:9200\": dial tcp 192.168.32.2:9200: connect: connection refused: no Elasticsearch node available","stacktrace":"main.main.func1\n\t./main.go:109\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/[email protected]/command.go:916\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/[email protected]/command.go:1044\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/[email protected]/command.go:968\nmain.main\n\t./main.go:240\nruntime.main\n\truntime/proc.go:250"}

[yurishkuro]

ES_SERVER_URLS: https://opensearch-node:9200

Are you sure you want httpS in this setup?

[rudolphm]

No, I don't want https I tried using http but it's giving me the same error message

…

________________________________ From: Yuri Shkuro ***@***.***> Sent: Friday, September 29, 2023 5:03:10 PM To: jaegertracing/jaeger ***@***.***> Cc: infemuswulf ***@***.***>; Comment ***@***.***> Subject: Re: [jaegertracing/jaeger] Jeager All in One with Opensearch Backend (Discussion #4447) ES_SERVER_URLS: https://opensearch-node:9200 Are you sure you want httpS in this setup? — Reply to this email directly, view it on GitHub<#4447 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABSREORMBDQDCQTSTXIMTHTX43PKRANCNFSM6AAAAAAX27UASU>. You are receiving this because you commented.Message ID: ***@***.***>