[Bug]: Docker compose file not working with adaptive Sampling

[slayer321]

What happened?

I'm trying to use Adaptive Sampling in Jaeger and following the blog I found this docker-compose file where we can run jaeger specifying SAMPLING_CONFIG_TYPE=adaptive env (the only changes I did in the collector env is I added this two env as well SPAN_STORAGE_TYPE=cassandra , SAMPLING_STORAGE_TYPE=cassandra ) After running this docker-compose file when I'm seeing the UI I cannot see any of the services ,on the HotROD application I tried booking some rides so that traces get generated but nothing happened .

I also check the cassendra database and below is the output I'm getting

cqlsh:jaeger_v1_dc1> DESCRIBE TABLES;

dependencies_v2  operation_names_v2      service_name_index       tag_index
duration_index   operation_throughput    service_names            traces   
leases           sampling_probabilities  service_operation_index

cqlsh:jaeger_v1_dc1> select * from service_names;

 service_name
--------------

(0 rows)
cqlsh:jaeger_v1_dc1> select * from service_name_index;

 service_name | bucket | start_time | trace_id
--------------+--------+------------+----------

(0 rows)
cqlsh:jaeger_v1_dc1> select * from traces;

 trace_id | span_id | span_hash | duration | flags | logs | operation_name | parent_id | process | refs | start_time | tags
----------+---------+-----------+----------+-------+------+----------------+-----------+---------+------+------------+------

(0 rows)

After checking the logs of hotROD container below is the error I'm getting

2023/09/20 09:36:12 traces export: Post "http://localhost:4318/v1/traces": dial tcp 127.0.0.1:4318: connect: connection refused
2023/09/20 09:36:13 traces export: Post "http://localhost:4318/v1/traces": dial tcp 127.0.0.1:4318: connect: connection refused
2023/09/20 09:36:13 traces export: Post "http://localhost:4318/v1/traces": dial tcp 127.0.0.1:4318: connect: connection refused
2023/09/20 09:36:13 traces export: Post "http://localhost:4318/v1/traces": dial tcp 127.0.0.1:4318: connect: connection refused

After checking for hotrod docker compose file in example dir I found that there was some recent changes done for hotrod default exporter.I tried making same changes to the above docker compose file .. but it didn't help .

Steps to reproduce

1. Run docker compose up from docker-compose dir
2. On the Jaeger UI no services are seen
3. Try booking ride on hotrod application no traces are seen on the UI

Expected behavior

Once we run docker compose up command all the services should be seen on Jaeger UI and then the traces should also be seen.

Relevant log output

No response

Screenshot

No response

Additional context

No response

Jaeger backend version

v1.49.0

SDK

No response

Pipeline

No response

Stogage backend

Cassendra

Operating system

Linux

Deployment model

Docker

Deployment configs

jaeger-docker-compose.yaml

[yurishkuro]

Action items:

• find out why this was not caught in CI, I thought the reason hotrod is included in that docker-compose is because we were validating it somehow
• fix the settings in docker-compose to use OTEL env vars, not the deprecated Jaeger SDK env vars
• validate that adaptive sampling works with OTEL SDK (again, I thought we had a CI test for that)

[james-ryans]

Hi! I've encountered this issue too while trying to tackle the other problem. I don't think this issue is specific to adaptive sampling, we'll get caught into this issue even by running docker compose up without modifying anything. After investigating a bit, the cause of this issue is that COLLECTOR_OTLP_ENABLED=true env was removed at https://github.com/jaegertracing/jaeger/pull/4698/files#diff-299f817cc4ab077ddb763f1e6a023d9d042d714e2fd3736cc40af3f218d44f1eL9 PR. Actually, it is my jaegertracing/all-in-one:latest image is not updated.

Investigated items:

1. I can't find any integration test between hotrod and all-in-one services. What I found is that hotrod tests only the frontend side scripts/hotrod-integration-test.sh
2. Is it enough if I only bring back the COLLECTOR_OTLP_ENABLED=true env? I'll try to continue investigating this issue.
3. I don't think hotrod with OTEL SDK works with adaptive sampling. I've tried to add OTEL_TRACES_SAMPLER=jaeger_remote env(found this env here) but output unsupported sampler: jaeger_remote, seems like hotrod using go.opentelemetry.io/otel package while jaeger_remote only supported in go.opentelemetry.io/contrib package

hotrod-hotrod-1  | 2023-10-05T15:49:18.529Z	INFO	[email protected]/command.go:946	Using expvar as metrics backend
hotrod-hotrod-1  | 2023-10-05T15:49:18.529Z	INFO	[email protected]/command.go:916	Starting all services
hotrod-hotrod-1  | 2023/10/05 15:49:18 unsupported sampler: jaeger_remote
hotrod-hotrod-1  | 2023/10/05 15:49:18 unsupported sampler: jaeger_remote
hotrod-hotrod-1  | 2023-10-05T15:49:18.530Z	INFO	frontend/server.go:74	Starting	{"service": "frontend", "address": "http://0.0.0.0:8080"}
hotrod-hotrod-1  | 2023/10/05 15:49:18 unsupported sampler: jaeger_remote
hotrod-hotrod-1  | 2023-10-05T15:49:18.530Z	INFO	customer/server.go:55	Starting	{"service": "customer", "address": "http://0.0.0.0:8081"}
hotrod-hotrod-1  | 2023/10/05 15:49:18 unsupported sampler: jaeger_remote
hotrod-hotrod-1  | 2023/10/05 15:49:18 unsupported sampler: jaeger_remote
hotrod-hotrod-1  | 2023/10/05 15:49:18 unsupported sampler: jaeger_remote
hotrod-hotrod-1  | 2023-10-05T15:49:18.531Z	INFO	route/server.go:57	Starting	{"service": "route", "address": "http://0.0.0.0:8083"}

[varshith257]

@yurishkuro
#4803 (comment)

Can it be closed?

[yurishkuro]

why? None of the checklist items seem to be addressed: #4768 (comment)