Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing tests: TestHarness_MetricsGoldenData/otlp-otlp #9557

Closed
codeboten opened this issue Apr 27, 2022 · 1 comment · Fixed by #9564
Closed

Failing tests: TestHarness_MetricsGoldenData/otlp-otlp #9557

codeboten opened this issue Apr 27, 2022 · 1 comment · Fixed by #9564
Assignees
@codeboten
Labels
flaky test a test is flaky

Comments

@codeboten
Copy link
Contributor

This test has been failing intermittently, see example:

https://github.com/open-telemetry/opentelemetry-collector-contrib/runs/6196786412?check_suite_focus=true

==================
WARNING: DATA RACE
Write at 0x000006d82c10 by goroutine 22:
  google.golang.org/grpc/grpclog.SetLoggerV2()
      /home/runner/go/pkg/mod/google.golang.org/[email protected]/grpclog/loggerv2.go:76 +0x11a
  go.opentelemetry.io/collector/service/internal/telemetrylogs.SetColGRPCLogger()
      /home/runner/go/pkg/mod/go.opentelemetry.io/[email protected]/service/internal/telemetrylogs/logger.go:76 +0xf6
  go.opentelemetry.io/collector/service.(*Collector).setupConfigurationComponents()
      /home/runner/go/pkg/mod/go.opentelemetry.io/[email protected]/service/collector.go:208 +0x3f2
  go.opentelemetry.io/collector/service.(*Collector).Run()
      /home/runner/go/pkg/mod/go.opentelemetry.io/[email protected]/service/collector.go:249 +0x2dd
  github.com/open-telemetry/opentelemetry-collector-contrib/testbed/testbed.(*inProcessCollector).Start.func1()
      /home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/testbed/in_process_collector.go:96 +0xc4

Previous read at 0x000006d82c10 by goroutine 21:
  google.golang.org/grpc/internal/grpclog.InfoDepth()
      /home/runner/go/pkg/mod/google.golang.org/[email protected]/internal/grpclog/grpclog.go:37 +0x93
  google.golang.org/grpc/grpclog.(*componentData).InfoDepth()
      /home/runner/go/pkg/mod/google.golang.org/[email protected]/grpclog/component.go:36 +0x173
  google.golang.org/grpc/internal/channelz.AddTraceEvent()
      /home/runner/go/pkg/mod/google.golang.org/[email protected]/internal/channelz/funcs.go:340 +0x402
  google.golang.org/grpc/internal/channelz.Info()
      /home/runner/go/pkg/mod/google.golang.org/[email protected]/internal/channelz/logging.go:35 +0xb3
  google.golang.org/grpc.(*Server).Serve()
      /home/runner/go/pkg/mod/google.golang.org/[email protected]/server.go:776 +0x486
  go.opentelemetry.io/collector/receiver/otlpreceiver.(*otlpReceiver).startGRPCServer.func1()
      /home/runner/go/pkg/mod/go.opentelemetry.io/[email protected]/receiver/otlpreceiver/otlp.go:80 +0xd8

Goroutine 22 (running) created at:
  github.com/open-telemetry/opentelemetry-collector-contrib/testbed/testbed.(*inProcessCollector).Start()
      /home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/testbed/in_process_collector.go:94 +0x648
  github.com/open-telemetry/opentelemetry-collector-contrib/testbed/correctnesstests/metrics.(*correctnessTestCase).startCollector()
      /home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/correctnesstests/metrics/correctness_test_case.go:57 +0x53e
  github.com/open-telemetry/opentelemetry-collector-contrib/testbed/correctnesstests/metrics.testWithMetricsGoldenDataset()
      /home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/correctnesstests/metrics/metrics_correctness_test.go:72 +0x3be
  github.com/open-telemetry/opentelemetry-collector-contrib/testbed/correctnesstests/metrics.TestHarness_MetricsGoldenData.func1()
      /home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/correctnesstests/metrics/metrics_correctness_test.go:44 +0xac
  testing.tRunner()
      /opt/hostedtoolcache/go/1.18.0/x64/src/testing/testing.go:1439 +0x213
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.18.0/x64/src/testing/testing.go:1486 +0x47

Goroutine 21 (running) created at:
  go.opentelemetry.io/collector/receiver/otlpreceiver.(*otlpReceiver).startGRPCServer()
      /home/runner/go/pkg/mod/go.opentelemetry.io/[email protected]/receiver/otlpreceiver/otlp.go:77 +0x1eb
  go.opentelemetry.io/collector/receiver/otlpreceiver.(*otlpReceiver).startProtocolServers()
      /home/runner/go/pkg/mod/go.opentelemetry.io/[email protected]/receiver/otlpreceiver/otlp.go:127 +0x50d
  go.opentelemetry.io/collector/receiver/otlpreceiver.(*otlpReceiver).Start()
      /home/runner/go/pkg/mod/go.opentelemetry.io/[email protected]/receiver/otlpreceiver/otlp.go:155 +0x44
  go.opentelemetry.io/collector/internal/sharedcomponent.(*SharedComponent).Start.func1()
      /home/runner/go/pkg/mod/go.opentelemetry.io/[email protected]/internal/sharedcomponent/sharedcomponent.go:74 +0x8d
  sync.(*Once).doSlow()
      /opt/hostedtoolcache/go/1.18.0/x64/src/sync/once.go:68 +0x101
  sync.(*Once).Do()
      /opt/hostedtoolcache/go/1.18.0/x64/src/sync/once.go:59 +0x46
  go.opentelemetry.io/collector/internal/sharedcomponent.(*SharedComponent).Start()
      /home/runner/go/pkg/mod/go.opentelemetry.io/[email protected]/internal/sharedcomponent/sharedcomponent.go:73 +0xc8
  github.com/open-telemetry/opentelemetry-collector-contrib/testbed/testbed.(*BaseOTLPDataReceiver).Start()
      /home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/testbed/receivers.go:110 +0xb33
  github.com/open-telemetry/opentelemetry-collector-contrib/testbed/correctnesstests/metrics.(*correctnessTestCase).startTestbedReceiver()
      /home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/correctnesstests/metrics/correctness_test_case.go:77 +0x14b
  github.com/open-telemetry/opentelemetry-collector-contrib/testbed/correctnesstests/metrics.testWithMetricsGoldenDataset()
      /home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/correctnesstests/metrics/metrics_correctness_test.go:71 +0x3b1
  github.com/open-telemetry/opentelemetry-collector-contrib/testbed/correctnesstests/metrics.TestHarness_MetricsGoldenData.func1()
      /home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/testbed/correctnesstests/metrics/metrics_correctness_test.go:44 +0xac
  testing.tRunner()
      /opt/hostedtoolcache/go/1.18.0/x64/src/testing/testing.go:1439 +0x213
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.18.0/x64/src/testing/testing.go:1486 +0x47
==================
@codeboten codeboten added the flaky test a test is flaky label Apr 27, 2022
@codeboten codeboten self-assigned this Apr 27, 2022
@dmitryax
Copy link
Member

Seen here as well https://github.com/open-telemetry/opentelemetry-collector-contrib/runs/6202444837?check_suite_focus=true

@codeboten codeboten mentioned this issue Apr 28, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@codeboten codeboten

Labels
flaky test a test is flaky
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[testbed] disable setting grpc log codeboten/opentelemetry-collector-contrib
2 participants
@codeboten @dmitryax