[exporter/awsxray] Adding Log Group Names to X-Ray Exporter Config #16859

atshaw43 · 2022-12-12T17:49:08Z

Description: Adding log group names to X-Ray exporter YAML config.

Link to tracking Issue: #16939

Testing: Unit Tests

Documentation:
Updated README.md
Will also update https://aws-otel.github.io/docs/getting-started/x-ray

runforesight · 2022-12-12T17:50:20Z

Foresight Summary

Major Impacts

TestConsumeLogsUnexpectedExporterType ❌ failed 24 times in 24 runs (100% fail rate).

TestConsumeTracesUnexpectedExporterType ❌ failed 24 times in 24 runs (100% fail rate).

TestLoadInvalidConfig_NoScrapers ❌ failed 24 times in 24 runs (100% fail rate).

build-and-test-windows duration(4 seconds) has decreased 39 minutes 55 seconds compared to main branch avg(39 minutes 59 seconds).

View More Details

⭕ build-and-test-windows workflow has finished in 4 seconds (39 minutes 53 seconds less than `main` branch avg.) and finished at 13th Dec, 2022.

Job	Failed Steps	Tests
windows-unittest-matrix	- 🔗	N/A	See Details
windows-unittest	- 🔗	N/A	See Details

✅ changelog workflow has finished in 1 minute 19 seconds (6 minutes 7 seconds less than `main` branch avg.) and finished at 13th Dec, 2022.

Job	Failed Steps	Tests
changelog	- 🔗	N/A	See Details

✅ check-links workflow has finished in 1 minute 24 seconds (1 minute 59 seconds less than `main` branch avg.) and finished at 13th Dec, 2022.

Job	Failed Steps	Tests
changed files	- 🔗	N/A	See Details
check-links	- 🔗	N/A	See Details

✅ tracegen workflow has finished in 1 minute 30 seconds (2 minutes 12 seconds less than `main` branch avg.) and finished at 13th Dec, 2022.

Job	Failed Steps	Tests
build-dev	- 🔗	N/A	See Details
publish-latest	- 🔗	N/A	See Details
publish-stable	- 🔗	N/A	See Details

✅ prometheus-compliance-tests workflow has finished in 3 minutes 17 seconds (6 minutes 13 seconds less than `main` branch avg.) and finished at 13th Dec, 2022.

Job	Failed Steps	Tests
prometheus-compliance-tests	- 🔗	✅ 21 ❌ 0 ⏭ 0 🔗	See Details

✅ build-and-test workflow has finished in 36 minutes 32 seconds (24 minutes 50 seconds less than `main` branch avg.) and finished at 13th Dec, 2022. There are 6 test failures.

Job	Failed Steps	Tests
unittest-matrix (1.19, internal)	- 🔗	✅ 592 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, internal)	- 🔗	✅ 592 ❌ 0 ⏭ 0 🔗	See Details
correctness-metrics	- 🔗	✅ 2 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, processor)	- 🔗	✅ 1465 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, processor)	- 🔗	✅ 1465 ❌ 0 ⏭ 0 🔗	See Details
correctness-traces	- 🔗	✅ 17 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, extension)	- 🔗	✅ 528 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, extension)	- 🔗	✅ 528 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, receiver-0)	- 🔗	✅ 2532 ❌ 1 ⏭ 0 🔗	See Details
unittest-matrix (1.19, receiver-0)	- 🔗	✅ 2532 ❌ 1 ⏭ 0 🔗	See Details
unittest-matrix (1.19, other)	- 🔗	✅ 4363 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, exporter)	- 🔗	✅ 2240 ❌ 2 ⏭ 0 🔗	See Details
unittest-matrix (1.18, other)	- 🔗	✅ 4363 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.18, exporter)	- 🔗	✅ 2240 ❌ 2 ⏭ 0 🔗	See Details
unittest-matrix (1.18, receiver-1)	- 🔗	✅ 1854 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, receiver-1)	- 🔗	✅ 1854 ❌ 0 ⏭ 0 🔗	See Details
integration-tests	- 🔗	✅ 53 ❌ 0 ⏭ 0 🔗	See Details
setup-environment	- 🔗	N/A	See Details
checks	- 🔗	N/A	See Details
check-collector-module-version	- 🔗	N/A	See Details
check-codeowners	- 🔗	N/A	See Details
build-examples	- 🔗	N/A	See Details
lint-matrix (receiver-0)	- 🔗	N/A	See Details
lint-matrix (receiver-1)	- 🔗	N/A	See Details
lint-matrix (processor)	- 🔗	N/A	See Details
lint-matrix (exporter)	- 🔗	N/A	See Details
lint-matrix (extension)	- 🔗	N/A	See Details
lint-matrix (internal)	- 🔗	N/A	See Details
lint-matrix (other)	- 🔗	N/A	See Details
lint	- 🔗	N/A	See Details
unittest (1.19)	- 🔗	N/A	See Details
unittest (1.18)	- 🔗	N/A	See Details
cross-compile (darwin, amd64)	- 🔗	N/A	See Details
cross-compile (darwin, arm64)	- 🔗	N/A	See Details
cross-compile (linux, 386)	- 🔗	N/A	See Details
cross-compile (linux, amd64)	- 🔗	N/A	See Details
cross-compile (linux, arm)	- 🔗	N/A	See Details
cross-compile (linux, arm64)	- 🔗	N/A	See Details
cross-compile (linux, ppc64le)	- 🔗	N/A	See Details
cross-compile (windows, 386)	- 🔗	N/A	See Details
cross-compile (windows, amd64)	- 🔗	N/A	See Details
build-package (deb)	- 🔗	N/A	See Details
build-package (rpm)	- 🔗	N/A	See Details
windows-msi	- 🔗	N/A	See Details
publish-check	- 🔗	N/A	See Details
publish-dev	- 🔗	N/A	See Details
publish-stable	- 🔗	N/A	See Details

✅ load-tests workflow has finished in 8 minutes 28 seconds (8 minutes 1 second less than `main` branch avg.) and finished at 13th Dec, 2022.

Job	Failed Steps	Tests
loadtest (TestTraceAttributesProcessor)	- 🔗	✅ 3 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestIdleMode)	- 🔗	✅ 1 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestMetric10kDPS\|TestMetricsFromFile)	- 🔗	✅ 6 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestTraceNoBackend10kSPS\|TestTrace1kSPSWithAttrs)	- 🔗	✅ 8 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestMetricResourceProcessor\|TestTrace10kSPS)	- 🔗	✅ 12 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestTraceBallast1kSPSWithAttrs\|TestTraceBallast1kSPSAddAttrs)	- 🔗	✅ 10 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestBallastMemory\|TestLog10kDPS)	- 🔗	✅ 19 ❌ 0 ⏭ 0 🔗	See Details
setup-environment	- 🔗	N/A	See Details

🔎 See details on Foresight

^{*You can configure Foresight comments in your organization settings page.}

This reverts commit 3b5c38a.

wangzlei · 2022-12-19T19:30:25Z

Can we have a xray segment example either in PR description for helping reviewer to understand

atshaw43 · 2022-12-19T22:54:55Z

The new field is aws_log_groups in the exporter YAML config for AWS. It is a set of cloud watch log group names.
Example config.:

awsxray:
awsxray/customname:
region: eu-west-1
resource_arn: "arn:aws:ec2:us-east1:123456789:instance/i-293hiuhe0u"
role_arn: "arn:aws:iam::123456789:role/monitoring-EKS-NodeInstanceRole"
indexed_attributes: [ "indexed_attr_0", "indexed_attr_1" ]
aws_log_groups: ["myLogGroup"]

This makes its way to cwl variable (Cloud Watch Logs) in aws.go. We use it as the lowest priority compared to the resource attributes.

mx-psi · 2022-12-20T10:30:10Z

@willarmiros could you review as a codeowner?

willarmiros

@wangzlei is in my organization, so I will approve based on his approval.

willarmiros · 2022-12-16T21:40:15Z

exporter/awsxrayexporter/internal/translator/segment.go

@@ -67,8 +67,8 @@ var (
 )

 // MakeSegmentDocumentString converts an OpenTelemetry Span to an X-Ray Segment and then serialzies to JSON
-func MakeSegmentDocumentString(span ptrace.Span, resource pcommon.Resource, indexedAttrs []string, indexAllAttrs bool) (string, error) {
-	segment, err := MakeSegment(span, resource, indexedAttrs, indexAllAttrs)
+func MakeSegmentDocumentString(span ptrace.Span, resource pcommon.Resource, indexedAttrs []string, indexAllAttrs bool, logGroupNames []string) (string, error) {


Mostly a question for @mx-psi or the OTel maintainer who's reviewing: is there a risk of a breaking change with changing the signature of this as a public method? I don't think so since it's part of the collector rather than an SDK, but want to confirm

@mx-psi
Bumping this question.

Hey, sorry, missed this.

is there a risk of a breaking change with changing the signature of this as a public method? I don't think so since it's part of the collector rather than an SDK, but want to confirm

This is part of an internal package, so it is not part of the public API and thus there is no risk.

In general, we do care about the Go API of the Collector: we aim for Collector components to be eventually stable in both their usage as part of a Collector binary and as a library, but in this case this is fine.

atshaw43 added 3 commits December 12, 2022 00:09

Adding Log Groups from Config

2172ccf

Fixing Name

c5e6904

Adding to README.md

aee07eb

atshaw43 requested review from a team and mx-psi December 12, 2022 17:49

github-actions bot added the exporter/awsxray label Dec 12, 2022

github-actions bot requested a review from willarmiros December 12, 2022 17:50

github-actions bot assigned mx-psi Dec 12, 2022

atshaw43 changed the title ~~Adding Log Group Names to X-Ray Exporter Config~~ [exporter/config]Adding Log Group Names to X-Ray Exporter Config Dec 12, 2022

atshaw43 changed the title ~~[exporter/config]Adding Log Group Names to X-Ray Exporter Config~~ [exporter/config] Adding Log Group Names to X-Ray Exporter Config Dec 12, 2022

Adding change log

f1ede02

atshaw43 changed the title ~~[exporter/config] Adding Log Group Names to X-Ray Exporter Config~~ [exporter/awsxray] Adding Log Group Names to X-Ray Exporter Config Dec 12, 2022

atshaw43 and others added 14 commits December 12, 2022 12:20

Golang style check fix

58a5415

Fixing linter

0e50d79

Running go mod tidy

c41d8c0

Updating go.mod

3b5c38a

Revert "Updating go.mod"

e2a1a07

This reverts commit 3b5c38a.

Reverting signalreceiver

bd6964f

Reverting tools update

09939bf

Fix imports

edfd221

Style check

feaf06b

Removing whitespace

de26eb6

Merge branch 'open-telemetry:main' into main

c41ac44

Ran goimports

e0287ee

Ran goimports again

545c6ea

Testing

9eab02a

wangzlei approved these changes Dec 19, 2022

View reviewed changes

willarmiros approved these changes Dec 20, 2022

View reviewed changes

mx-psi approved these changes Dec 28, 2022

View reviewed changes

mx-psi merged commit 3b25b98 into open-telemetry:main Dec 28, 2022

rapphil mentioned this pull request Feb 2, 2023

XRay traces not connected to log entries in Cloudwatch aws-observability/aws-otel-java-instrumentation#205

Closed

willarmiros mentioned this pull request Mar 31, 2023

Setting cloudwatch log group so that logs appear in Cloudwatch Traces aws-observability/aws-otel-collector#1766

Closed

plantfansam mentioned this pull request Jul 21, 2023

disregard #24427

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[exporter/awsxray] Adding Log Group Names to X-Ray Exporter Config #16859

[exporter/awsxray] Adding Log Group Names to X-Ray Exporter Config #16859

atshaw43 commented Dec 12, 2022 •

edited

Loading

runforesight bot commented Dec 12, 2022 •

edited

Loading

`TestConsumeLogsUnexpectedExporterType` ❌ failed 24 times in 24 runs (100% fail rate).

`TestConsumeTracesUnexpectedExporterType` ❌ failed 24 times in 24 runs (100% fail rate).

`TestLoadInvalidConfig_NoScrapers` ❌ failed 24 times in 24 runs (100% fail rate).

`build-and-test-windows` duration(4 seconds) has decreased 39 minutes 55 seconds compared to main branch avg(39 minutes 59 seconds).

⭕ build-and-test-windows workflow has finished in 4 seconds (39 minutes 53 seconds less than `main` branch avg.) and finished at 13th Dec, 2022.

✅ changelog workflow has finished in 1 minute 19 seconds (6 minutes 7 seconds less than `main` branch avg.) and finished at 13th Dec, 2022.

✅ check-links workflow has finished in 1 minute 24 seconds (1 minute 59 seconds less than `main` branch avg.) and finished at 13th Dec, 2022.

✅ tracegen workflow has finished in 1 minute 30 seconds (2 minutes 12 seconds less than `main` branch avg.) and finished at 13th Dec, 2022.

✅ prometheus-compliance-tests workflow has finished in 3 minutes 17 seconds (6 minutes 13 seconds less than `main` branch avg.) and finished at 13th Dec, 2022.

✅ build-and-test workflow has finished in 36 minutes 32 seconds (24 minutes 50 seconds less than `main` branch avg.) and finished at 13th Dec, 2022. There are 6 test failures.

✅ load-tests workflow has finished in 8 minutes 28 seconds (8 minutes 1 second less than `main` branch avg.) and finished at 13th Dec, 2022.

wangzlei commented Dec 19, 2022 •

edited

Loading

atshaw43 commented Dec 19, 2022

mx-psi commented Dec 20, 2022

willarmiros left a comment

willarmiros Dec 16, 2022

atshaw43 Dec 27, 2022

mx-psi Dec 28, 2022

[exporter/awsxray] Adding Log Group Names to X-Ray Exporter Config #16859

[exporter/awsxray] Adding Log Group Names to X-Ray Exporter Config #16859

Conversation

atshaw43 commented Dec 12, 2022 • edited Loading

runforesight bot commented Dec 12, 2022 • edited Loading

Foresight Summary

TestConsumeLogsUnexpectedExporterType ❌ failed 24 times in 24 runs (100% fail rate).

TestConsumeTracesUnexpectedExporterType ❌ failed 24 times in 24 runs (100% fail rate).

TestLoadInvalidConfig_NoScrapers ❌ failed 24 times in 24 runs (100% fail rate).

build-and-test-windows duration(4 seconds) has decreased 39 minutes 55 seconds compared to main branch avg(39 minutes 59 seconds).

⭕ build-and-test-windows workflow has finished in 4 seconds (39 minutes 53 seconds less than main branch avg.) and finished at 13th Dec, 2022.

✅ changelog workflow has finished in 1 minute 19 seconds (6 minutes 7 seconds less than main branch avg.) and finished at 13th Dec, 2022.

✅ check-links workflow has finished in 1 minute 24 seconds (1 minute 59 seconds less than main branch avg.) and finished at 13th Dec, 2022.

✅ tracegen workflow has finished in 1 minute 30 seconds (2 minutes 12 seconds less than main branch avg.) and finished at 13th Dec, 2022.

✅ prometheus-compliance-tests workflow has finished in 3 minutes 17 seconds (6 minutes 13 seconds less than main branch avg.) and finished at 13th Dec, 2022.

✅ build-and-test workflow has finished in 36 minutes 32 seconds (24 minutes 50 seconds less than main branch avg.) and finished at 13th Dec, 2022. There are 6 test failures.

✅ load-tests workflow has finished in 8 minutes 28 seconds (8 minutes 1 second less than main branch avg.) and finished at 13th Dec, 2022.

wangzlei commented Dec 19, 2022 • edited Loading

atshaw43 commented Dec 19, 2022

mx-psi commented Dec 20, 2022

willarmiros left a comment

Choose a reason for hiding this comment

willarmiros Dec 16, 2022

Choose a reason for hiding this comment

atshaw43 Dec 27, 2022

Choose a reason for hiding this comment

mx-psi Dec 28, 2022

Choose a reason for hiding this comment

atshaw43 commented Dec 12, 2022 •

edited

Loading

runforesight bot commented Dec 12, 2022 •

edited

Loading

`TestConsumeLogsUnexpectedExporterType` ❌ failed 24 times in 24 runs (100% fail rate).

`TestConsumeTracesUnexpectedExporterType` ❌ failed 24 times in 24 runs (100% fail rate).

`TestLoadInvalidConfig_NoScrapers` ❌ failed 24 times in 24 runs (100% fail rate).

`build-and-test-windows` duration(4 seconds) has decreased 39 minutes 55 seconds compared to main branch avg(39 minutes 59 seconds).

⭕ build-and-test-windows workflow has finished in 4 seconds (39 minutes 53 seconds less than `main` branch avg.) and finished at 13th Dec, 2022.

✅ changelog workflow has finished in 1 minute 19 seconds (6 minutes 7 seconds less than `main` branch avg.) and finished at 13th Dec, 2022.

✅ check-links workflow has finished in 1 minute 24 seconds (1 minute 59 seconds less than `main` branch avg.) and finished at 13th Dec, 2022.

✅ tracegen workflow has finished in 1 minute 30 seconds (2 minutes 12 seconds less than `main` branch avg.) and finished at 13th Dec, 2022.

✅ prometheus-compliance-tests workflow has finished in 3 minutes 17 seconds (6 minutes 13 seconds less than `main` branch avg.) and finished at 13th Dec, 2022.

✅ build-and-test workflow has finished in 36 minutes 32 seconds (24 minutes 50 seconds less than `main` branch avg.) and finished at 13th Dec, 2022. There are 6 test failures.

✅ load-tests workflow has finished in 8 minutes 28 seconds (8 minutes 1 second less than `main` branch avg.) and finished at 13th Dec, 2022.

wangzlei commented Dec 19, 2022 •

edited

Loading