Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[7.x] Move setting of data_stream fields to processor (backport #5717) #5728

Closed
wants to merge 1 commit into from
Closed

[7.x] Move setting of data_stream fields to processor (backport #5717) #5728

wants to merge 1 commit into from

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Jul 15, 2021 •
edited by axw
Loading

This is an automatic backport of pull request #5717 done by Mergify.
Cherry-pick of 29cfed3 has failed:

On branch mergify/bp/7.x/pr-5717
Your branch is up to date with 'origin/7.x'.

You are currently cherry-picking commit 29cfed31.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   model/datastream.go
	modified:   model/error.go
	new file:   model/modelprocessor/datastream.go
	new file:   model/modelprocessor/datastream_test.go
	modified:   model/modelprocessor/environment_test.go
	modified:   model/span.go
	modified:   processor/otel/consumer_test.go
	modified:   processor/otel/test_approved/jaeger_sampling_rate.approved.json
	modified:   processor/otel/test_approved/metadata_jaeger-no-language.approved.json
	modified:   processor/otel/test_approved/metadata_jaeger-version.approved.json
	modified:   processor/otel/test_approved/metadata_jaeger.approved.json
	modified:   processor/otel/test_approved/span_jaeger_custom.approved.json
	modified:   processor/otel/test_approved/span_jaeger_db.approved.json
	modified:   processor/otel/test_approved/span_jaeger_http.approved.json
	modified:   processor/otel/test_approved/span_jaeger_http_status_code.approved.json
	modified:   processor/otel/test_approved/span_jaeger_https_default_port.approved.json
	modified:   processor/otel/test_approved/span_jaeger_messaging.approved.json
	modified:   processor/otel/test_approved/span_jaeger_subtype_component.approved.json
	modified:   processor/otel/test_approved/transaction_jaeger_custom.approved.json
	modified:   processor/otel/test_approved/transaction_jaeger_full.approved.json
	modified:   processor/otel/test_approved/transaction_jaeger_no_attrs.approved.json
	modified:   processor/otel/test_approved/transaction_jaeger_type_component.approved.json
	modified:   processor/otel/test_approved/transaction_jaeger_type_messaging.approved.json
	modified:   processor/otel/test_approved/transaction_jaeger_type_request.approved.json
	modified:   processor/otel/test_approved/transaction_jaeger_type_request_result.approved.json
	modified:   processor/stream/processor_test.go
	modified:   processor/stream/test_approved_es_documents/testIntakeIntegrationErrors.approved.json
	modified:   processor/stream/test_approved_es_documents/testIntakeIntegrationEvents.approved.json
	modified:   processor/stream/test_approved_es_documents/testIntakeIntegrationInvalidEvent.approved.json
	modified:   processor/stream/test_approved_es_documents/testIntakeIntegrationInvalidJSONEvent.approved.json
	modified:   processor/stream/test_approved_es_documents/testIntakeIntegrationMetadataNullValues.approved.json
	modified:   processor/stream/test_approved_es_documents/testIntakeIntegrationMetricsets.approved.json
	modified:   processor/stream/test_approved_es_documents/testIntakeIntegrationMinimalService.approved.json
	modified:   processor/stream/test_approved_es_documents/testIntakeIntegrationOptionalTimestamps.approved.json
	modified:   processor/stream/test_approved_es_documents/testIntakeIntegrationRumErrors.approved.json
	modified:   processor/stream/test_approved_es_documents/testIntakeIntegrationRumTransactions.approved.json
	modified:   processor/stream/test_approved_es_documents/testIntakeIntegrationSpans.approved.json
	modified:   processor/stream/test_approved_es_documents/testIntakeIntegrationTransactions.approved.json
	modified:   processor/stream/test_approved_es_documents/testIntakeRUMV3Errors.approved.json
	modified:   processor/stream/test_approved_es_documents/testIntakeRUMV3Events.approved.json
	modified:   publish/pub.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   beater/beater.go
	both modified:   model/apmevent.go
	both modified:   model/error_test.go
	both modified:   model/metricset.go
	both modified:   model/metricset_test.go
	both modified:   model/profile.go
	both modified:   model/profile_test.go
	both modified:   model/span_test.go
	both modified:   model/transaction.go
	both modified:   model/transaction_test.go
	both modified:   transform/transform.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/github/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

  • @Mergifyio refresh will re-evaluate the rules
  • @Mergifyio rebase will rebase this PR on its base branch
  • @Mergifyio update will merge the base branch into this PR
  • @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

  • look at your merge queues
  • generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.io/

@axw
Move setting of data_stream fields to processor (#5717)
0f67eb8 
(cherry picked from commit 29cfed3)

# Conflicts:
#	beater/beater.go
#	model/apmevent.go
#	model/error_test.go
#	model/metricset.go
#	model/metricset_test.go
#	model/profile.go
#	model/profile_test.go
#	model/span_test.go
#	model/transaction.go
#	model/transaction_test.go
#	transform/transform.go
@mergify mergify bot added the conflicts There is a conflict in the backported pull request label Jul 15, 2021
@mergify mergify bot assigned axw Jul 15, 2021
@apmmachine
Copy link
Contributor

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-07-15T06:02:49.418+0000

  • Duration: 5 min 10 sec

  • Commit: 0f67eb8

Trends 🧪

Image of Build Times

Steps errors 1

Expand to view the steps failures

Run intake
  • Took 2 min 37 sec . View more details on here
  • Description: ./.ci/scripts/intake.sh

Log output

Expand to view the last 100 lines of log output 

[2021-07-15T06:06:21.132Z] + export TEMP_PYTHON_ENV=/tmp/tmp.Z6h5FwOpfW
[2021-07-15T06:06:21.132Z] + TEMP_PYTHON_ENV=/tmp/tmp.Z6h5FwOpfW
[2021-07-15T06:06:21.132Z] + export PYTHON_ENV=/tmp/tmp.Z6h5FwOpfW/python-env
[2021-07-15T06:06:21.132Z] + PYTHON_ENV=/tmp/tmp.Z6h5FwOpfW/python-env
[2021-07-15T06:06:21.132Z] + export MAGEFILE_CACHE=/var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/.magefile
[2021-07-15T06:06:21.132Z] + MAGEFILE_CACHE=/var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/.magefile
[2021-07-15T06:06:21.132Z] + trap cleanup EXIT
[2021-07-15T06:06:21.132Z] + make check-full
[2021-07-15T06:06:25.364Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/.gvm/versions/go1.16.5.linux.amd64/bin/go build -o build/linux/bin/mage github.com/magefile/mage
[2021-07-15T06:06:25.364Z] go: downloading github.com/magefile/mage v1.11.0
[2021-07-15T06:06:26.305Z] ./build/linux/bin/mage -compile=build/linux/mage
[2021-07-15T06:07:22.632Z] 2021/07/15 06:07:17 exec: go list -m
[2021-07-15T06:07:22.632Z] Generated fields.yml for apm-server to /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/build/fields/fields.yml
[2021-07-15T06:07:22.632Z] Generated fields.yml for apm-server to /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/build/fields/fields.all.yml
[2021-07-15T06:07:49.238Z] go: downloading golang.org/x/mod v0.4.2
[2021-07-15T06:07:49.238Z] go: downloading golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/transform/transform.go:35:1: expected '}', found '<<'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/transform/transform.go:35:3: expected ';', found '<<'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/transform/transform.go:47:1: expected '}', found '=='
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/transform/transform.go:47:3: expected ';', found '=='
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/transform/transform.go:48:68: illegal character U+0023 '#'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/apmevent.go:31:1: expected '}', found '<<'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/apmevent.go:31:3: expected ';', found '<<'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/apmevent.go:40:2: expected declaration, found 'return'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/apmevent.go:57:2: expected declaration, found 'switch'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/apmevent.go:73:68: illegal character U+0023 '#'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/metricset.go:180:1: expected declaration, found '<<'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/metricset.go:184:68: illegal character U+0023 '#'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/metricset.go:201:3: expected declaration, found 'for'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/metricset.go:231:2: expected declaration, found 'for'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/metricset.go:233:3: expected declaration, found m
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/metricset.go:255:68: illegal character U+0023 '#'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/metricset.go:263:2: expected declaration, found fields
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/metricset.go:269:2: expected declaration, found fields
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/metricset.go:280:2: expected declaration, found fields
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/profile.go:67:1: expected statement, found '<<'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/profile.go:74:1: expected statement, found '>>'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/profile.go:74:68: illegal character U+0023 '#'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/profile.go:156:1: expected statement, found '<<'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/profile.go:161:1: expected statement, found '=='
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/profile.go:167:68: illegal character U+0023 '#'
[2021-07-15T06:07:51.796Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/profile.go:173:3: expected ';', found 'EOF'
[2021-07-15T06:07:51.797Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/profile.go:173:3: expected ';', found 'EOF'
[2021-07-15T06:07:51.797Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/profile.go:173:3: expected '}', found 'EOF'
[2021-07-15T06:07:51.797Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/profile.go:173:3: expected '}', found 'EOF'
[2021-07-15T06:07:51.797Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/transaction.go:122:1: expected declaration, found '<<'
[2021-07-15T06:07:51.797Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/transaction.go:126:68: illegal character U+0023 '#'
[2021-07-15T06:07:51.797Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/transaction.go:142:2: expected declaration, found parent
[2021-07-15T06:07:51.797Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/metricset.go:268:6: fields redeclared in this block
[2021-07-15T06:07:51.797Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/metricset.go:262:6: 	other declaration of fields
[2021-07-15T06:07:51.797Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/metricset.go:279:6: fields redeclared in this block
[2021-07-15T06:07:51.797Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/metricset.go:262:6: 	other declaration of fields
[2021-07-15T06:07:51.797Z] panic: packages load error
[2021-07-15T06:07:51.797Z] 
[2021-07-15T06:07:51.797Z] goroutine 1 [running]:
[2021-07-15T06:07:51.797Z] main.generateV2()
[2021-07-15T06:07:51.797Z] 	/var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/modeldecoder/generator/cmd/main.go:53 +0x2f1
[2021-07-15T06:07:51.797Z] main.main()
[2021-07-15T06:07:51.797Z] 	/var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728/src/github.com/elastic/apm-server/model/modeldecoder/generator/cmd/main.go:43 +0x25
[2021-07-15T06:07:51.797Z] exit status 2
[2021-07-15T06:07:51.797Z] main.go:20: running "go": exit status 1
[2021-07-15T06:07:51.797Z] go: downloading gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b
[2021-07-15T06:07:52.736Z] Makefile:137: recipe for target 'go-generate' failed
[2021-07-15T06:07:52.736Z] make: *** [go-generate] Error 1
[2021-07-15T06:07:52.736Z] + cleanup
[2021-07-15T06:07:52.736Z] + rm -rf /tmp/tmp.Z6h5FwOpfW
[2021-07-15T06:07:54.244Z] Stage "Build and Test" skipped due to earlier failure(s)
[2021-07-15T06:07:54.832Z] Stage "linux build" skipped due to earlier failure(s)
[2021-07-15T06:07:54.849Z] Stage "windows build-test" skipped due to earlier failure(s)
[2021-07-15T06:07:54.865Z] Stage "OSX build-test" skipped due to earlier failure(s)
[2021-07-15T06:07:54.880Z] Stage "Unit Test" skipped due to earlier failure(s)
[2021-07-15T06:07:54.894Z] Stage "System and Environment Tests" skipped due to earlier failure(s)
[2021-07-15T06:07:54.909Z] Stage "Benchmarking" skipped due to earlier failure(s)
[2021-07-15T06:07:54.931Z] Stage "Check kibana Obj. Updated" skipped due to earlier failure(s)
[2021-07-15T06:07:54.946Z] Stage "Hey-Apm" skipped due to earlier failure(s)
[2021-07-15T06:07:54.965Z] Stage "Package" skipped due to earlier failure(s)
[2021-07-15T06:07:54.974Z] Stage "APM Integration Tests" skipped due to earlier failure(s)
[2021-07-15T06:07:55.078Z] Stage "Package" skipped due to earlier failure(s)
[2021-07-15T06:07:55.462Z] Failed in branch linux build
[2021-07-15T06:07:55.471Z] Failed in branch windows build-test
[2021-07-15T06:07:55.481Z] Failed in branch OSX build-test
[2021-07-15T06:07:55.490Z] Failed in branch Unit Test
[2021-07-15T06:07:55.501Z] Failed in branch System and Environment Tests
[2021-07-15T06:07:55.511Z] Failed in branch Benchmarking
[2021-07-15T06:07:55.521Z] Failed in branch Check kibana Obj. Updated
[2021-07-15T06:07:55.532Z] Failed in branch Hey-Apm
[2021-07-15T06:07:55.541Z] Failed in branch APM Integration Tests
[2021-07-15T06:07:55.658Z] Stage "Package" skipped due to earlier failure(s)
[2021-07-15T06:07:55.906Z] Failed in branch Package
[2021-07-15T06:07:57.881Z] Running on worker-1095690 in /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5728
[2021-07-15T06:07:58.051Z] [INFO] getVaultSecret: Getting secrets
[2021-07-15T06:07:58.119Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-07-15T06:08:00.790Z] + chmod 755 generate-build-data.sh
[2021-07-15T06:08:00.790Z] + ./generate-build-data.sh https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-server/apm-server-mbp/PR-5728/ https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-server/apm-server-mbp/PR-5728/runs/1 FAILURE 309933
[2021-07-15T06:08:00.790Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-server/apm-server-mbp/PR-5728/runs/1/steps/?limit=10000 -o steps-info.json
[2021-07-15T06:08:01.496Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-server/apm-server-mbp/PR-5728/runs/1/tests/?status=FAILED -o tests-errors.json
[2021-07-15T06:08:02.206Z] Retry 1/3 exited 22, retrying in 1 seconds...
[2021-07-15T06:08:03.677Z] Retry 2/3 exited 22, retrying in 2 seconds...
[2021-07-15T06:08:05.934Z] Retry 3/3 exited 22, no more retries left.
[2021-07-15T06:08:05.935Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-server/apm-server-mbp/PR-5728/runs/1/log/ -o pipeline-log.txt

@mergify
Copy link
Contributor Author

mergify bot commented Jul 15, 2021

This pull request is now in conflicts. Could you fix it @mergify[bot]? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b mergify/bp/7.x/pr-5717 upstream/mergify/bp/7.x/pr-5717
git merge upstream/7.x
git push upstream mergify/bp/7.x/pr-5717

@axw axw closed this Jul 15, 2021
@axw axw deleted the mergify/bp/7.x/pr-5717 branch July 15, 2021 07:26
@mergify mergify bot mentioned this pull request Jul 15, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@axw axw

Labels
conflicts There is a conflict in the backported pull request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@apmmachine @axw