Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Wire agent config #4485

Closed
wants to merge 2 commits into from
Closed

Wire agent config #4485

wants to merge 2 commits into from

Conversation

jalvz
Copy link
Contributor

@jalvz jalvz commented Dec 2, 2020

Motivation/summary

Read configuration from Elastic Agent and use the namespace for the indices

Checklist

I have considered changes for:

How to test these changes

Related issues

@jalvz jalvz marked this pull request as draft December 2, 2020 15:05
@apmmachine
Copy link
Contributor

apmmachine commented Dec 2, 2020 •
edited
Loading

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #4485 updated

  • Start Time: 2020-12-02T16:43:51.973+0000

  • Duration: 9 min 58 sec

Steps errors 1

Expand to view the steps failures

Run intake

  • Took 4 min 9 sec . View more details on here
  • Description: ./.ci/scripts/intake.sh

Log output

Expand to view the last 100 lines of log output 

[2020-12-02T16:50:49.015Z] >> Generating docs/fields.asciidoc for apm-server
[2020-12-02T16:50:49.015Z] exec: /tmp/tmp.ELEjkHikM3/python-env/build/ve/linux/bin/python3 /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-4485/pkg/mod/github.com/elastic/beats/[email protected]/libbeat/scripts/generate_fields_docs.py build/fields/fields.all.yml apm-server /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-4485/pkg/mod/github.com/elastic/beats/[email protected] --output_path /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-4485/src/github.com/elastic/apm-server
[2020-12-02T16:50:50.389Z] 2020/12/02 16:50:50 exec: go list -m
[2020-12-02T16:50:50.389Z] Running target: Check
[2020-12-02T16:50:50.389Z] >> check: Checking source code for common problems
[2020-12-02T16:50:50.389Z] Running dependency: github.com/elastic/beats/v7/dev-tools/mage.CheckYAMLNotExecutable
[2020-12-02T16:50:50.389Z] Running dependency: github.com/elastic/beats/v7/dev-tools/mage.GoVet
[2020-12-02T16:50:50.389Z] exec: go vet ./...
[2020-12-02T16:50:50.389Z] Running dependency: github.com/elastic/beats/v7/dev-tools/mage.CheckPythonTestNotExecutable
[2020-12-02T16:50:50.648Z] go: downloading github.com/elastic/badger/v2 v2.2007.2-beats
[2020-12-02T16:50:50.648Z] go: downloading github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e
[2020-12-02T16:50:50.648Z] go: downloading github.com/aws/aws-sdk-go-v2 v0.9.0
[2020-12-02T16:50:50.648Z] go: downloading github.com/xeipuuv/gojsonschema v0.0.0-20181112162635-ac52e6811b56
[2020-12-02T16:50:50.648Z] go: downloading github.com/santhosh-tekuri/jsonschema v1.2.4
[2020-12-02T16:50:50.648Z] go: downloading github.com/elastic/elastic-agent-client/v7 v7.0.0-20200709172729-d43b7ad5833a
[2020-12-02T16:50:50.648Z] go: downloading github.com/stretchr/testify v1.6.1
[2020-12-02T16:50:50.648Z] go: downloading github.com/ianlancetaylor/demangle v0.0.0-20200715173712-053cf528c12f
[2020-12-02T16:50:50.907Z] go: downloading github.com/cloudfoundry-community/go-cfclient v0.0.0-20190808214049-35bcce23fc5f
[2020-12-02T16:50:50.907Z] go: downloading github.com/elastic/go-hdrhistogram v0.1.0
[2020-12-02T16:50:50.907Z] go: downloading code.cloudfoundry.org/go-loggregator v7.4.0+incompatible
[2020-12-02T16:50:50.907Z] go: downloading github.com/pmezard/go-difflib v1.0.0
[2020-12-02T16:50:50.907Z] go: downloading github.com/cloudfoundry/sonde-go v0.0.0-20171206171820-b33733203bb4
[2020-12-02T16:50:50.907Z] go: downloading gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c
[2020-12-02T16:50:50.907Z] go: downloading github.com/tidwall/sjson v1.1.1
[2020-12-02T16:50:50.907Z] go: downloading github.com/mailru/easyjson v0.7.1
[2020-12-02T16:50:50.907Z] go: downloading github.com/ugorji/go v1.1.8
[2020-12-02T16:50:50.907Z] go: downloading github.com/Masterminds/semver v1.4.2
[2020-12-02T16:50:50.907Z] go: downloading github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415
[2020-12-02T16:50:50.907Z] go: downloading github.com/ugorji/go/codec v1.1.8
[2020-12-02T16:50:50.907Z] go: downloading github.com/tidwall/gjson v1.6.0
[2020-12-02T16:50:50.907Z] go: downloading code.cloudfoundry.org/gofileutils v0.0.0-20170111115228-4d0c80011a0f
[2020-12-02T16:50:50.907Z] go: downloading github.com/cloudfoundry/noaa v2.1.0+incompatible
[2020-12-02T16:50:50.907Z] go: downloading github.com/tidwall/match v1.0.1
[2020-12-02T16:50:50.907Z] go: downloading github.com/tidwall/pretty v1.0.1
[2020-12-02T16:50:50.907Z] go: downloading github.com/gorilla/websocket v1.4.1
[2020-12-02T16:50:50.907Z] go: downloading github.com/dgraph-io/ristretto v0.0.3-0.20200630154024-f66de99634de
[2020-12-02T16:50:51.166Z] go: downloading github.com/dgryski/go-farm v0.0.0-20190423205320-6a90982ecee2
[2020-12-02T16:50:51.166Z] go: downloading code.cloudfoundry.org/go-diodes v0.0.0-20190809170250-f77fb823c7ee
[2020-12-02T16:50:51.166Z] go: downloading github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb
[2020-12-02T16:50:51.166Z] go: downloading github.com/DataDog/zstd v1.4.1
[2020-12-02T16:50:51.166Z] go: downloading code.cloudfoundry.org/rfc5424 v0.0.0-20180905210152-236a6d29298a
[2020-12-02T16:50:53.698Z] go: downloading github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af
[2020-12-02T16:52:15.163Z] exec: git update-index -q --refresh
[2020-12-02T16:52:15.163Z] exec: git diff-index HEAD -- .
[2020-12-02T16:52:15.163Z] go build -o build/linux/golint golang.org/x/lint/golint
[2020-12-02T16:52:15.163Z] go: downloading golang.org/x/lint v0.0.0-20200302205851-738671d3881b
[2020-12-02T16:52:15.163Z] go build -o build/linux/reviewdog github.com/reviewdog/reviewdog/cmd/reviewdog
[2020-12-02T16:52:15.163Z] go: downloading github.com/reviewdog/reviewdog v0.9.17
[2020-12-02T16:52:15.163Z] go: downloading github.com/google/go-github/v29 v29.0.2
[2020-12-02T16:52:15.163Z] go: downloading github.com/reviewdog/errorformat v0.0.0-20200109134752-8983be9bc7dd
[2020-12-02T16:52:15.163Z] go: downloading github.com/mattn/go-shellwords v1.0.7
[2020-12-02T16:52:15.163Z] go: downloading github.com/xanzy/go-gitlab v0.22.3
[2020-12-02T16:52:15.163Z] go: downloading github.com/haya14busa/go-actions-toolkit v0.0.0-20200105081403-ca0307860f01
[2020-12-02T16:52:15.163Z] go: downloading cloud.google.com/go v0.51.0
[2020-12-02T16:52:15.163Z] go: downloading github.com/bradleyfalzon/ghinstallation v1.1.0
[2020-12-02T16:52:15.163Z] go: downloading github.com/dgrijalva/jwt-go v3.2.1-0.20190620180102-5e25c22bd5d6+incompatible
[2020-12-02T16:52:15.163Z] go: downloading github.com/google/go-github/v28 v28.1.1
[2020-12-02T16:52:15.163Z] go: downloading github.com/google/go-querystring v1.0.0
[2020-12-02T16:52:15.164Z] go: downloading cloud.google.com/go/datastore v1.0.0
[2020-12-02T16:52:15.164Z] go: downloading google.golang.org/api v0.15.0
[2020-12-02T16:52:15.164Z] go: downloading go.opencensus.io v0.22.2
[2020-12-02T16:52:15.164Z] go: downloading github.com/googleapis/gax-go/v2 v2.0.5
[2020-12-02T16:52:15.164Z] go: downloading github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7
[2020-12-02T16:52:47.287Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-4485/src/github.com/elastic/apm-server/beater/config/integration.go:31:2: struct field Id should be ID
[2020-12-02T16:52:47.287Z] Makefile:212: recipe for target 'golint' failed
[2020-12-02T16:52:47.287Z] make: *** [golint] Error 1
[2020-12-02T16:52:47.287Z] + cleanup
[2020-12-02T16:52:47.287Z] + rm -rf /tmp/tmp.ELEjkHikM3
[2020-12-02T16:52:48.399Z] Stage "Build and Test" skipped due to earlier failure(s)
[2020-12-02T16:52:48.468Z] Stage "linux build" skipped due to earlier failure(s)
[2020-12-02T16:52:48.469Z] Stage "windows build-test" skipped due to earlier failure(s)
[2020-12-02T16:52:48.470Z] Stage "OSX build-test" skipped due to earlier failure(s)
[2020-12-02T16:52:48.470Z] Stage "Unit Test" skipped due to earlier failure(s)
[2020-12-02T16:52:48.471Z] Stage "System and Environment Tests" skipped due to earlier failure(s)
[2020-12-02T16:52:48.472Z] Stage "Benchmarking" skipped due to earlier failure(s)
[2020-12-02T16:52:48.472Z] Stage "Check kibana Obj. Updated" skipped due to earlier failure(s)
[2020-12-02T16:52:48.473Z] Stage "Hey-Apm" skipped due to earlier failure(s)
[2020-12-02T16:52:48.473Z] Stage "Package" skipped due to earlier failure(s)
[2020-12-02T16:52:48.474Z] Stage "APM Integration Tests" skipped due to earlier failure(s)
[2020-12-02T16:52:48.503Z] Stage "Package" skipped due to earlier failure(s)
[2020-12-02T16:52:48.623Z] Failed in branch linux build
[2020-12-02T16:52:48.624Z] Failed in branch windows build-test
[2020-12-02T16:52:48.624Z] Failed in branch OSX build-test
[2020-12-02T16:52:48.625Z] Failed in branch Unit Test
[2020-12-02T16:52:48.625Z] Failed in branch System and Environment Tests
[2020-12-02T16:52:48.626Z] Failed in branch Benchmarking
[2020-12-02T16:52:48.626Z] Failed in branch Check kibana Obj. Updated
[2020-12-02T16:52:48.627Z] Failed in branch Hey-Apm
[2020-12-02T16:52:48.627Z] Failed in branch APM Integration Tests
[2020-12-02T16:52:48.671Z] Stage "Package" skipped due to earlier failure(s)
[2020-12-02T16:52:48.725Z] Failed in branch Package
[2020-12-02T16:52:48.941Z] Running on Jenkins in /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-4485
[2020-12-02T16:52:49.031Z] [INFO] getVaultSecret: Getting secrets
[2020-12-02T16:52:49.214Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-12-02T16:52:49.912Z] + chmod 755 generate-build-data.sh
[2020-12-02T16:52:49.912Z] + ./generate-build-data.sh https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-server/apm-server-mbp/PR-4485/ https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-server/apm-server-mbp/PR-4485/runs/3 FAILURE 537681
[2020-12-02T16:52:50.462Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-server/apm-server-mbp/PR-4485/runs/3/steps/?limit=10000 -o steps-info.json
[2020-12-02T16:52:51.013Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-server/apm-server-mbp/PR-4485/runs/3/tests/?status=FAILED -o tests-errors.json
[2020-12-02T16:52:51.013Z] Retry 1/3 exited 22, retrying in 1 seconds...
[2020-12-02T16:52:52.355Z] Retry 2/3 exited 22, retrying in 2 seconds...

@jalvz jalvz marked this pull request as ready for review December 2, 2020 16:39
@jalvz jalvz force-pushed the wire-agent-config branch from 6e3038b to 14fd032 Compare December 2, 2020 16:43
jalvz added 2 commits December 2, 2020 17:59
@jalvz
Iterate package (elastic#4473)
5f1f2dc 
Improves package generation.

- Splits metrics data stream into app metrics, internal and profiles. This is following the indexing strategy proposal.  - Adds the dataset property to the manifest files so the right templates are created.
- Sort and dedup fields (within each yml file). Fields are now flattened.
- Generates data stream pipelines out of our definitions.json file
@jalvz
wire config coming from agent
c8a9e0b
@jalvz jalvz force-pushed the wire-agent-config branch from 14fd032 to c8a9e0b Compare December 2, 2020 16:59
@jalvz jalvz closed this Dec 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jalvz @apmmachine