Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pre-build docker images used for filebeat integration testing #18969

Merged
merged 3 commits into from
Jun 9, 2020
Merged

Pre-build docker images used for filebeat integration testing #18969

merged 3 commits into from
Jun 9, 2020

Conversation

jsoriano
Copy link
Member

@jsoriano jsoriano commented Jun 4, 2020 •
edited
Loading

What does this PR do?

Pre-build docker images used for filebeat integration testing.

Why is it important?

To reduce flakiness of tests because of having to build images.

Related issues

@jsoriano jsoriano added :Testing Team:Automation Label for the Observability productivity team labels Jun 4, 2020
@jsoriano jsoriano requested a review from kuisathaverat June 4, 2020 11:15
@jsoriano jsoriano self-assigned this Jun 4, 2020
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jun 4, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented Jun 4, 2020 •
edited
Loading

💔 Tests Failed

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #18969 updated]

  • Start Time: 2020-06-04T16:14:41.288+0000

  • Duration: 79 min 37 sec

Test stats 🧪

Test Results
Failed 1
Passed 9316
Skipped 1574
Total 10891

Test errors

Expand to view the tests failures

  • Name: Build and Test / Filebeat Windows / test_tail_files – test_crawler.Test

    • Age: 1
    • Duration: 16.239
    • Error Details: Timeout waiting for 'cond' to be true. Waited 15 seconds.

Steps errors

Expand to view the steps failures

  • Name: General Build Step

    • Description: [2020-06-04T16:36:49.132Z] [Google Cloud Storage Plugin] Found 1 files to download from pattern: gs:

    • Duration: 0 min 5 sec

    • Start Time: 2020-06-04T16:36:49.086+0000

    • log

  • Name: Mage build unitTest

    • Description: mage build unitTest

    • Duration: 11 min 44 sec

    • Start Time: 2020-06-04T16:41:49.318+0000

    • log

  • Name: Install Go 1.13.10

    • Description: .ci/scripts/install-go.sh

    • Duration: 2 min 37 sec

    • Start Time: 2020-06-04T16:37:04.318+0000

    • log

  • Name: Install Go 1.13.10

    • Description: .ci/scripts/install-go.sh

    • Duration: 2 min 22 sec

    • Start Time: 2020-06-04T16:37:05.147+0000

    • log

  • Name: Install Go 1.13.10

    • Description: .ci/scripts/install-go.sh

    • Duration: 2 min 22 sec

    • Start Time: 2020-06-04T16:37:09.145+0000

    • log

  • Name: Report to Codecov

    • Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done

    • Duration: 2 min 22 sec

    • Start Time: 2020-06-04T16:56:04.722+0000

    • log

Log output

Expand to view the last 100 lines of log output 

[2020-06-04T17:32:50.031Z] + [ -f packetbeat/build/coverage/full.cov ]
[2020-06-04T17:32:50.031Z] + FILE=winlogbeat/build/coverage/full.cov
[2020-06-04T17:32:50.031Z] + [ -f winlogbeat/build/coverage/full.cov ]
[2020-06-04T17:32:50.031Z] + FILE=journalbeat/build/coverage/full.cov
[2020-06-04T17:32:50.031Z] + [ -f journalbeat/build/coverage/full.cov ]
[2020-06-04T17:32:50.756Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats
[2020-06-04T17:32:51.069Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-06-04T17:32:51.082Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Lint
[2020-06-04T17:32:51.167Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Winlogbeat-oss
[2020-06-04T17:32:51.241Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Elastic-Agent-x-pack
[2020-06-04T17:32:51.327Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Dockerlogbeat
[2020-06-04T17:32:51.416Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Auditbeat-crosscompile
[2020-06-04T17:32:51.498Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Elastic-Agent-Mac-OS-X
[2020-06-04T17:32:51.582Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Journalbeat-oss
[2020-06-04T17:32:51.675Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Generators-Metricbeat-Linux
[2020-06-04T17:32:51.761Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Functionbeat-x-pack
[2020-06-04T17:32:51.846Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Filebeat-x-pack-Mac-OS-X
[2020-06-04T17:32:51.932Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Auditbeat-oss-Mac-OS-X
[2020-06-04T17:32:52.017Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Auditbeat-x-pack-Mac-OS-X
[2020-06-04T17:32:52.105Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Metricbeat-OSS-Unit-tests
[2020-06-04T17:32:52.189Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Filebeat-Mac-OS-X
[2020-06-04T17:32:52.278Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Auditbeat-oss-Windows
[2020-06-04T17:32:52.369Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Heartbeat-oss
[2020-06-04T17:32:52.461Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Auditbeat-x-pack
[2020-06-04T17:32:52.553Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Metricbeat-crosscompile
[2020-06-04T17:32:52.641Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Auditbeat-x-pack-Windows
[2020-06-04T17:32:52.718Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Winlogbeat-Windows-x-pack
[2020-06-04T17:32:52.798Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Filebeat-Windows
[2020-06-04T17:32:52.875Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Libbeat-x-pack
[2020-06-04T17:32:52.962Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Winlogbeat-Windows
[2020-06-04T17:32:53.042Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Filebeat-x-pack-Windows
[2020-06-04T17:32:53.128Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Auditbeat-oss-Linux
[2020-06-04T17:32:53.218Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Packetbeat-oss
[2020-06-04T17:32:53.319Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Metricbeat-Mac-OS-X
[2020-06-04T17:32:53.420Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Metricbeat-Windows
[2020-06-04T17:32:53.511Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Functionbeat-Mac-OS-X-x-pack
[2020-06-04T17:32:53.604Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Heartbeat-Mac-OS-X
[2020-06-04T17:32:53.694Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Metricbeat-x-pack-Windows
[2020-06-04T17:32:53.783Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Generators-Beat-Linux
[2020-06-04T17:32:53.863Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Filebeat-x-pack
[2020-06-04T17:32:53.948Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Metricbeat-x-pack-Mac-OS-X
[2020-06-04T17:32:54.027Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Filebeat-oss
[2020-06-04T17:32:54.104Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Heartbeat-Windows
[2020-06-04T17:32:54.180Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests
[2020-06-04T17:32:54.266Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Metricbeat-Python-integration-tests
[2020-06-04T17:32:54.340Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Libbeat-oss
[2020-06-04T17:32:54.417Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Functionbeat-Windows
[2020-06-04T17:32:54.501Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Generators-Metricbeat-Mac-OS-X
[2020-06-04T17:32:54.574Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Generators-Beat-Mac-OS-X
[2020-06-04T17:32:54.645Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Libbeat-crosscompile
[2020-06-04T17:32:54.719Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Libbeat-stress-tests
[2020-06-04T17:32:54.793Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Metricbeat-x-pack
[2020-06-04T17:32:55.157Z] + cat
[2020-06-04T17:32:55.157Z] + /usr/local/bin/runbld ./runbld-script
[2020-06-04T17:32:55.157Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-06-04T17:33:01.816Z] runbld>>> runbld started
[2020-06-04T17:33:01.816Z] runbld>>> 1.6.11/a66728ff8f4356963772e6e6d2069392fa06acbe
[2020-06-04T17:33:03.201Z] runbld>>> The following profiles matched the job 'Beats/beats-beats-mbp/PR-18969' in order of occurrence in the config (last value wins).
[2020-06-04T17:33:04.591Z] runbld>>> Debug logging enabled.
[2020-06-04T17:33:04.591Z] runbld>>> Storing result
[2020-06-04T17:33:04.852Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-06-04T17:33:04.852Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200604173304-C567770E
[2020-06-04T17:33:04.852Z] runbld>>> Adding system facts.
[2020-06-04T17:33:05.797Z] runbld>>> Adding vcs info for the latest commit:  d9885fb326ddcf44569a1d17dea4482b63b3a998
[2020-06-04T17:33:05.797Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-06-04T17:33:05.797Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-06-04T17:33:05.797Z] Processing JUnit reports with runbld...
[2020-06-04T17:33:05.797Z] + echo 'Processing JUnit reports with runbld...'
[2020-06-04T17:33:06.373Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-06-04T17:33:06.373Z] runbld>>> DURATION: 14ms
[2020-06-04T17:33:06.373Z] runbld>>> STDOUT: 40 bytes
[2020-06-04T17:33:06.373Z] runbld>>> STDERR: 49 bytes
[2020-06-04T17:33:06.373Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-06-04T17:33:06.373Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats
[2020-06-04T17:33:07.756Z] runbld>>> Storing build metadata: 
[2020-06-04T17:33:07.756Z] runbld>>> Adding test report.
[2020-06-04T17:33:07.757Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats
[2020-06-04T17:33:08.704Z] runbld>>> Found 115 test output files
[2020-06-04T17:33:08.966Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-graphite.xml
[2020-06-04T17:33:08.966Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-windows.xml
[2020-06-04T17:33:09.228Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-openmetrics.xml
[2020-06-04T17:33:09.228Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-cloudfoundry.xml
[2020-06-04T17:33:09.228Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-iis.xml
[2020-06-04T17:33:09.228Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-istio.xml
[2020-06-04T17:33:09.228Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-activemq.xml
[2020-06-04T17:33:09.228Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-tomcat.xml
[2020-06-04T17:33:11.144Z] runbld>>> Test output logs contained: Errors: 1 Failures: 0 Tests: 10741 Skipped: 1335
[2020-06-04T17:33:11.144Z] runbld>>> Storing result
[2020-06-04T17:33:11.144Z] runbld>>> FAILURES: 1
[2020-06-04T17:33:11.716Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-06-04T17:33:11.716Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200604173304-C567770E
[2020-06-04T17:33:11.716Z] runbld>>> Email notification disabled by environment variable.
[2020-06-04T17:33:11.716Z] runbld>>> Slack notification disabled by environment variable.
[2020-06-04T17:33:17.237Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18969
[2020-06-04T17:33:17.503Z] [INFO] getVaultSecret: Getting secrets
[2020-06-04T17:33:17.560Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-06-04T17:33:18.306Z] + chmod 755 generate-build-data.sh
[2020-06-04T17:33:18.306Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18969/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18969/runs/3 FAILURE 4716758
[2020-06-04T17:33:18.306Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18969/runs/3/steps/?limit=10000 -o steps-info.json
[2020-06-04T17:33:20.789Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18969/runs/3/tests/?status=FAILED -o tests-errors.json

@jsoriano
Copy link
Member Author

jsoriano commented Jun 4, 2020

Testing the new pipeline locally I have seen that the script cannot be found, I wonder why this doesn't happen in beats-ci/apm-ci.
I have added a change to explicitly change to the base directory.

@jsoriano jsoriano requested a review from kuisathaverat June 4, 2020 16:15
@jsoriano
Copy link
Member Author

jsoriano commented Jun 9, 2020

@kuisathaverat could you take another look after my latest changes? Thanks!

@jsoriano jsoriano merged commit 247a00b into elastic:master Jun 9, 2020
@jsoriano jsoriano deleted the add-filebeat-docker-builds branch June 9, 2020 14:53
melchiormoulin pushed a commit to melchiormoulin/beats that referenced this pull request Oct 14, 2020
@jsoriano @melchiormoulin
Pre-build docker images used for filebeat integration testing (elasti…
0f85568 
…c#18969)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kuisathaverat kuisathaverat kuisathaverat approved these changes

Assignees

@jsoriano jsoriano

Labels
Team:Automation Label for the Observability productivity team :Testing
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jsoriano @elasticmachine @kuisathaverat