Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: enable synthetics tests for heartbeats #32710

Merged
merged 3 commits into from
Aug 24, 2022

Conversation

v1v
Copy link
Member

@v1v v1v commented Aug 16, 2022

What

So, we'd want our CI boxes that run the unit tests for x-pack/heartbeat to:

  1. Have node available
  2. Have npm i -g @elastic/synthetics installed
  3. Have the ELASTIC_SYNTHETICS_CAPABLE flag set to true to signal that it can run these tests.

Why

With #32542 we now have improved testing of browser based monitors in heartbeat. We don't, however, have a good way to run these tests using the actual synthetics node package. Heartbeat fork/execs the elastic-synthetics binary provided by npm i -g @elastic/synthetics to run browser monitors.

Test

The first build ran the below steps:
image

heartbeat

NOTE: As requested it will not run on heartbeat but x-pack/heartbeat only.

This log is the one with the npm i -g @elastic/synthetics execution and this other log is the one with the traces for running mage build unitTest for the hearbeat

x-pack/heartbeat when mage unitTest

image

This log is the one with the traces for running mage build unitTest for the x-pack/hearbeat, @andrewvc , can you please verify if that's the expected behaviour?

x-pack/heartbeat when mage goIntegTest

image

@v1v v1v added the backport-v8.4.0 Automated backport with mergify label Aug 16, 2022
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Aug 16, 2022
@mergify mergify bot assigned v1v Aug 16, 2022
@v1v v1v added automation Team:Automation Label for the Observability productivity team labels Aug 16, 2022
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 16, 2022
@elasticmachine
Copy link
Collaborator

elasticmachine commented Aug 16, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-08-17T08:36:31.977+0000

  • Duration: 109 min 27 sec

Test stats 🧪

Test Results
Failed 0
Passed 22621
Skipped 1937
Total 24558

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

heartbeat/Jenkinsfile.yml Outdated Show resolved Hide resolved
@v1v v1v marked this pull request as ready for review August 17, 2022 14:41
@v1v v1v requested review from a team as code owners August 17, 2022 14:41
@v1v v1v requested review from belimawr, cmacknz and a team and removed request for a team August 17, 2022 14:41
@cmacknz cmacknz removed request for belimawr, a team and cmacknz August 17, 2022 17:25
@v1v v1v requested a review from andrewvc August 18, 2022 11:18
Copy link
Contributor

@amannocci amannocci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@andrewvc andrewvc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I'm not perhaps totally qualified to review CI stuff, but looks correct from what I can tell.

Thanks so much!

@v1v
Copy link
Member Author

v1v commented Aug 24, 2022

Thanks @andrewvc for the review, there are some questions regarding how to verify if those tests were executed, would you mind going through them so we can validate these changes run the expected tests?

Scenarios

For x-pack/heartbeat then mage unitTest with the nodejs environment

image

What's the output for the mage build unitTest command?

10:10:01  + mage build unitTest
10:10:10  >> build: Building heartbeat
10:11:31  >> go test: Unit Testing
10:11:31  exec: gotestsum --no-color -f standard-quiet --junitfile build/TEST-go-unit.xml --jsonfile build/TEST-go-unit.out.json -- -tags null oracle -covermode=atomic -coverprofile=build/TEST-go-unit.cov ./...
10:12:18  ok  	github.com/elastic/beats/v7/x-pack/heartbeat/monitors/browser	0.013s	coverage: 54.7% of statements
10:12:18  ok  	github.com/elastic/beats/v7/x-pack/heartbeat/monitors/browser/synthexec	1.758s	coverage: 82.1% of statements
10:12:18  ok  	github.com/elastic/beats/v7/x-pack/heartbeat/scenarios	2.861s	coverage: 90.5% of statements
10:12:18  ok  	github.com/elastic/beats/v7/x-pack/heartbeat/monitors/browser/source	5.226s	coverage: 65.9% of statements
10:12:21  ok  	github.com/elastic/beats/v7/x-pack/heartbeat	0.155s	coverage: 0.0% of statements
10:12:21  ?   	github.com/elastic/beats/v7/x-pack/heartbeat/cmd	[no test files]
10:12:21  ?   	github.com/elastic/beats/v7/x-pack/heartbeat/include	[no test files]
10:12:21  ?   	github.com/elastic/beats/v7/x-pack/heartbeat/monitors/browser/source/fixtures	[no test files]
10:12:21  ?   	github.com/elastic/beats/v7/x-pack/heartbeat/monitors/browser/synthexec/testcmd	[no test files]
10:12:21  
10:12:21  DONE 152 tests in 61.109s
10:12:22  >> go test: Unit Test Passed
10:12:34  >> python test: Unit Testing
10:13:03  >> python test: Unit Testing - No tests found.

This log is the one with the traces for running mage build unitTest for the x-pack/hearbeat, @andrewvc , can you please verify if that's the expected behaviour?

For x-pack/heartbeat then mage goIntegTest with the nodejs environment

image

image

What's the output for the mage goIntegTest command?

10:09:56  + mage goIntegTest
10:10:05  >> integTest: Complete (no tests require the integ test environment)

This log is the one with the traces for running mage goIntegTest for the x-pack/hearbeat, @andrewvc , can you please verify if that's the expected behaviour?

@andrewvc
Copy link
Contributor

That's correct for now @v1v , our first integration test lands in this PR here: https://github.com/elastic/beats/pull/30632/files#diff-fc428ce882a18d78740376d34382dc5996f9908d014ec985f31b71d2a2ca11edR18

along with magefile improvements here: https://github.com/elastic/beats/pull/30632/files#diff-943c67c8c343aa1bdb9cf4ef7c141f388218835dc996232fec36d77970e05344R8

@v1v v1v merged commit 29fe9bb into elastic:main Aug 24, 2022
mergify bot pushed a commit that referenced this pull request Aug 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automation backport-v8.4.0 Automated backport with mergify Team:Automation Label for the Observability productivity team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants