Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding GitHub Self Hosted Runner tests #8721

Merged
merged 11 commits into from
Sep 1, 2021

Conversation

GuptaManan100
Copy link
Member

@GuptaManan100 GuptaManan100 commented Aug 30, 2021

Description

Adds infrastructure for shifting end to end tests to self-hosted runners. It also adds the documentation to how to use the self-hosted runners to find all the debug information.

Checklist

  • Should this PR be backported?
  • Tests were added or are not required
  • Documentation was added or is not required

@GuptaManan100 GuptaManan100 added Type: Enhancement Logical improvement (somewhere between a bug and feature) Component: Build/CI labels Aug 30, 2021
@GuptaManan100 GuptaManan100 marked this pull request as ready for review August 31, 2021 14:48
@GuptaManan100 GuptaManan100 requested review from deepthi, shlomi-noach and a team August 31, 2021 15:47
Copy link
Member

@frouioui frouioui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is amazing! Looks very good to me. I left several suggestions and questions.
Can't wait to get rid of this flakiness, thanks a bunch @GuptaManan100 🚀

@frouioui
Copy link
Member

By the way, @GuptaManan100, do you have an approximate idea of how flaky the tests are, (if they are), with this new configuration?

Copy link
Member

@deepthi deepthi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one suggestion, the rest LGTM.
We can merge once all feedback is addressed.

@GuptaManan100
Copy link
Member Author

By the way, @GuptaManan100, do you have an approximate idea of how flaky the tests are, (if they are), with this new configuration?

Yes, @frouioui The unit tests fail about once in 3 runs and vreplication tests fail once in 4 runs. It is less flakier than GitHub runners and faster too. The major motivation of this however, is to be able to look at the logs of the vtdataroot directory and fix the flakiness. With this change we will be able to easily move a test to the self-hosted runners if it is flaky and fix the issue

@frouioui
Copy link
Member

frouioui commented Sep 1, 2021

By the way, @GuptaManan100, do you have an approximate idea of how flaky the tests are, (if they are), with this new configuration?

Yes, @frouioui The unit tests fail about once in 3 runs and vreplication tests fail once in 4 runs. It is less flakier than GitHub runners and faster too. The major motivation of this however, is to be able to look at the logs of the vtdataroot directory and fix the flakiness. With this change we will be able to easily move a test to the self-hosted runners if it is flaky and fix the issue

This is amazing

Copy link
Member

@frouioui frouioui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@systay systay merged commit 517f5a5 into vitessio:main Sep 1, 2021
@systay systay deleted the github-self-hosted branch September 1, 2021 07:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Build/CI Type: Enhancement Logical improvement (somewhere between a bug and feature)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants