Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fully automate stress cluster buildout and add support for azure file share mounting #2106

Merged
10 commits merged into from
Oct 22, 2021
Merged

Fully automate stress cluster buildout and add support for azure file share mounting #2106

10 commits merged into from
Oct 22, 2021

Conversation

benbp
Copy link
Member

@benbp benbp commented Oct 14, 2021 •
edited
Loading

  • Fully automate cluster buildout. Add azure file share mount to stress tests.
    • Moving the test/ad-hoc cluster back to the playground subscription
    • Upgrading kubernetes cluster version to 1.21.x to pull in support for the azure csi file driver
    • Adding high memory agent nodes to the base deployment
    • Enabling node autoscaler in the base deployment
  • Publish stress watcher image in CI. Run docker build on PR
    • Using common image location across stress clusters to simplify buildout+deployment
  • Add stress test debug file share usage example

Resolves #1903

@benbp benbp requested review from ckairen and a team as code owners October 14, 2021 21:42
@benbp benbp added the Central-EngSys This issue is owned by the Engineering System team. label Oct 14, 2021
@benbp benbp self-assigned this Oct 14, 2021
@benbp benbp changed the title benbp/stress volume Fully automate stress cluster buildout and add support for azure file share mounting Oct 14, 2021
@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

@benbp benbp requested review from weshaggard and anuchandy October 14, 2021 21:53
@benbp benbp force-pushed the benbp/stress-volume branch from 0979698 to 3785762 Compare October 14, 2021 21:56
benbp
benbp commented Oct 14, 2021
View reviewed changes
eng/containers/ci.yml Outdated Show resolved Hide resolved
@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

@benbp benbp force-pushed the benbp/stress-volume branch from 3785762 to 4cc64b7 Compare October 14, 2021 22:01
@benbp benbp added pillar-reliability The issue is related to reliability, one of our core engineering pillars. (includes stress testing) Stress This issue is related to stress testing, part of our reliability pillar. labels Oct 14, 2021
weshaggard
weshaggard reviewed Oct 14, 2021
View reviewed changes
eng/containers/ci.yml Outdated Show resolved Hide resolved
weshaggard
weshaggard reviewed Oct 14, 2021
View reviewed changes
eng/containers/ci.yml Outdated Show resolved Hide resolved
@benbp benbp force-pushed the benbp/stress-volume branch from 7bc1654 to 27e997f Compare October 14, 2021 22:43
@benbp
Copy link
Member Author

benbp commented Oct 14, 2021

@weshaggard I added some docs if you care to take a look: https://github.com/Azure/azure-sdk-tools/blob/27e997f44b251400927c1264e9a3839a6d7cabf3/tools/stress-cluster/cluster/README.md#development

@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

@benbp benbp force-pushed the benbp/stress-volume branch from bda2226 to 6ae6dad Compare October 14, 2021 22:56
@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

weshaggard
weshaggard reviewed Oct 18, 2021
View reviewed changes
tools/stress-cluster/cluster/README.md Outdated
# Dependencies

- [Powershell Core](https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell-core-on-linux?view=powershell-7.1#ubuntu-2004) (if using Linux)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend powershell core for all OS's, as there is a lot of improvements in networking over top of windows powershell as well.

weshaggard
weshaggard approved these changes Oct 18, 2021
View reviewed changes
Copy link
Member

@weshaggard weshaggard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable. The true test for the docs will be when someone actually uses them.

@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

This was referenced Oct 22, 2021
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
@ghost
Copy link

ghost commented Oct 22, 2021

Hello @azure-sdk!

Because this pull request has the auto-merge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

@ghost ghost merged commit dd1c8ca into Azure:main Oct 22, 2021
@benbp benbp deleted the benbp/stress-volume branch October 22, 2021 22:08
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@weshaggard weshaggard weshaggard approved these changes

@scbedd scbedd scbedd approved these changes

@ckairen ckairen Awaiting requested review from ckairen ckairen is a code owner

@anuchandy anuchandy Awaiting requested review from anuchandy

Assignees

@benbp benbp

Labels
Central-EngSys This issue is owned by the Engineering System team. pillar-reliability The issue is related to reliability, one of our core engineering pillars. (includes stress testing) Stress This issue is related to stress testing, part of our reliability pillar.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[stress testing] Prefix init container names with "init-"
4 participants
@benbp @azure-sdk @weshaggard @scbedd