Eventhub Stress Test Onboarding #28320

m-redding · 2022-04-21T19:16:14Z

Contributing to the Azure SDK

Please see our CONTRIBUTING.md if you are not familiar with contributing to this repository or have questions.

For specific information about pull request etiquette and best practices, see this section.

check-enforcer · 2022-04-21T19:26:32Z

This pull request is protected by Check Enforcer.
For more information about how to run a pipeline against this pull request, see this.

...nthub/Azure.Messaging.EventHubs.Shared/src/Azure.Messaging.EventHubs.Shared.Stress.projitems

sdk/eventhub/Azure.Messaging.EventHubs/stress/Chart.yaml

benbp · 2022-04-21T20:21:38Z

sdk/eventhub/Azure.Messaging.EventHubs/stress/Dockerfile

+COPY /sdk/eventhub/Azure.Messaging.EventHubs/ /app/sdk/eventhub/Azure.Messaging.EventHubs/
+COPY /eng /app/eng
+COPY /tools /app/tools


Do these also need to be ./ instead of /?

They're referencing the root, and aren't relative to the current directory, so I believe they're correct as written.

@benbp: Any thoughts on templatizing some of this? The inclusions here will be needed for any .NET library to build due to the engsys dependencies.

@jsquire I'm definitely in favor of common stress components, either global or language level. For this dockerfile, my initial thought would be to create an intermediate builder image, so you then you can do something like FROM azsdkengsys.azurecr.io/stress/dotnet/base AS build-env?

Development could be somewhat annoying if you're iterating on both, so we may want to provide a way to build multiple dockerfiles in sequence if it starts getting iterated on heavily.

I really like the idea of the intermediate image.

benbp · 2022-04-21T20:23:47Z

sdk/eventhub/Azure.Messaging.EventHubs/stress/buildstress.ps1

+# TODO: how to update registry path
+
+Set-Location -Path ..\..\..\..\
+Set-Location -Path .\artifacts\bin\Azure.Messaging.EventHubs.Stress\Debug\netcoreapp3.1


This script will fail the first time you run it if you have no eventhubs build.

sdk/eventhub/Azure.Messaging.EventHubs/stress/Chart.yaml

benbp · 2022-04-21T20:28:07Z

sdk/eventhub/Azure.Messaging.EventHubs/stress/README.md

+```
+
+### Running Tests
+When tests are run locally, Azure resources need to be created prior to running the test. This can be done through the Azure CLI, an ARM template or bicep file, or the Azure Portal. The user is required to input the connection strings upon request on the command line when the test is being run. For more information about what resources are needed for each test, see the "Scenario Information" section below. 


@ckairen I think we should also support a "deploy resources only" type mode to make local testing easier with resources.

sdk/eventhub/Azure.Messaging.EventHubs/stress/Dockerfile

benbp · 2022-04-21T21:20:13Z

sdk/eventhub/Azure.Messaging.EventHubs/stress/stress-test-resources.json

@@ -0,0 +1,165 @@
+{


I would avoid checking this file in and also add a .gitignore for it (example) since it's an intermediate artifact.

Omitting it will help ensure we don't have confusion where someone who wants to make a change edits this template rather than the bicep file and gets confused when it doesn't work.

benbp · 2022-04-21T21:21:34Z

sdk/eventhub/Azure.Messaging.EventHubs/stress/stress-test-resources.bicep

+@description('The location of the resource group.')
+param location string = resourceGroup().location
+
+var eventHubsNamespace_var = 'eh-${resourceGroup().name}'


If you only use the resource group name here, it will be a little simpler to use the auto-generated namespace URL in the network blocking config, but it should still work fine with eh- we'll just have to remember to add it in other places.

benbp · 2022-04-21T21:23:12Z

A couple nits/suggestions but the stress configuration pieces LGTM.

benbp · 2022-04-25T17:43:14Z

sdk/eventhub/Azure.Messaging.EventHubs/stress/README.md

+
+Please see our [contributing guide](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/eventhub/Azure.Messaging.EventHubs/CONTRIBUTING.md) for more information.
+
+![Impressions](https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-net%2Fsdk%2Feventhub%2FAzure.Messaging.EventHubs%2stress%2FREADME.png)


This png fails to load on the readme.

Loads for me; it's one of our tracking pixels, so it will look like nothing. @benbp: Are you seeing it 404 or just nothing show up?

@jsquire it looks like the problem is a missing F character in the url encoding: EventHubs %2 stress. I added a commit suggestion below.

It looks like this on the readme, that's how I caught it:

ahhh.... I see it now. Yeah, that's on me. Thanks for the catch!

Chrome/Edge do a nice job of fixing the malformed encoding. Checking in the browser worked.

sdk/eventhub/Azure.Messaging.EventHubs/stress/README.md

benbp · 2022-04-26T00:11:10Z

sdk/eventhub/Azure.Messaging.EventHubs/stress/stress-test-resources.json

@@ -5,7 +5,7 @@
    "_generator": {
      "name": "bicep",
      "version": "0.4.1318.3566",
-      "templateHash": "9849786101028970025"


I think because you previously checked this in, the .gitignore won't automatically remove it. So you have to manually remove it with git rm and commit the file deletion.

eng/Packages.Data.props

sdk/eventhub/Azure.Messaging.EventHubs/stress/Dockerfile

sdk/eventhub/Azure.Messaging.EventHubs/stress/src/Infrastructure/BufferedProducerTestConfig.cs

sdk/eventhub/Azure.Messaging.EventHubs/stress/src/Infrastructure/BufferedPublisher.cs

sdk/eventhub/Azure.Messaging.EventHubs/stress/src/Program.cs

sdk/eventhub/Azure.Messaging.EventHubs/stress/src/Scenarios/BufferedProducerTest.cs

sdk/eventhub/Azure.Messaging.EventHubs/stress/src/Scenarios/EventProducerTest.cs

sdk/eventhub/Azure.Messaging.EventHubs/stress/stress-test-resources.bicep

jsquire · 2022-04-26T15:11:40Z

We should add the ConfigureAwait(false) to all the things at some point in the near future. It'll be helpful for ensuring that things keep flowing efficiently since we're in a resource-limited host environment.

sdk/eventhub/Azure.Messaging.EventHubs/stress/README.md

Co-authored-by: Ben Broderick Phillips <[email protected]>

sdk/eventhub/Azure.Messaging.EventHubs/stress/Dockerfile

sdk/eventhub/Azure.Messaging.EventHubs/stress/README.md

sdk/eventhub/Azure.Messaging.EventHubs/stress/src/Configurations/BufferedProducerTestConfig.cs

jsquire · 2022-04-29T13:13:57Z

sdk/eventhub/Azure.Messaging.EventHubs/stress/src/Infrastructure/BufferedPublisher.cs

+
+                // Create the buffered producer client and register the success and failure handlers
+
+                var options = new EventHubBufferedProducerClientOptions


The Buffered Producer has some adjustments to the standard defaults for retry policy if the options aren't explicitly passed. We should mirror them for the test, since that'll be our majority case.

Source: https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/eventhub/Azure.Messaging.EventHubs/src/Producer/EventHubBufferedProducerClient.cs#L68

If you have ideas on how we can better detect and adjust retry configuration when the individual values aren't set, I'd be very interested in hearing them. The best that I can think of is creating a mirrored type inside the buffered options. Maybe we should explore that as a quick interlude between phase 1 and phase 2.

sdk/eventhub/Azure.Messaging.EventHubs/stress/src/Infrastructure/BufferedPublisher.cs

jsquire · 2022-04-29T13:19:23Z

sdk/eventhub/Azure.Messaging.EventHubs/stress/src/Infrastructure/BufferedPublisher.cs

+                {
+                    // If this catch is hit, it means the producer has restarted, collect metrics.
+
+                    _metrics.Client.GetMetric(_metrics.ProducerRestarted).TrackValue(1);


Does this increment the count or are we collecting these as individual events and then aggregating via Kusto?

@jsquire The GetMetric method is cool it aggregates (in this case sums) up all the values sent via "TrackValue" and sends them all at once:
"In summary GetMetric() is the recommended approach since it does pre-aggregation, it accumulates values from all the Track() calls and sends a summary/aggregate once every minute. This can significantly reduce the cost and performance overhead by sending fewer data points, while still collecting all relevant information." ref
The long-term aggregation like total sum is done in Kusto.

sdk/eventhub/Azure.Messaging.EventHubs/stress/src/Infrastructure/EnvironmentReader.cs

sdk/eventhub/Azure.Messaging.EventHubs/stress/src/Infrastructure/Publisher.cs

…re/Publisher.cs Co-authored-by: Jesse Squire <[email protected]>

…ns/BufferedProducerTestConfig.cs Co-authored-by: Jesse Squire <[email protected]>

m-redding and others added 12 commits March 28, 2022 15:37

stress test initial setup

31ef454

initial setup of publish test after refactoring/app insights integration

2c97223

added basic buffered test

7e67bda

deployment infrastructure update

bcf27f8

updated buffered producer test

32d1a79

buffered producer test cleanup

99f6135

buffered producer test

d68b9a8

buffered producer test

3bb3cfc

Adding more scenarios and switching to event producer test

72de30e

restructuring metrics

8e3ee3f

more metrics updates

9c24f75

Merge branch 'Azure:main' into eventhub-stress-tests

7f63000

m-redding requested review from chidozieononiwu, weshaggard, benbp, serkantkaraca and jsquire as code owners April 21, 2022 19:16

ghost added the Event Hubs label Apr 21, 2022

benbp reviewed Apr 21, 2022

View reviewed changes

...nthub/Azure.Messaging.EventHubs.Shared/src/Azure.Messaging.EventHubs.Shared.Stress.projitems Outdated Show resolved Hide resolved

benbp reviewed Apr 21, 2022

View reviewed changes

sdk/eventhub/Azure.Messaging.EventHubs/stress/Chart.yaml Show resolved Hide resolved

benbp reviewed Apr 21, 2022

View reviewed changes

sdk/eventhub/Azure.Messaging.EventHubs/stress/Chart.yaml Show resolved Hide resolved

benbp reviewed Apr 21, 2022

View reviewed changes

sdk/eventhub/Azure.Messaging.EventHubs/stress/Dockerfile Show resolved Hide resolved

benbp reviewed Apr 21, 2022

View reviewed changes

README updates

2ee7f56

benbp reviewed Apr 25, 2022

View reviewed changes

sdk/eventhub/Azure.Messaging.EventHubs/stress/README.md Show resolved Hide resolved

benbp reviewed Apr 25, 2022

View reviewed changes

sdk/eventhub/Azure.Messaging.EventHubs/stress/README.md Outdated Show resolved Hide resolved

m-redding added 2 commits April 25, 2022 11:02

initial refactoring

25ec412

deploy yaml file updates

f16edd8

benbp reviewed Apr 26, 2022

View reviewed changes

m-redding and others added 2 commits April 25, 2022 17:17

readme updates

8c6ac6d

Delete stress-test-resources.json

dbb7258

jsquire reviewed Apr 26, 2022

View reviewed changes

m-redding and others added 2 commits April 26, 2022 10:15

dockerfile updates

1013982

Delete stress-test-resources.json

ae204ce

benbp reviewed Apr 26, 2022

View reviewed changes

sdk/eventhub/Azure.Messaging.EventHubs/stress/README.md Outdated Show resolved Hide resolved

m-redding and others added 2 commits April 28, 2022 14:57

comment and main updates

bf4230b

Update sdk/eventhub/Azure.Messaging.EventHubs/stress/README.md

925888e

Co-authored-by: Ben Broderick Phillips <[email protected]>

jsquire approved these changes Apr 29, 2022

View reviewed changes

m-redding and others added 9 commits April 29, 2022 10:39

Update sdk/eventhub/Azure.Messaging.EventHubs/stress/src/Infrastructu…

a86d0a6

…re/Publisher.cs Co-authored-by: Jesse Squire <[email protected]>

Update sdk/eventhub/Azure.Messaging.EventHubs/stress/src/Configuratio…

518aa6d

…ns/BufferedProducerTestConfig.cs Co-authored-by: Jesse Squire <[email protected]>

Update sdk/eventhub/Azure.Messaging.EventHubs/stress/src/Configuratio…

eafe6a5

…ns/BufferedProducerTestConfig.cs Co-authored-by: Jesse Squire <[email protected]>

feedback updates

da89c76

Added Azure Event Listener

c5cd920

buffered producer options

3dab67c

typo

02ce4ee

Event Hubs Stress Test Onboarding

2161001

Event Hubs Stress Test Onboarding

ca66f38

m-redding merged commit adf95c7 into Azure:mredding/stress-test-onboarding Apr 29, 2022

m-redding added a commit that referenced this pull request Apr 30, 2022

Eventhub Stress Test Onboarding (#28320) (#28471)

2154830

paterasMSFT pushed a commit that referenced this pull request Jun 15, 2022

Eventhub Stress Test Onboarding (#28320) (#28471)

1b7f402

zhihaoxue pushed a commit to zhihaoxue/azure-sdk-for-net that referenced this pull request Jul 27, 2022

Eventhub Stress Test Onboarding (Azure#28320) (Azure#28471)

221abba

m-redding deleted the eventhub-stress-tests branch January 10, 2024 16:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Eventhub Stress Test Onboarding #28320

Eventhub Stress Test Onboarding #28320

m-redding commented Apr 21, 2022

check-enforcer bot commented Apr 21, 2022

benbp Apr 21, 2022

jsquire Apr 22, 2022

jsquire Apr 22, 2022

benbp Apr 22, 2022

benbp Apr 22, 2022

jsquire Apr 22, 2022

benbp Apr 21, 2022

benbp Apr 21, 2022

benbp Apr 21, 2022

jsquire Apr 22, 2022

benbp Apr 21, 2022

benbp commented Apr 21, 2022 •

edited

Loading

benbp Apr 25, 2022

jsquire Apr 26, 2022 •

edited

Loading

benbp Apr 26, 2022 •

edited

Loading

jsquire Apr 27, 2022

jsquire Apr 27, 2022

benbp Apr 26, 2022

jsquire commented Apr 26, 2022

jsquire Apr 29, 2022

jsquire Apr 29, 2022

jsquire Apr 29, 2022

m-redding Apr 29, 2022 •

edited

Loading


		Please see our [contributing guide](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/eventhub/Azure.Messaging.EventHubs/CONTRIBUTING.md) for more information.

		![Impressions](https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-net%2Fsdk%2Feventhub%2FAzure.Messaging.EventHubs%2stress%2FREADME.png)


		// Create the buffered producer client and register the success and failure handlers

		var options = new EventHubBufferedProducerClientOptions

Eventhub Stress Test Onboarding #28320

Eventhub Stress Test Onboarding #28320

Conversation

m-redding commented Apr 21, 2022

Contributing to the Azure SDK

check-enforcer bot commented Apr 21, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benbp commented Apr 21, 2022 • edited Loading

Choose a reason for hiding this comment

jsquire Apr 26, 2022 • edited Loading

Choose a reason for hiding this comment

benbp Apr 26, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsquire commented Apr 26, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

m-redding Apr 29, 2022 • edited Loading

Choose a reason for hiding this comment

benbp commented Apr 21, 2022 •

edited

Loading

jsquire Apr 26, 2022 •

edited

Loading

benbp Apr 26, 2022 •

edited

Loading

m-redding Apr 29, 2022 •

edited

Loading