[Perf] [Service Bus] [Event Hubs] Event Based SDKs Perf Framework - part 2 #18859

HarshaNalluru · 2021-11-29T21:55:22Z

What's in the PR?

Event-based(SDKs) perf test class for the AMQP SDKs
Example mock receiver with the subscribe method(drawing parallels with service-bus)
Perf test for the subscribe method offered by the
- mock receiver similar to service-bus receiver
- mock receiver similar to event-hubs receiver
- service-bus receiver
- event-hubs receiver
Minor refactorings/bug fixes

Fixes #18471
Fixes #17952

…ates

… harshan/issue/18471-event-perf-framework-part-2

sdk/test-utils/perf/test/event/mockEventHubConsumerClient.ts

azure-sdk · 2022-02-08T18:59:15Z

API changes have been detected in @azure/event-hubs. You can review API changes here

jeremymeng

Overall looks good to me. I have some comments (mostly stylish)

sdk/eventhub/perf-tests/event-hubs/test/subscribe.spec.ts

sdk/test-utils/perf/src/eventPerfTest.ts

jeremymeng · 2022-02-11T19:13:11Z

sdk/test-utils/perf/test/event/mockEventHubConsumerClient.ts

+export interface SubscribeOptions {
+  /**
+   * The maximum number of partitions to receive from.
+   * - **Default**: `12`.


curious how the default is determined?

it was just a random number, do you have any preference/suggestion?

sdk/test-utils/perf/src/eventPerfTest.ts

sdk/test-utils/perf/test/event/mockEventHubConsumerClient.ts

jeremymeng · 2022-02-11T20:02:00Z

sdk/test-utils/perf/test/event/mockEventHubConsumerClient.ts

+  /**
+   * Raises the error after the specified time.
+   */
+  raiseErrorAfterInSeconds: number;


nit: does raiseErrorDelayInSeconds read better?

delayToRaiseErrorInSeconds probably too long...

- Methods do not perform any async work, and sync methods add less overhead to perf framework

- Now matches NoOpTest * AverageBatchSize - Also update options to match .NET

mikeharder

Please fix raiseErrorAfterInSeconds and I can continue reviewing.

mikeharder · 2022-02-15T02:29:32Z

sdk/test-utils/perf/test/event/mockEventHubConsumerClientTest.spec.ts

+    },
+    maxEventsPerSecond: {
+      description: "Max rate to receive in events/sec",
+      defaultValue: 1000,


maxEventsPerSecond should default to unspecified (unthrottled)

Updated 😎

mikeharder · 2022-02-15T02:32:43Z

sdk/test-utils/perf/test/event/mockEventHubConsumerClientTest.spec.ts

+    },
+    raiseErrorAfterInSeconds: {
+      description: "Raises error after the given duration - in seconds",
+      defaultValue: 15,


raiseErrorAfterInSeconds should default to unspecified (error never thrown).

mikeharder · 2022-02-16T02:31:52Z

sdk/test-utils/perf/test/event/mockEventHubConsumerClient.ts

+        this.handlePartition(handlers.processEvents, i, maxEventsPerSecondPerPartition)
+      );
+    }
+    if (options?.raiseErrorAfterInSeconds) {


raiseErrorAfterInSeconds does not appear to be working for me. If I set this to say 3, the test keeps running for the normal duration and no error is raised.

If I set this to say 3, the test keeps running for the normal duration and no error is raised.

Error should have been thrown(or logged at 3 seconds), but the test is active since the duration timeout wasn't cleared.
Updated the "program" to clear the timeout once the runAll calls were done.

… harshan/issue/18471-event-perf-framework-part-2

…ps://github.com/HarshaNalluru/azure-sdk-for-js into harshan/issue/18471-event-perf-framework-part-2

mikeharder

Approved pending remaining feedback

mikeharder · 2022-03-18T23:29:59Z

sdk/eventhub/perf-tests/event-hubs/test/subscribe.spec.ts

+      {
+        processEvents: async (events: ReceivedEventData[], _context: PartitionContext) => {
+          for (const _event of events) {
+            console.log(_event.sequenceNumber, _context.partitionId);


Remove debugging console.log

mikeharder · 2022-03-18T23:33:17Z

sdk/eventhub/perf-tests/event-hubs/test/subscribe.spec.ts

+      );
+      await producer.sendBatch(batch);
+      numberOfEventsSent = numberOfEventsSent + batch.count;
+      console.log(numberOfEventsSent);


Remove debugging console.log

mikeharder · 2022-03-18T23:42:23Z

sdk/eventhub/perf-tests/event-hubs/test/subscribe.spec.ts

+    const batch = await producer.createBatch({ partitionId });
+    let numberOfEventsSent = 0;
+    // add events to our batch
+    while (numberOfEventsSent <= numberOfEventsPerPartition) {


Suggested change

while (numberOfEventsSent <= numberOfEventsPerPartition) {

while (numberOfEventsSent < numberOfEventsPerPartition) {

Off-by-one bug

update (Azure#18859)

HarshaNalluru added 26 commits November 24, 2021 07:13

clean up old scripts

3037243

tests -> perfTest

e437726

base, event, batch perf test classes

2a13a51

program.ts

1d00bd8

incorrect comment

b6303ed

postSetup and preCleanup

47692c1

ParsedPerfOptions<TOptions = Record<string, unknown>> and related upd…

d20e8d8

…ates

remove ! from options

4d1db88

MockReceiverTest

c5b6ff5

import "@azure/abort-controller" at perf classes

daa37e8

BatchReceiveTest

3cc719c

remove !

8c8ceb1

changelog

b30f0f0

receiveBatch test

9191472

remove aborter

6987bd9

docs

8a1c33c

BatchSendTest

7fb6d39

receiveMessages

d42de56

no need of timer

c7e2f21

- Adds PerfTestBase and BatchPerfTest.

d5b7c5d

lint fixes

04cc2f3

checkpoitn

12ce89f

checkpoint

1eb110f

manage parallels in base class

b4656aa

parallels in base class

cce3365

MockEventReceiverTest and the framework

f810647

ghost added Service Bus Storage Storage Service (Queues, Blobs, Files) labels Nov 29, 2021

HarshaNalluru added 2 commits January 13, 2022 22:20

Merge branch 'main' of https://github.com/Azure/azure-sdk-for-js into…

66c2a88

… harshan/issue/18471-event-perf-framework-part-2

fix readme

4c54a3a

HarshaNalluru changed the title ~~[Perf] [Service Bus] Event Based SDKs Perf Framework - part 2~~ [Perf] [Service Bus] [Event Hubs] Event Based SDKs Perf Framework - part 2 Feb 7, 2022

HarshaNalluru requested a review from deyaaeldeen February 7, 2022 22:46

HarshaNalluru added 2 commits February 7, 2022 23:45

Merge branch 'main' of https://github.com/Azure/azure-sdk-for-js into…

8e1b45f

… harshan/issue/18471-event-perf-framework-part-2

Merge branch 'main' of https://github.com/Azure/azure-sdk-for-js into…

0f42b4f

… harshan/issue/18471-event-perf-framework-part-2

HarshaNalluru commented Feb 8, 2022

View reviewed changes

sdk/test-utils/perf/test/event/mockEventHubConsumerClient.ts Outdated Show resolved Hide resolved

HarshaNalluru commented Feb 8, 2022

View reviewed changes

sdk/test-utils/perf/test/event/mockEventHubConsumerClient.ts Outdated Show resolved Hide resolved

Apply suggestions from code review

306d0ad

HarshaNalluru mentioned this pull request Feb 10, 2022

[ServiceBus] add delay between infinite retry cycles #20316

Merged

1 task

jeremymeng approved these changes Feb 11, 2022

View reviewed changes

mikeharder added 4 commits February 15, 2022 16:29

Make EventPerfTest.eventRaise() and errorRaised() sync instead of async

b626970

- Methods do not perform any async work, and sync methods add less overhead to perf framework

Cache event arrays to increase MockEventHubConsumerClient throughput

2952bb0

Add MockReceiverTest to test list

5e2cf7c

Improve throughput of MockReceiverTest

03233e9

- Now matches NoOpTest * AverageBatchSize - Also update options to match .NET

mikeharder requested changes Feb 16, 2022

View reviewed changes

HarshaNalluru added 5 commits February 18, 2022 09:07

Merge branch 'main' of https://github.com/Azure/azure-sdk-for-js into…

0b6abd0

… harshan/issue/18471-event-perf-framework-part-2

Merge branch 'harshan/issue/18471-event-perf-framework-part-2' of htt…

e4e03d4

…ps://github.com/HarshaNalluru/azure-sdk-for-js into harshan/issue/18471-event-perf-framework-part-2

Jeremy's feedback

1875154

Clear the duration timer if the test ended sooner

4c675f8

Mike's feedbakc

4331035

HarshaNalluru requested a review from mikeharder February 18, 2022 09:45

lint:fix

50b5b38

mikeharder approved these changes Mar 18, 2022

View reviewed changes

minor feedback

cfe07fe

HarshaNalluru enabled auto-merge (squash) March 21, 2022 18:58

HarshaNalluru merged commit efe321c into Azure:main Mar 21, 2022

HarshaNalluru deleted the harshan/issue/18471-event-perf-framework-part-2 branch March 22, 2022 18:03

This was referenced Mar 22, 2022

[Service Bus] Perf Test for "subscribe" method #20977

Closed

[Event Hubs] Perf Test for "subscribe" method #20978

Closed

azure-sdk pushed a commit to azure-sdk/azure-sdk-for-js that referenced this pull request Apr 29, 2022

CodeGen from PR 18859 in Azure/azure-rest-api-specs

7e20b4a

update (Azure#18859)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Perf] [Service Bus] [Event Hubs] Event Based SDKs Perf Framework - part 2 #18859

[Perf] [Service Bus] [Event Hubs] Event Based SDKs Perf Framework - part 2 #18859

HarshaNalluru commented Nov 29, 2021 •

edited

Loading

azure-sdk commented Feb 8, 2022

jeremymeng left a comment

jeremymeng Feb 11, 2022

HarshaNalluru Feb 18, 2022

jeremymeng Feb 11, 2022

jeremymeng Feb 11, 2022

mikeharder left a comment

mikeharder Feb 15, 2022

HarshaNalluru Feb 18, 2022

mikeharder Feb 15, 2022

HarshaNalluru Feb 18, 2022

mikeharder Feb 16, 2022

HarshaNalluru Feb 18, 2022 •

edited

Loading

mikeharder left a comment

mikeharder Mar 18, 2022

mikeharder Mar 18, 2022

mikeharder Mar 18, 2022

	while (numberOfEventsSent <= numberOfEventsPerPartition) {
	while (numberOfEventsSent < numberOfEventsPerPartition) {

[Perf] [Service Bus] [Event Hubs] Event Based SDKs Perf Framework - part 2 #18859

[Perf] [Service Bus] [Event Hubs] Event Based SDKs Perf Framework - part 2 #18859

Conversation

HarshaNalluru commented Nov 29, 2021 • edited Loading

What's in the PR?

azure-sdk commented Feb 8, 2022

jeremymeng left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mikeharder left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HarshaNalluru Feb 18, 2022 • edited Loading

Choose a reason for hiding this comment

mikeharder left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HarshaNalluru commented Nov 29, 2021 •

edited

Loading

HarshaNalluru Feb 18, 2022 •

edited

Loading