Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AMQP Perf Test Framework support (Batch and Event Processor APIs) #26031

Merged
merged 13 commits into from
Mar 22, 2022

Conversation

g2vinay
Copy link
Member

@g2vinay g2vinay commented Dec 14, 2021

Adds Performance Testing Support for AMQP Batch and Event Processor APIs.

Restructured the Performance Framework in following manner:

image

  • The Batch Perf Test class allows to add performance tests for batch APIs in AMQP and HTTP land, example batch receiving.

  • The Event Perf Test class allows to add performance tests fpr Event Processor APIs in AMQP land, example Event Processor.

  • Note, The class names do not have Stress in them, that is intentional, as these are not meant for Stress tests.

  • The name of existing PerfStressTest class will also be renamed to remove Stress in a following PR.


Tests added to prove functionality:


image

Regression Tests:

No Op Test:

Concurrency Type Main PR
1 async 2,293,753.76 2,215,610.77
32 async 7,171,413.84 7,343,269.63
1 sync 13,246,290.48 13,230,261.64
32 sync 39,063,083.17 39,441,495.25

Storage Download Blob Test:
Size: 10240 bytes.

Concurrency Type Main PR
1 async 307.34 304.56
32 async 6,765 6,907

Functionality Tests:

Event Perf Test (MockEventProcessor):
Java Event Processors are all sync only.
Max Events Per Second: null

Concurrency Type Main (ops/s)
1 Sync 11,482,996.28

Batch Perf Test (Mock Batch Receiver)
Min Messages = 0;
Max Messages = 10;

Concurrency Type Main (ops/s)
1 Async 4,953,801.19
32 Async 14,939,589.71
1 sync 12,124,112.65
32 sync 16,845,403.99

@g2vinay g2vinay marked this pull request as ready for review December 14, 2021 17:31
@g2vinay g2vinay requested a review from srnagar January 21, 2022 23:54
@g2vinay g2vinay changed the title Batch Event APIs Perf Test Framework support. AMQP Perf Test Framework support (Batch and Event Processor APIs) Jan 21, 2022
Copy link
Member

@mikeharder mikeharder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved pending remaining comments.

@check-enforcer
Copy link

This pull request is protected by Check Enforcer.

What is Check Enforcer?

Check Enforcer helps ensure all pull requests are covered by at least one check-run (typically an Azure Pipeline). When all check-runs associated with this pull request pass then Check Enforcer itself will pass.

Why am I getting this message?

You are getting this message because Check Enforcer did not detect any check-runs being associated with this pull request within five minutes. This may indicate that your pull request is not covered by any pipelines and so Check Enforcer is correctly blocking the pull request being merged.

What should I do now?

If the check-enforcer check-run is not passing and all other check-runs associated with this PR are passing (excluding license-cla) then you could try telling Check Enforcer to evaluate your pull request again. You can do this by adding a comment to this pull request as follows:
/check-enforcer evaluate
Typically evaulation only takes a few seconds. If you know that your pull request is not covered by a pipeline and this is expected you can override Check Enforcer using the following command:
/check-enforcer override
Note that using the override command triggers alerts so that follow-up investigations can occur (PRs still need to be approved as normal).

What if I am onboarding a new service?

Often, new services do not have validation pipelines associated with them, in order to bootstrap pipelines for a new service, you can issue the following command as a pull request comment:
/azp run prepare-pipelines
This will run a pipeline that analyzes the source tree and creates the pipelines necessary to build and validate your pull request. Once the pipeline has been created you can trigger the pipeline using the following comment:
/azp run java - [service] - ci

@g2vinay g2vinay merged commit 3f013f9 into Azure:main Mar 22, 2022
azure-sdk pushed a commit to azure-sdk/azure-sdk-for-java that referenced this pull request Oct 4, 2023
Release machinelearningservices microsoft.machine learning services 2023 08 01 preview (Azure#24762)

* Adds base for updating Microsoft.MachineLearningServices from version preview/2023-06-01-preview to version 2023-08-01-preview

* Updates readme

* Updates API version in new specs and examples

* MFE 2023-08-01-preview mfe.json (Azure#24764)

* add mfe.json

* labeling jobs update descriptions

* add examples for ServerlessEndpoint

* update labelingjob examples

* label job examples remove additional properties

* adding sku to patch

* add identity to patch

* add in operationresult to examples

---------

Co-authored-by: Kayla Ames <[email protected]>

* Adding Resources at sweep level (Azure#25090)

Co-authored-by: Anubha98 <[email protected]>

* HoboVM Preview Features (Azure#24769)

* Update Pricesheet properties for savingsPlan (Azure#24708)

* Update consumption.json

* Update PriceSheet.json

* Update consumption.json

* Update PriceSheet.json

* Update consumption.json

* Update PriceSheet.json

* Update consumption.json

* Update consumption.json

* Update consumption.json

* Update PriceSheet.json

* elasticsan preview refresh microsoft.elasticsan 2022-12-01-preview (Azure#24494)

* Adds base for updating Microsoft.ElasticSan from version preview/2021-11-20-preview to version 2022-12-01-preview

* Updates readme

* Updates API version in new specs and examples

* Added PE related changes

* Added Private Link resource API

* Nit change to correct the path for privateLinkResource API

* Updated Lint errors

* prettier check

* resolved comments & added publicNetworkAccess

* removed publicNetworkAccess & NIT changes

* Updated the list operations

* resolved lintDiff

* proper value for examples and removed value from PECollections

* resolved comments

* spell check resolved

* removing volumeCreateParameter

* [Hub Generated] Publish private branch 'InbalZim-storagemover-Microsoft.StorageMover-RPSaaSMaster-2023-07-01-preview' (Azure#24390)

* add or modify files

* Updated `credentials` models according to PR Azure#24318 + reverted unwanted changes to `readme.md`

* Removed URI format from SMB credentials to support it receiving empty string in order to clean-up their value

* Removed extra hierarchy of `AzureStorageFileShareEndpointBaseProperties`

* improve NR Examples (Azure#24636)

* improve NR Examples

* addressing AutoRun Errors

* Fixing Prettier Issue

* Adding NextLink in Max Examples

* Formatting Azure Resource Id

* Updating Format of azure subscriptionIds

* update code owner (Azure#24754)

* Update readme.python.md (Azure#24758)

* [TypeSpecValidation] Refactor package.json (Azure#24756)

- Improves upon Azure#24745
- Creates eng/tools/package.json for all EngSys tools
- Moves "postinstall" to eng/tools/TypeSpecValidation/package.json
  - Technically incorrect if the package were ever to be published to a registry
  - In practice, the best way I have found so far to use source-based tools in a TypeScript repo

* Add resize APIs

* workflow-bot comments: update breaking changes alternative C to point to alternative B (Azure#24770)

* Add properties for os patching (Azure#24775)

Co-authored-by: Chris Jiang <[email protected]>

* [Communication] Job Router Public Preview v2022-07-18 (Azure#23804)

* Updated JR swagger with maxNumberOfJobs

* job scheduling swagger changes

* Enforce DistributionMode.kind to be read-only

* retry with removing Required tag from abstract property for DistributionMode

* WorkerState stopped showing up in generated swagger

* Add reofferTimeUtc to decline offer API

* Prettier

* Prettier

* rename WorkerState to RouterWorkerState

* more fixes in polymorphic types

* Rename decline offer request

* prettier fix

* add example for scheduled job

* cpsell fix

* fix response schema validation error

* address review feedback item round 1

* SDK review changes

* More API changes

* Update distribution mode required properties

* Renamings

* job matching mode added

* prettier fixes

* fix examples refs

* update ref examples

* update swagger spec

* fix enum mismatch

* change operator to labelOperator in classification policy get and update ops example

* Update specification/communication/data-plane/JobRouter/preview/2022-07-18-preview/communicationservicejobrouter.json

Co-authored-by: Liangying.Wei <[email protected]>

* hide redundant communicationerror and communicationerrorresponse during codegen

* revert commit

* fix typo

* test whether removing kind from request works

* revert removing kind from request

* make kind read-only

* run prettier

* try adding properties to empty classes

---------

Co-authored-by: Rajarshi Sarkar 🦄 <[email protected]>
Co-authored-by: Rajarshi Sarkar <[email protected]>
Co-authored-by: Liangying.Wei <[email protected]>

* [OpenAI] Rename properties for Java (Azure#24596)

* renames

* more renames

* testing on the deploymentOrModelName

* revert for no breaking changes

* [TypeSpec] Upgrade to 0.46.0 (Azure#24776)

* Release quota on stop (Azure#24778)

* Release quota on stop

* add ReleaseQuotaOnStop to computeinstance properties

* update

* rm

* [OpenAI] Update generated autorest (Azure#24788)

- Continuation of Azure#24596

* Dynatrace.Observability merge to public repo (Azure#24101)

* Initial Commit

* Updating version in all files

* Removed version from readme

* removing version which are not available from readme

* Resolving PR comments (Azure#24205)

* Features/users/abhargava/merge to public repo (Azure#24230)

* Resolving PR comments

* Features/users/abhargava/merge to public repo (Azure#24233)

* Resolving PR comments

* Responding to the comment in PR review

* Fixed typo

* Features/users/abhargava/merge to public repo (Azure#24253)

* Resolving PR comments

* Responding to the comment in PR review

* Fixed typo

* Added a suppression for Avacado issue

* Adding the required field

* Updating suppression reason

* Updating path for suppression

* Adding suppression for deprecated API

* suppression

* suppress

* Update specification/dynatrace/resource-manager/readme.md

Co-authored-by: Yuchao Yan <[email protected]>

* updating path in suppression

* updating suppression

* updating suppression

* Update specification/dynatrace/resource-manager/readme.md: WIP: fixing suppressions

Iteratively work on fixing suppressions

* Update readme.md: update suppressions

* Update readme.md: update suppressions

* Update readme.md: remove Avocado suppressions

---------

Co-authored-by: kanupriyasinghh <[email protected]>
Co-authored-by: Yuchao Yan <[email protected]>
Co-authored-by: Konrad Jamrozik <[email protected]>

* Double Header Fix (Azure#24760)

* [OpenAI.Inference] Adding content filter support (Azure#24652)

* first-pass adding annotation

* FilterResult definitions

* move Filter definitions to common

* cleaning up naming and imports

* adding version headers

* added descriptions to objects

* correct self_harm JSON projection

* Correct to "prompt_annotations"

* "prompt_annotations" for chat, too

* Update specification/cognitiveservices/OpenAI.Inference/models/completions.common.tsp

Co-authored-by: Shawn Fang <[email protected]>

* Add per-choice content_filter_results to /completions

* tsp compile after merge

* recompile with 0.46.0 after today's update

---------

Co-authored-by: Travis Wilson <[email protected]>
Co-authored-by: Shawn Fang <[email protected]>

* Update OpenAI PR assignment (Azure#24794)

* Update OpenAI PR assignment

* Update pull_request_assignment.yml

* [typespec-ci] Automatically detect spec folders to validate (Azure#24629)

* [fist] mgmt, update service name (Azure#24781)

* [typespec-pr] Add pipeline (Azure#24802)

* [typespec-ci] Remove path wildcard (Azure#24804)

* Update engsys CODEOWNERS (Azure#24808)

* Align optionality of content filter categories (Azure#24819)

* Add spec ARM PR review workflow diagram to ARM PR description template. (Azure#24823)

* Add spec PR review workflow diagram to ARM PR description template.

* Update control_plane_template.md: fix link to diagram (Azure#24828)

* Added generator config for OpenAI `Functions` for the Java SDK (Azure#24765)

* Remove repeatability support for token revocation endpoint (Azure#24589)

* Update clientTYpe (Azure#24503)

* [Hub Generated] Publish private branch 'dev-maintenance-Microsoft.Maintenance-2023-04-01' (Azure#24048)

* Revert "Revert "[Hub Generated] Publish private branch 'dev-maintenance-Microsoft.Maintenance-2023-04-01' (Azure#23832)" (Azure#23923)"

This reverts commit e6c0e188a74d051a3a44728ce058e85447553392.

* Removed 202 and Async supression.

* Update documentation.

* Updated documentation

---------

Co-authored-by: Kalpesh Chavan <[email protected]>

* [package-lock.json] Add names to local dependencies (Azure#24846)

- Command "npm update" removes these names, but "npm install" adds them
- It's probably better to keep the names, since users are more likely to run "npm install"

* Update ci-fix.md: add API Doc Preview TSG (Azure#24843)

* Update ci-fix.md: add API Doc Preview TSG

* Remove unsupported SQL Pool Rename/move operation (Azure#24517)

* Add enableOSPatching and address comments (Azure#24849)

* Updated text for property isSoftDeleteFeatureStateEditable (Azure#24713)

* Updating folder structure for Microsoft.Migrate (Azure#24550)

* folders restructure and update readme

* revert removed offazure in readme

* Removing readonly flag from top level object (Azure#24848)

* [ACS JobRouter] Add misssing descriptions (Azure#24842)

* Updated JR swagger with maxNumberOfJobs

* sync with upstream main

* add descriptions

* prettier fix

* add more missing descriptions

* add more description fixes

---------

Co-authored-by: williamzhao87 <[email protected]>

* a couple of representation tweaks for codegen (Azure#24847)

* Removed pricing not relevant to CI

* Added Resizing state

* Change targetVMSize name

* Revert "Merge branch 'main' of https://github.com/Azure/azure-rest-api-specs into teom-hobovm08-01-preview"

This reverts commit 47adff637482e53956bcde7a4331f0638a1ddfbc, reversing
changes made to 4b9165da959a09109ecf96e7e70a940ec3989732.

* Resolve comments for os patching (Azure#24872)

* Add properties for os patching

* resolve comments

* remove autorestartafterupdates

* update description

* rename property

---------

Co-authored-by: Chris Jiang <[email protected]>

* Rename osPatchingStatus to upper case (Azure#24954)

* Add properties for os patching

* resolve comments

* remove autorestartafterupdates

* update description

* rename property

* Change property osPatchingStatus to upper case

* rename osPatchingStatus to upper case

---------

Co-authored-by: Chris Jiang <[email protected]>

* Fix some minor issues

* Fix comments

* Minor changes

* Fix more builds

* 200 and nextLink

* Remove nextlink

* Try 204 not 200

* Change GET call to POST

* Remove 204

* Add 200 response

* Change example state

* Add enum for patch status (Azure#25208)

* Add properties for os patching

* resolve comments

* remove autorestartafterupdates

* update description

* rename property

* Change property osPatchingStatus to upper case

* rename osPatchingStatus to upper case

* add enum for patch status

* add enum for patch status

---------

Co-authored-by: Chris Jiang <[email protected]>

* LRO does not return 200

* Format readme

* Format readme

* Format readme

* Fix readme

* Rename enum for patchStatus (Azure#25257)

* Add properties for os patching

* resolve comments

* remove autorestartafterupdates

* update description

* rename property

* Change property osPatchingStatus to upper case

* rename osPatchingStatus to upper case

* add enum for patch status

* add enum for patch status

* rename enum's name

* rename enum

---------

Co-authored-by: Chris Jiang <[email protected]>

---------

Co-authored-by: Jayati Tehri <[email protected]>
Co-authored-by: PreetPRG <[email protected]>
Co-authored-by: InbalZim <[email protected]>
Co-authored-by: VIPRAY JAIN <[email protected]>
Co-authored-by: Dapeng Zhang <[email protected]>
Co-authored-by: Yuchao Yan <[email protected]>
Co-authored-by: Mike Harder <[email protected]>
Co-authored-by: Teo Magnino Chaban <[email protected]>
Co-authored-by: Konrad Jamrozik <[email protected]>
Co-authored-by: Chris Jiang <[email protected]>
Co-authored-by: Chris Jiang <[email protected]>
Co-authored-by: williamzhao87 <[email protected]>
Co-authored-by: Rajarshi Sarkar 🦄 <[email protected]>
Co-authored-by: Rajarshi Sarkar <[email protected]>
Co-authored-by: Liangying.Wei <[email protected]>
Co-authored-by: Shawn Fang <[email protected]>
Co-authored-by: mqwang <[email protected]>
Co-authored-by: abhargavams <[email protected]>
Co-authored-by: kanupriyasinghh <[email protected]>
Co-authored-by: JainRah <[email protected]>
Co-authored-by: Brandon Miller <[email protected]>
Co-authored-by: Travis Wilson <[email protected]>
Co-authored-by: Laurent Mazuel <[email protected]>
Co-authored-by: Albert Cheng <[email protected]>
Co-authored-by: Weidong Xu <[email protected]>
Co-authored-by: Wes Haggard <[email protected]>
Co-authored-by: Jose Alvarez <[email protected]>
Co-authored-by: Maxim Rytych <[email protected]>
Co-authored-by: Daniel Hu <[email protected]>
Co-authored-by: Kalpesh Chavan <[email protected]>
Co-authored-by: Kalpesh Chavan <[email protected]>
Co-authored-by: kashyap-kunal <[email protected]>
Co-authored-by: ishanjindalms <[email protected]>
Co-authored-by: athulvijayanmsft <[email protected]>
Co-authored-by: Thomas Ricci <[email protected]>
Co-authored-by: williamzhao87 <[email protected]>

* Adding new query param on list job api (Azure#25375)

Co-authored-by: Shail Paragbhai Shah <[email protected]>

* Remove list materialization jobs api (Azure#25372)

* Update mfe.json

* prettify swagger

* Remove example

* Update readme.md

* Update readme.md

* Add backfill api contract changes

* Fixing json

* Add property bag

* Modifying example

---------

Co-authored-by: Shail Paragbhai Shah <[email protected]>

* sweep over pipeline component (Azure#25616)

* Add SSO and RootAccess (Azure#25754)

Co-authored-by: Teo Magnino Chaban <[email protected]>

* add EnvironmentVariables in sparkjob (Azure#25814)

* Add model monitor 2023-08-01-preview changes (Azure#25785)

* Add model monitor 2023-08-01-preview changes

* update

* run prettier

* update comments

* add

* Add WS Vnet properties for Serverless GA (Azure#25780)

* add serverless ws vnet properties

* add to workspace update properties

* update format

* update description

---------

Co-authored-by: Aaheli Chattopadhyay <[email protected]>

* Resource to ProxyResource for preview version (Azure#25906)

Co-authored-by: Shail Paragbhai Shah <[email protected]>

* August-2023-preview fix circular reference (Azure#25870)

* TriggerType->ComputeTriggerType

* RecurrenceFrequency->ComputeRecurrenceFrequency

* RecurrenceSchedule->ComputeRecurrenceSchedule

---------

Co-authored-by: Kayla Ames <[email protected]>

* Update ServerlessEndpoints design (Azure#25593)

* first draft redesign of ServerlessEndpoints

* add getStatus.json

* update examples

* add LRO location header to example

* add CapacityReservationGroups API

* spellcheck fix

* example fix

* capacity reservation groups should be under resource groups

* fix examples

* inferenceUri -> uri

* CapacityReservation -> ServerlessEndpointCapacityReservation

* update description

* remove ExcessCapacityReservation concept

* remove parentGroupId concept

* rename /status api to /getStatus

* label metrics as readOnly

* lintdiff GET fix

* more fixes

* add listBySubscription

* add/fix examples

* add suppression for lint error (Azure#25941)

* minor fixes (Azure#25940)

* Update 2023-08-01-preview swagger for workspaceRP (Azure#25532)

* Update swagger

* Bug fix

* update

* Update testconnection and example

* Fix format

* update custom words and example

* Update missing custom words

* reflect comment remove LRO opetions

* make metadata a string dictionary

* reflect comment

* Revert "reflect comment"

This reverts commit 9b85252250fd442dd891b27509f5d2e315d8aa2a.

* Update with managedNetwork change included

* Update managed network schema

* revert metadata change

* remove the RBAC properties related with shared connection

* Update readme

* Update suppression  reason

* Updated preview API to add Kernel Config (Azure#25361)

* Updated preview API to add Kernel Config and LRO Headers

* Fixing formatting issue

* Updated as per comments

* Updated as per comments

* Reverting changes

* Reverted header changes

* Reverting change

* Reverting example

* Added version parameter and removed addition properties as per guidelines

---------

Co-authored-by: Srivatsa Sinha <[email protected]>

* Thtodoro/serverless endpoints 2023 08 01 preview fix (Azure#25993)

* Capacity Reservation Groups is not an async resource, and thus does not have 202 as a response type

* PATCH response must have same resource as GET/PUT APIs

* add missing description

* Adding Inference Pools, Groups and Endpoints recreated for Azure/azure-rest-api-specs#25783 for rebased issue (Azure#25819)

* Adding Inference Pools, Groups and Endpoints

* PR comments

* remove EventLog and change data

* remove EventLog and change data

* moving endpoint as sibling to group

* Moving back to the tracked resources

* force change

* fix the missing field

---------

Co-authored-by: Raushan Kumar <[email protected]>

* Make suppression more specific (Azure#26031)

Co-authored-by: Teo Magnino Chaban <[email protected]>

* add serverless compute settings (Azure#26007)

Co-authored-by: Aaheli Chattopadhyay <[email protected]>

---------

Co-authored-by: Kayla Ames <[email protected]>
Co-authored-by: Anubha Jain <[email protected]>
Co-authored-by: Anubha98 <[email protected]>
Co-authored-by: teochaban <[email protected]>
Co-authored-by: Jayati Tehri <[email protected]>
Co-authored-by: PreetPRG <[email protected]>
Co-authored-by: InbalZim <[email protected]>
Co-authored-by: VIPRAY JAIN <[email protected]>
Co-authored-by: Dapeng Zhang <[email protected]>
Co-authored-by: Yuchao Yan <[email protected]>
Co-authored-by: Mike Harder <[email protected]>
Co-authored-by: Teo Magnino Chaban <[email protected]>
Co-authored-by: Konrad Jamrozik <[email protected]>
Co-authored-by: Chris Jiang <[email protected]>
Co-authored-by: Chris Jiang <[email protected]>
Co-authored-by: williamzhao87 <[email protected]>
Co-authored-by: Rajarshi Sarkar 🦄 <[email protected]>
Co-authored-by: Rajarshi Sarkar <[email protected]>
Co-authored-by: Liangying.Wei <[email protected]>
Co-authored-by: Shawn Fang <[email protected]>
Co-authored-by: mqwang <[email protected]>
Co-authored-by: abhargavams <[email protected]>
Co-authored-by: kanupriyasinghh <[email protected]>
Co-authored-by: JainRah <[email protected]>
Co-authored-by: Brandon Miller <[email protected]>
Co-authored-by: Travis Wilson <[email protected]>
Co-authored-by: Laurent Mazuel <[email protected]>
Co-authored-by: Albert Cheng <[email protected]>
Co-authored-by: Weidong Xu <[email protected]>
Co-authored-by: Wes Haggard <[email protected]>
Co-authored-by: Jose Alvarez <[email protected]>
Co-authored-by: Maxim Rytych <[email protected]>
Co-authored-by: Daniel Hu <[email protected]>
Co-authored-by: Kalpesh Chavan <[email protected]>
Co-authored-by: Kalpesh Chavan <[email protected]>
Co-authored-by: kashyap-kunal <[email protected]>
Co-authored-by: ishanjindalms <[email protected]>
Co-authored-by: athulvijayanmsft <[email protected]>
Co-authored-by: Thomas Ricci <[email protected]>
Co-authored-by: williamzhao87 <[email protected]>
Co-authored-by: shail2208 <[email protected]>
Co-authored-by: Shail Paragbhai Shah <[email protected]>
Co-authored-by: ghsutirtha <[email protected]>
Co-authored-by: ycheng35xo <[email protected]>
Co-authored-by: Mathieu St-Louis <[email protected]>
Co-authored-by: ac923 <[email protected]>
Co-authored-by: Aaheli Chattopadhyay <[email protected]>
Co-authored-by: Teddy Todorov <[email protected]>
Co-authored-by: ZhidaLiu <[email protected]>
Co-authored-by: srivatsasinha <[email protected]>
Co-authored-by: Srivatsa Sinha <[email protected]>
Co-authored-by: Raushan Kumar <[email protected]>
Co-authored-by: Raushan Kumar <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants