[Event hubs] Separate event hubs tests into public and internal and enable min-max tests #12798

KarishmaGhiya · 2020-12-07T23:06:32Z

For detailed information on what we are doing in mim-max testing read this blog - https://devblogs.microsoft.com/azure-sdk/testing-semver-dependency-ranges/
Segregate the test files into internal and public folders. Code separation done where required to increase the coverage for min-max tests.
Enable Min max testing and solve any complications with it
Ensure the number of tests run in min-max = no of tests in the public folder
Min Max tests pass - https://dev.azure.com/azure-sdk/internal/_build/results?buildId=698908&view=logs&j=9c8a71a2-d309-50a1-ebfc-15dfea2fe8ca
After merging - https://dev.azure.com/azure-sdk/internal/_build/results?buildId=699877&view=logs&j=38753cb9-b155-5978-2d3f-2d89d306e985

KarishmaGhiya · 2020-12-09T20:27:53Z

@richardpark-msft @chradek Live min-max tests passed woohoo!! https://dev.azure.com/azure-sdk/internal/_build/results?buildId=648059&view=logs&j=458fae0f-1dd6-5242-ab00-2d189819d1f1

KarishmaGhiya · 2020-12-09T20:54:35Z

/azp run js - eventhubs-client - tests

azure-pipelines · 2020-12-09T20:54:46Z

Azure Pipelines successfully started running 1 pipeline(s).

richardpark-msft · 2020-12-09T21:00:29Z

sdk/eventhub/event-hubs/package.json

@@ -58,7 +58,7 @@
    "extract-api": "tsc -p . && api-extractor run --local",
    "format": "prettier --write --config ../../../.prettierrc.json --ignore-path ../../../.prettierignore \"src/**/*.ts\" \"test/**/*.ts\" \"*.{js,json}\"",
    "integration-test:browser": "karma start --single-run",
-    "integration-test:node": "nyc mocha -r esm --require source-map-support/register --reporter ../../../common/tools/mocha-multi-reporter.js --timeout 1200000 --full-trace dist-esm/test/*.spec.js dist-esm/test/**/*.spec.js",
+    "integration-test:node": "nyc mocha -r esm --require source-map-support/register --reporter ../../../common/tools/mocha-multi-reporter.js --timeout 1200000 --full-trace dist-esm/test/public/*.spec.js dist-esm/test/internal/*.spec.js dist-esm/test/public/**/*.spec.js dist-esm/test/internal/**/*.spec.js",


@willmtemple - tell me that someday we'll make devtool compose this line (or just invoke devtool directly).

For metrics-advisor we are using devtool

@richardpark-msft What about something like dev-tool run-default integration-test:node that would be able to run a standard default configuration?

Ultimately my hope is that this can be supported in Rush, since it seems hacky to do such a thing in dev-tool.

chradek

Thanks for working on this @KarishmaGhiya! Left some minor comments.

sdk/eventhub/event-hubs/package.json

sdk/eventhub/event-hubs/test/public/eventHubConsumerClient.spec.ts

sdk/eventhub/event-hubs/test/public/receiver.spec.ts

sdk/eventhub/event-hubs/test/public/utils/testInMemoryCheckpointStore.ts

sdk/eventhub/event-hubs/test/public/utils/testUtils.ts

sdk/eventhub/event-hubs/test/public/utils/testInMemoryCheckpointStore.ts

richardpark-msft

Looks good and thanks for doing this. @chradek seems to have the most amount of feedback so I'll defer to him on giving a final approval.

chradek · 2020-12-10T22:52:11Z

/azp run js - eventhubs-client - tests

azure-pipelines · 2020-12-10T22:52:23Z

Azure Pipelines successfully started running 1 pipeline(s).

chradek

Looks good to me, thanks for doing this work! I triggered a live test run because I'm superstitious and need to see those green check marks, but once that finishes (and passes) ship it!

KarishmaGhiya · 2020-12-11T01:57:29Z

/azp run js - eventhubs-client - tests

azure-pipelines · 2020-12-11T01:57:39Z

Azure Pipelines successfully started running 1 pipeline(s).

ghost · 2020-12-11T01:57:56Z

Hello @KarishmaGhiya!

Because this pull request has the auto-merge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (`@msftbot`) and give me an instruction to get started! Learn more here.

check-enforcer · 2020-12-14T22:10:01Z

This pull request is protected by Check Enforcer.

What is Check Enforcer?

Check Enforcer helps ensure all pull requests are covered by at least one check-run (typically an Azure Pipeline). When all check-runs associated with this pull request pass then Check Enforcer itself will pass.

Why am I getting this message?

You are getting this message because Check Enforcer did not detect any check-runs being associated with this pull request within five minutes. This may indicate that your pull request is not covered by any pipelines and so Check Enforcer is correctly blocking the pull request being merged.

What should I do now?

If the check-enforcer check-run is not passing and all other check-runs associated with this PR are passing (excluding license-cla) then you could try telling Check Enforcer to evaluate your pull request again. You can do this by adding a comment to this pull request as follows:
/check-enforcer evaluate
Typically evaulation only takes a few seconds. If you know that your pull request is not covered by a pipeline and this is expected you can override Check Enforcer using the following command:
/check-enforcer override
Note that using the override command triggers alerts so that follow-up investigations can occur (PRs still need to be approved as normal).

What if I am onboarding a new service?

Often, new services do not have validation pipelines associated with them, in order to bootstrap pipelines for a new service, you can issue the following command as a pull request comment:
/azp run prepare-pipelines
This will run a pipeline that analyzes the source tree and creates the pipelines necessary to build and validate your pull request. Once the pipeline has been created you can trigger the pipeline using the following comment:
/azp run js - [service] - ci

KarishmaGhiya · 2021-01-22T22:01:59Z

Max tests pass, min fails - https://dev.azure.com/azure-sdk/internal/_build/results?buildId=698908&view=logs&j=458fae0f-1dd6-5242-ab00-2d189819d1f1
The same test passes in re-run Attempt 2 - https://dev.azure.com/azure-sdk/internal/_build/results?buildId=698908&view=logs&j=9c8a71a2-d309-50a1-ebfc-15dfea2fe8ca

KarishmaGhiya requested review from chradek, danieljurek, HarshaNalluru, ramya-rao-a and richardpark-msft as code owners December 7, 2020 23:06

ghost added the Event Hubs label Dec 7, 2020

KarishmaGhiya linked an issue Dec 7, 2020 that may be closed by this pull request

[eventhub] Separation of internal and public tests into separate folders for min-max testing #9457

Closed

1 task

KarishmaGhiya requested review from mikeharder, mitchdenny, praveenkuttappan and weshaggard as code owners December 8, 2020 22:15

richardpark-msft reviewed Dec 9, 2020

View reviewed changes

chradek reviewed Dec 9, 2020

View reviewed changes

richardpark-msft reviewed Dec 9, 2020

View reviewed changes

sdk/eventhub/event-hubs/test/public/utils/testInMemoryCheckpointStore.ts Show resolved Hide resolved

richardpark-msft reviewed Dec 9, 2020

View reviewed changes

chradek approved these changes Dec 10, 2020

View reviewed changes

KarishmaGhiya added the auto-merge label Dec 11, 2020

KarishmaGhiya removed the auto-merge label Dec 11, 2020

KarishmaGhiya force-pushed the separate-event-hubs branch from 8a216d0 to e9b709f Compare December 15, 2020 00:05

KarishmaGhiya mentioned this pull request Dec 15, 2020

[eventhub] Separation of internal and public tests into separate folders for min-max testing #9457

Closed

1 task

ramya-rao-a requested a review from chradek December 16, 2020 23:34

KarishmaGhiya and others added 9 commits January 21, 2021 12:14

added comments for testInMemoryCheckpointStore

76631be

combined the receiver.spec.ts tests with EvenetHubConsumerClient-tests

91e8b5b

moved receiver tests into eventHubConsumerClient tests

7010177

package.json update

54d78f6

debug test code

29e1de3

add/ remove logging

f3e9766

try reducing the error size

311a6fc

timeout

0f70a63

add logging to test!

b1fc046

KarishmaGhiya force-pushed the separate-event-hubs branch from 5ab8070 to b1fc046 Compare January 21, 2021 20:19

KarishmaGhiya added 12 commits January 21, 2021 13:15

path made stronglly string type

7cb38c7

env import update

e7a17d1

debug that test

f26a39d

debug that test

a613386

added more debug statements to the test

b17f6ca

added more debug statements to the test

362326e

added more debug statements to the test

80b4d87

added more debug statements to the test

1bd6970

skip the failing test

617d0e3

added more debug statements to the test

6f96eb0

debug tests

43dacc4

eventhub processor timeout to 1 sec

28e2a57

KarishmaGhiya merged commit 2fa54f8 into Azure:master Jan 23, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Event hubs] Separate event hubs tests into public and internal and enable min-max tests #12798

[Event hubs] Separate event hubs tests into public and internal and enable min-max tests #12798

KarishmaGhiya commented Dec 7, 2020 •

edited

Loading

KarishmaGhiya commented Dec 9, 2020

KarishmaGhiya commented Dec 9, 2020

azure-pipelines bot commented Dec 9, 2020

richardpark-msft Dec 9, 2020

KarishmaGhiya Dec 9, 2020

willmtemple Dec 11, 2020

willmtemple Dec 11, 2020

chradek left a comment

richardpark-msft left a comment

chradek commented Dec 10, 2020

azure-pipelines bot commented Dec 10, 2020

chradek left a comment

KarishmaGhiya commented Dec 11, 2020

azure-pipelines bot commented Dec 11, 2020

ghost commented Dec 11, 2020

check-enforcer bot commented Dec 14, 2020

KarishmaGhiya commented Jan 22, 2021 •

edited

Loading

[Event hubs] Separate event hubs tests into public and internal and enable min-max tests #12798

[Event hubs] Separate event hubs tests into public and internal and enable min-max tests #12798

Conversation

KarishmaGhiya commented Dec 7, 2020 • edited Loading

KarishmaGhiya commented Dec 9, 2020

KarishmaGhiya commented Dec 9, 2020

azure-pipelines bot commented Dec 9, 2020

richardpark-msft Dec 9, 2020

Choose a reason for hiding this comment

KarishmaGhiya Dec 9, 2020

Choose a reason for hiding this comment

willmtemple Dec 11, 2020

Choose a reason for hiding this comment

willmtemple Dec 11, 2020

Choose a reason for hiding this comment

chradek left a comment

Choose a reason for hiding this comment

richardpark-msft left a comment

Choose a reason for hiding this comment

chradek commented Dec 10, 2020

azure-pipelines bot commented Dec 10, 2020

chradek left a comment

Choose a reason for hiding this comment

KarishmaGhiya commented Dec 11, 2020

azure-pipelines bot commented Dec 11, 2020

ghost commented Dec 11, 2020

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

check-enforcer bot commented Dec 14, 2020

What is Check Enforcer?

Why am I getting this message?

What should I do now?

What if I am onboarding a new service?

KarishmaGhiya commented Jan 22, 2021 • edited Loading

KarishmaGhiya commented Dec 7, 2020 •

edited

Loading

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (`@msftbot`) and give me an instruction to get started! Learn more here.

KarishmaGhiya commented Jan 22, 2021 •

edited

Loading