Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Event Hubs Client] Track Two: Third Preview (Event Processor Load Balancing) #7414

Merged
merged 35 commits into from
Sep 4, 2019
Merged

[Event Hubs Client] Track Two: Third Preview (Event Processor Load Balancing) #7414

merged 35 commits into from
Sep 4, 2019

Conversation

kinelski
Copy link
Member

@kinelski kinelski commented Aug 28, 2019 •
edited
Loading

These changes provide automatic load balancing for Event Processors.

Important points:

  • These changes also include some updates in PartitionPump so we can have a better exception handling. The expected behavior is still under discussion.
  • Checkpoints are still not supported. This will be included in a future PR for the Third Preview.
  • Solves [EventHubs] Make Event Processor Live Tests more deterministic #7375, but tests still need polishing.

Related issue: #7377

@kinelski kinelski added Event Hubs Client This issue points to a problem in the data-plane of the library. labels Aug 28, 2019
@kinelski kinelski self-assigned this Aug 28, 2019
kinelski added 17 commits August 28, 2019 05:23
@kinelski
Support all close reason scenarios
76108cb
@kinelski
WaitTillBalancedStatus helper added in EventProcessorManager.
40bd062
@kinelski
Live Tests make use of the wait load balance help method
08787fc
@kinelski
Revert "Live Tests make use of the wait load balance help method"
b6434bc 
This reverts commit 08787fc.
@kinelski
Update in manager to wait for stabilization instead of waiting for ba…
bc951cd 
…lancing
@kinelski
Update in tests to reflect stabilization changes in manager
cb169af
@kinelski
Make loop and expiration times mockable
f00ca13
@kinelski
Setup mock Event Processor in test base so it doesn't take too long u…
2153575 
…ntil stabilization
@kinelski
A lot of docs added to Event Processor
10d480c
@kinelski
Docs in pump
73e7790
@kinelski
Final comments added in Event Hub Manager. A few tests are flaky
3bd37f5
@kinelski
Time spans extended and stabilization time fixed to 1 minute. No more…
919e426 
… flaky test
@kinelski
Ignore failing test
f8dabd5
@kinelski
Updated comment
ce13126
@kinelski
Update code to support pump start failure
5ae69c3
@kinelski
Added stopwatch to manage the cycle time
27e3fba
@kinelski
Merge remote-tracking branch 'upstream/master' into balancing
ea6d28d
@kinelski kinelski marked this pull request as ready for review August 30, 2019 10:02
jsquire
jsquire approved these changes Aug 30, 2019
View reviewed changes
Copy link
Member

@jsquire jsquire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, overall. I left a few thoughts, but there's nothing that I'd consider blocking. Approving now, but would appreciate the chance to take another look before you merge.

sdk/eventhub/Azure.Messaging.EventHubs/src/Processor/EventProcessor.cs Outdated Show resolved Hide resolved
sdk/eventhub/Azure.Messaging.EventHubs/src/Processor/EventProcessor.cs Outdated Show resolved Hide resolved
sdk/eventhub/Azure.Messaging.EventHubs/src/Processor/EventProcessor.cs Show resolved Hide resolved
sdk/eventhub/Azure.Messaging.EventHubs/src/Processor/EventProcessor.cs Outdated Show resolved Hide resolved
sdk/eventhub/Azure.Messaging.EventHubs/src/Processor/EventProcessor.cs Outdated Show resolved Hide resolved
sdk/eventhub/Azure.Messaging.EventHubs/src/Processor/EventProcessor.cs Outdated Show resolved Hide resolved
sdk/eventhub/Azure.Messaging.EventHubs/src/Processor/PartitionPump.cs Outdated Show resolved Hide resolved
sdk/eventhub/Azure.Messaging.EventHubs/src/Processor/PartitionPump.cs Outdated Show resolved Hide resolved
sdk/eventhub/Azure.Messaging.EventHubs/tests/Infrastructure/EventProcessorManager.cs Outdated Show resolved Hide resolved
sdk/eventhub/Azure.Messaging.EventHubs/tests/Infrastructure/EventProcessorManager.cs Show resolved Hide resolved
@kinelski kinelski merged commit eea0295 into Azure:master Sep 4, 2019
@kinelski kinelski deleted the balancing branch September 4, 2019 17:42
This was referenced Sep 4, 2019
Closed
Closed
JoshLove-msft pushed a commit to JoshLove-msft/azure-sdk-for-net that referenced this pull request Sep 10, 2019
@kinelski @JoshLove-msft
[Event Hubs Client] Track Two: Third Preview (Event Processor Load Ba…
7df4f33 
…lancing) (Azure#7414)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jsquire jsquire jsquire approved these changes

@serkantkaraca serkantkaraca Awaiting requested review from serkantkaraca

Assignees

@kinelski kinelski

Labels
Client This issue points to a problem in the data-plane of the library. Event Hubs
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@kinelski @jsquire