Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Outbox Sweeper - Bulk Operations #1764

Closed
preardon opened this issue Oct 18, 2021 · 2 comments
Closed

Outbox Sweeper - Bulk Operations #1764

preardon opened this issue Oct 18, 2021 · 2 comments

Comments

@preardon
Copy link
Member

At them minute when calling Sweep on the outbox the code path is

  • Get a list of the first 100 dispatched messages older than a certain age
  • Past the IDs of those messages into the Clear Function which then does a ForEach loop of
  • Get the Individual Message
  • Send the Individual Message
  • Mark the Individual Message (if post back is not supported)

This means Per Clear operation we are doing up to 201 SQL calls and 100 Bus calls

Considering the Outbox supports Bulk get if we were to implement bulk message send on our Producers and Bulk Mark Dispatched we could get this number down from a potential 2 SQL calls and 1 bus call

I know this isn't a high propriety, and to be honest at the time of starting to write this I thought we already had a Bulk Send so its a little more work that I was thinking.

Also I think it could be good to make the max amount of messages to sweep configurable

@iancooper Thoughts

@iancooper
Copy link
Member

@preardon I think we might want to triage this into v9.1, so that we can get v9.0 out. You are not the first person to ask for a bulk operation, but it has become more of an issue now we have an Outbox, particularly if folks want to use the Sweeper to clear rather than doing it explicitly.

So, great suggestion, but let's do 9.1. I'll add some 9.next and 10 labels so we can begin to triage stuff into releases

@preardon
Copy link
Member Author

Agreed, lets put it on the 9.1 backlog

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants