-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow out-of-band, async upload functionality #80
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I few questions and a couple nitpicks. Looking great overall. I'll try and think of other tests we might want.
963399c
to
9a8701d
Compare
Curious if you had considered the Rack Events APIto defer the upload work to after the request was complete? There's some folks at Github exploring this approach as an alternative/successor to the OpenTelemetry rack middleware open-telemetry/opentelemetry-ruby-contrib#342 so that we can defer more work to outside of the request context. |
I did not, but: This has impact on capacity, at least for |
Rebased and squashed the commit. I will 🎩 this against core and make sure nothing breaks. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this 🙏
60d40b3
to
c52e343
Compare
Squashed and rebased. Also, I saw a test failure on my last push which I tracked down to a race condition between the assertion and the background thread. I added this method to stub the thread creation in tests:
|
c52e343
to
e0cda0c
Compare
Rebased and resolved the merge conflicts. @casperisfine -- Can you please give this a read, it is somewhat relevant to the work you have done in #87. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Implementation looks fine to me. I particularly appreciate the use of a SizedQueue to avoid piling work faster it's dequeued.
I'm not 100% sure I understand how it's meant to be used though. I see how it could be useful to reduce the impact of profiles triggered programaticaly, but I don't get the need for the X-Profile-Async
header.
Discussed with Jean in in a DM, about how we intend to use this -- it is for programatically triggered profiles. I will keep Will squash the commits and push. |
273470c
to
8cc71ab
Compare
TL;DR; Adds a new query string param
async
. When specified, profile uploads happen in a background thread.One use case for this is when we want to profile a certain % of traffic without end users explicitly requesting it. In such situations uploading profiles inline does not scale.
From the
README
:If the
async
parameter is specified we add to a queue/buffer -- which we process everyupload_queue_interval_secs
. The queue is SizedQueue.CC @jasonhl