-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Propagate headers/warnings/stats from quantile downstreams #13881
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see now what I did wrong. The stats, warnings etc are joined in Downstream
here but my accumulator is returning only one result.
I don't know how adventurous your are but the current state has a little code smell. I think the join of the stats should be moved out of Downstream
. Furthermore, if you follow the code for the streams you'll see that we run iter.NewSortEntryIterator
on the results
. However, NewStreamAccumulator(params)
returns only one result. So the sort iterator is not really required.
I suggest to change the Accumulator.Result
return type to logqlmodel.Result
instead of a slice. And then move the logic of the merge for the BufferedAccumulator
. I'd do this as a follow up. The big work is that this would require moving the logic of NewMergeLastOverTimeStepEvaluator
, NewMergeFirstOverTimeStepEvaluator
and NewConcatStepEvaluator
into new Accumulators.
Stepping back a little, my refactoring from #11863 was intended to go down that path.
Let me explain why this makes sense:
- It will enable us to move the joining of stats into a central logic so it's not lost on further refactoring.
- Accumulate and drop samples as they arrive. E.g. last_over_time does not have to buffer all the results anymore. Same with
sum by
in the future. This should reduce the memory footprint a little. - This will allow us to define special
topk
orsum
by accumulators in the future.. - The previous point will then enable us to print and investigate a query plan without running the step evaluators.
TLDR:
- We should land this and verify that it works.
- Replace
BufferedAccumulator
withConcatAccumulator
,LastOverTimeAccumulator
andFirstOverTimeAccumulator
.
What this PR does / why we need it:
Passes headers from Downstreams back to the caller
While debugging, I added a Span for when the
CacheGenHeader
is added, and you can see the span is triggered, and the header injected, in the Downstream call but never passed all the way up to the ResultsCache.Do because of this bug:Trace from my local instance showing that 3/4 intervals now hit the cache: