-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Loki: Adds an interval
paramater to query_range queries allowing a sampling of events to be returned based on the provided interval
#1965
Conversation
cmd/logcli/main.go
Outdated
@@ -229,7 +229,8 @@ func newQuery(instant bool, cmd *kingpin.CmdClause) *query.Query { | |||
cmd.Flag("since", "Lookback window.").Default("1h").DurationVar(&since) | |||
cmd.Flag("from", "Start looking for logs at this absolute time (inclusive)").StringVar(&from) | |||
cmd.Flag("to", "Stop looking for logs at this absolute time (exclusive)").StringVar(&to) | |||
cmd.Flag("step", "Query resolution step width").DurationVar(&q.Step) | |||
cmd.Flag("step", "Query resolution step width, for metric queries. Evaluate the query at the specified step over the time range.").DurationVar(&q.Step) | |||
cmd.Flag("interval", "Query interval, for log queries. Return entries at the specified interval, ignoring those between.").DurationVar(&q.Interval) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't want to bikeshed here but I'm worried the difference between step and interval isn't totally clear and might be hard to remember. Would log_step
or entry_step
be easier here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That would be more confusing to me personally.
I would prefer to work on making the description clearer if we can do so.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I could be wrong, but I'm concerned about having naming conventions that requires you to pull up the docs to differentiate between the two. I'm okay with this as-is, but I think it's something to think about.
Codecov Report
@@ Coverage Diff @@
## master #1965 +/- ##
==========================================
- Coverage 64.35% 64.31% -0.04%
==========================================
Files 129 129
Lines 9950 9977 +27
==========================================
+ Hits 6403 6417 +14
- Misses 3065 3072 +7
- Partials 482 488 +6
|
pkg/loghttp/params.go
Outdated
if d, err := strconv.ParseFloat(value, 64); err == nil { | ||
ts := d * float64(time.Second) | ||
if ts > float64(math.MaxInt64) || ts < float64(math.MinInt64) { | ||
return 0, errors.Errorf("cannot parse %q to a valid duration. It overflows int64", value) | ||
} | ||
return time.Duration(ts), nil | ||
} | ||
// Or parse as a duration | ||
if d, err := model.ParseDuration(value); err == nil { | ||
return time.Duration(d), nil | ||
} | ||
return 0, errors.Errorf("cannot parse %q to a valid duration", value) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This whole thing could be a function, parseDuration and reused in step and interval.
pkg/logql/engine.go
Outdated
for ; respSize < size && i.Next(); respSize++ { | ||
// lastEntry should be a really old time so that the first comparison is always true, we use a negative | ||
// value here because many unit tests start at time.Unix(0,0) | ||
lastEntry := time.Unix(-100, 0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this could be an external const.
var minTime =...
(i.Entry().Timestamp.Equal(lastEntry.Add(interval)) || i.Entry().Timestamp.After(lastEntry.Add(interval))) | ||
backwardShouldOutput := dir == logproto.BACKWARD && | ||
(i.Entry().Timestamp.Equal(lastEntry.Add(-interval)) || i.Entry().Timestamp.Before(lastEntry.Add(-interval))) | ||
// If step == 0 output every line. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// If step == 0 output every line. | |
// If interval == 0 output every line. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's make this experimental, I'd like to explore a way to have this in the logql language itself but that is may be too far away so I think we should merge this.
…ng of events to be returned based on the provided interval Signed-off-by: Ed Welch <[email protected]>
Signed-off-by: Ed Welch <[email protected]>
Signed-off-by: Ed Welch <[email protected]>
Signed-off-by: Ed Welch <[email protected]>
Signed-off-by: Ed Welch <[email protected]>
Signed-off-by: Ed Welch <[email protected]>
Signed-off-by: Cyril Tovena <[email protected]>
* Fix a bad rebase between #1970 and #1965. Signed-off-by: Cyril Tovena <[email protected]> * fmt Signed-off-by: Cyril Tovena <[email protected]>
What this PR does / why we need it:
This is a possible solution #1779 which creates a new query parameter for range_queries called interval. It is being marked experimental in the docs as it's very possible we will remove this flag one day in favor of a LogQL based solution.
Signed-off-by: Ed Welch [email protected]
Special notes for your reviewer:
Checklist