Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add support for aggregator type in DataDog metric provider #3293

Merged
merged 9 commits into from
Jan 26, 2024
Merged

feat: Add support for aggregator type in DataDog metric provider #3293

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
e56337a
Add ability to select metric aggregator
NaurisSadovskis Jan 8, 2024
d8cf865
Add check and tests
NaurisSadovskis Jan 8, 2024
e82257a
extra codegen
NaurisSadovskis Jan 8, 2024
c7c7238
aggregator: switch to using enum
NaurisSadovskis Jan 10, 2024
5032336
aggregator: add test for validating enum values
NaurisSadovskis Jan 10, 2024
30889fe
aggregator: add test for default value
NaurisSadovskis Jan 10, 2024
7ccca79
docs: add a section on overriding default aggregator
NaurisSadovskis Jan 10, 2024
8c9b882
Update docs/analysis/datadog.md
NaurisSadovskis Jan 19, 2024
68cf420
Merge branch 'master' into nauris/datadog-v2-allow-selecting-aggregator
NaurisSadovskis Jan 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions docs/analysis/datadog.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,30 @@ However, empty query results yielding a `nil` value can be handled using the `de
successCondition: default(result, 0) < 0.05
```

#### Metric aggregation (v2 only)

By default, Datadog analysis run is configured to use `last` metric aggregator when querying Datadog v2 API. This value can be overriden by specifying a new `aggregator` value from a list of supported aggregators (`avg,min,max,sum,last,percentile,mean,l2norm,area`) for the V2 API ([docs](https://docs.datadoghq.com/api/latest/metrics/#query-scalar-data-across-multiple-products)).

For example, using count-based distribution metric (`count:metric{*}.as_count()`) with values `1,9,3,7,5` in a given `interval` will make `last` aggregator return `5`. To return a sum of all values (`25`), set `aggregator: sum` in Datadog provider block and use `moving_rollup()` function to aggregate values in the specified rollup interval. These functions can be combined in a `formula` to perform additional calculations:

```yaml
...<snip>
metrics:
- name: error-percentage
interval: 30s
successCondition: default(result, 0) < 5
failureLimit: 3
provider:
datadog:
apiVersion: v2
interval: 5m
aggregator: sum # override default aggregator
queries:
a: count:requests.errors{service:my-service}.as_count()
b: count:requests{service:my-service}.as_count()
formula: "moving_rollup(a, 300, 'sum') / moving_rollup(b, 300, 'sum') * 100" # percentage of requests with errors
```

#### Templates and Helm

Helm and Argo Rollouts both try to parse things between `{{ ... }}` when rendering templates. If you use Helm to deliver your manifests, you will need to escape `{{ args.whatever }}`. Using the example above, here it is set up for Helm:
Expand Down
45 changes: 45 additions & 0 deletions docs/features/kustomize/rollout_cr_schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,21 @@
},
"datadog": {
"properties": {
"aggregator": {
"default": "last",
"enum": [
"avg",
"min",
"max",
"sum",
"last",
"percentile",
"mean",
"l2norm",
"area"
],
"type": "string"
},
"apiVersion": {
"default": "v1",
"enum": [
Expand Down Expand Up @@ -4861,6 +4876,21 @@
},
"datadog": {
"properties": {
"aggregator": {
"default": "last",
"enum": [
"avg",
"min",
"max",
"sum",
"last",
"percentile",
"mean",
"l2norm",
"area"
],
"type": "string"
},
"apiVersion": {
"default": "v1",
"enum": [
Expand Down Expand Up @@ -9479,6 +9509,21 @@
},
"datadog": {
"properties": {
"aggregator": {
"default": "last",
"enum": [
"avg",
"min",
"max",
"sum",
"last",
"percentile",
"mean",
"l2norm",
"area"
],
"type": "string"
},
"apiVersion": {
"default": "v1",
"enum": [
Expand Down
13 changes: 13 additions & 0 deletions manifests/crds/analysis-run-crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,19 @@ spec:
type: object
datadog:
properties:
aggregator:
default: last
enum:
- avg
- min
- max
- sum
- last
- percentile
- mean
- l2norm
- area
type: string
NaurisSadovskis marked this conversation as resolved.
Show resolved Hide resolved
apiVersion:
default: v1
enum:
Expand Down
13 changes: 13 additions & 0 deletions manifests/crds/analysis-template-crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,19 @@ spec:
type: object
datadog:
properties:
aggregator:
default: last
enum:
- avg
- min
- max
- sum
- last
- percentile
- mean
- l2norm
- area
type: string
apiVersion:
default: v1
enum:
Expand Down
13 changes: 13 additions & 0 deletions manifests/crds/cluster-analysis-template-crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,19 @@ spec:
type: object
datadog:
properties:
aggregator:
default: last
enum:
- avg
- min
- max
- sum
- last
- percentile
- mean
- l2norm
- area
type: string
apiVersion:
default: v1
enum:
Expand Down
39 changes: 39 additions & 0 deletions manifests/install.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,19 @@ spec:
type: object
datadog:
properties:
aggregator:
default: last
enum:
- avg
- min
- max
- sum
- last
- percentile
- mean
- l2norm
- area
type: string
apiVersion:
default: v1
enum:
Expand Down Expand Up @@ -3300,6 +3313,19 @@ spec:
type: object
datadog:
properties:
aggregator:
default: last
enum:
- avg
- min
- max
- sum
- last
- percentile
- mean
- l2norm
- area
type: string
apiVersion:
default: v1
enum:
Expand Down Expand Up @@ -6307,6 +6333,19 @@ spec:
type: object
datadog:
properties:
aggregator:
default: last
enum:
- avg
- min
- max
- sum
- last
- percentile
- mean
- l2norm
- area
type: string
apiVersion:
default: v1
enum:
Expand Down
14 changes: 9 additions & 5 deletions metricproviders/datadog/datadog.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,7 @@ func (p *Provider) createRequest(dd *v1alpha1.DatadogMetric, now int64, interval
dd.Queries = map[string]string{"query": dd.Query}
}

return p.createRequestV2(dd.Queries, dd.Formula, now, interval, url)
return p.createRequestV2(dd.Queries, dd.Formula, now, interval, dd.Aggregator, url)
}

func (p *Provider) createRequestV1(query string, now int64, interval int64, url *url.URL) (*http.Request, error) {
Expand All @@ -201,11 +201,11 @@ func (p *Provider) createRequestV1(query string, now int64, interval int64, url
return &http.Request{Method: "GET"}, nil
}

func buildQueriesPayload(queries map[string]string) []map[string]string {
func buildQueriesPayload(queries map[string]string, aggregator string) []map[string]string {
qp := make([]map[string]string, 0, len(queries))
for k, v := range queries {
p := map[string]string{
"aggregator": "last",
"aggregator": aggregator,
"data_source": "metrics",
"name": k,
"query": v,
Expand All @@ -215,7 +215,7 @@ func buildQueriesPayload(queries map[string]string) []map[string]string {
return qp
}

func (p *Provider) createRequestV2(queries map[string]string, formula string, now int64, interval int64, url *url.URL) (*http.Request, error) {
func (p *Provider) createRequestV2(queries map[string]string, formula string, now int64, interval int64, aggregator string, url *url.URL) (*http.Request, error) {
formulas := []map[string]string{}
// ddAPI supports multiple formulas but doesn't make sense in our context
// can't have a 'blank' formula, so have to guard
Expand All @@ -229,7 +229,7 @@ func (p *Provider) createRequestV2(queries map[string]string, formula string, no
// Datadog requires milliseconds for v2 api
From: (now - interval) * 1000,
To: now * 1000,
Queries: buildQueriesPayload(queries),
Queries: buildQueriesPayload(queries, aggregator),
Formulas: formulas,
}

Expand Down Expand Up @@ -411,6 +411,10 @@ func validateIncomingProps(dd *v1alpha1.DatadogMetric) error {
return errors.New("When multiple queries are provided you must include a formula.")
}

if dd.ApiVersion == "v1" && dd.Aggregator != "" {
return errors.New("Aggregator is not supported in v1. Please review the Analysis Template.")
}

return nil
}

Expand Down
40 changes: 40 additions & 0 deletions metricproviders/datadog/datadog_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,26 @@ func TestDatadogSpecDefaults(t *testing.T) {
defaultInterval := string(ddSpec.Properties["interval"].Default.Raw)
assert.Equal(t, "\"5m\"", defaultInterval, "Default interval should be \"5m\" ")
})

t.Run("aggregator: Validate enum exists to restrict aggregator to 9 options", func(t *testing.T) {
aggregatorEnums := ddSpec.Properties["aggregator"].Enum
assert.Equal(t, 9, len(aggregatorEnums), "Expecting 9 enum options")
assert.Equal(t, "\"avg\"", string(aggregatorEnums[0].Raw), "\"avg\" expected, got %s", string(aggregatorEnums[0].Raw))
assert.Equal(t, "\"min\"", string(aggregatorEnums[1].Raw), "\"min\" expected, got %s", string(aggregatorEnums[1].Raw))
assert.Equal(t, "\"max\"", string(aggregatorEnums[2].Raw), "\"max\" expected, got %s", string(aggregatorEnums[2].Raw))
assert.Equal(t, "\"sum\"", string(aggregatorEnums[3].Raw), "\"sum\" expected, got %s", string(aggregatorEnums[3].Raw))
assert.Equal(t, "\"last\"", string(aggregatorEnums[4].Raw), "\"last\" expected, got %s", string(aggregatorEnums[4].Raw))
assert.Equal(t, "\"percentile\"", string(aggregatorEnums[5].Raw), "\"percentile\" expected, got %s", string(aggregatorEnums[5].Raw))
assert.Equal(t, "\"mean\"", string(aggregatorEnums[6].Raw), "\"mean\" expected, got %s", string(aggregatorEnums[6].Raw))
assert.Equal(t, "\"l2norm\"", string(aggregatorEnums[7].Raw), "\"l2norm\" expected, got %s", string(aggregatorEnums[7].Raw))
assert.Equal(t, "\"area\"", string(aggregatorEnums[8].Raw), "\"area\" expected, got %s", string(aggregatorEnums[8].Raw))
})

t.Run("aggregator: Validate default is last", func(t *testing.T) {
defaultAggregator := string(ddSpec.Properties["aggregator"].Default.Raw)
assert.Equal(t, "\"last\"", defaultAggregator, "Default aggregator should be \"last\" ")
})

}

func TestValidateIncomingProps(t *testing.T) {
Expand Down Expand Up @@ -104,6 +124,15 @@ func TestValidateIncomingProps(t *testing.T) {
},
expectedErrorMessage: "Formula are only valid when queries are set",
},
{
name: "v1 query with aggregator",
metric: &v1alpha1.DatadogMetric{
ApiVersion: "v1",
Query: "foo",
Aggregator: "sum",
},
expectedErrorMessage: "Aggregator is not supported in v1. Please review the Analysis Template.",
},
{
name: "More than 1 queries with no formula",
metric: &v1alpha1.DatadogMetric{
Expand Down Expand Up @@ -131,6 +160,17 @@ func TestValidateIncomingProps(t *testing.T) {
},
expectedErrorMessage: "",
},
{
name: "valid queries with v2 and an aggregator",
metric: &v1alpha1.DatadogMetric{
ApiVersion: "v2",
Query: "",
Queries: map[string]string{"a": "sum:api_gateway.request.count{*}.as_count()", "b": "fish bike"},
Formula: "a + b",
Aggregator: "avg",
},
expectedErrorMessage: "",
},
}

for _, test := range tests {
Expand Down
4 changes: 4 additions & 0 deletions pkg/apiclient/rollout/rollout.swagger.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1181,6 +1181,10 @@
"apiVersion": {
"type": "string",
"title": "ApiVersion refers to the Datadog API version being used (default: v1). v1 will eventually be deprecated.\n+kubebuilder:validation:Enum=v1;v2\n+kubebuilder:default=v1"
},
"aggregator": {
"type": "string",
"title": "+kubebuilder:default=\"last\"\n+kubebuilder:validation:Enum=avg;min;max;sum;last;percentile;mean;l2norm;area\nAggregator is a type of aggregator to use for metrics-based queries (default: last). Used for v2"
}
}
},
Expand Down
4 changes: 4 additions & 0 deletions pkg/apis/rollouts/v1alpha1/analysis_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -582,4 +582,8 @@ type DatadogMetric struct {
// +kubebuilder:validation:Enum=v1;v2
// +kubebuilder:default=v1
ApiVersion string `json:"apiVersion,omitempty" protobuf:"bytes,5,opt,name=apiVersion"`
// +kubebuilder:default="last"
// +kubebuilder:validation:Enum=avg;min;max;sum;last;percentile;mean;l2norm;area
// Aggregator is a type of aggregator to use for metrics-based queries (default: last). Used for v2
Aggregator string `json:"aggregator,omitempty" protobuf:"bytes,6,opt,name=aggregator"`
}
Loading
Loading