Skip to content

Commit

Permalink
ELE-3044: New dimensions features docs (#1520)
Browse files Browse the repository at this point in the history
* exclude insignificant

* add dimensions to column anomalies tests
  • Loading branch information
dapollak authored May 9, 2024
1 parent de1c02d commit 2e0a0a6
Show file tree
Hide file tree
Showing 5 changed files with 38 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ sidebarTitle: "dimensions"

`dimensions: [list of SQL expressions]`

Configuration for the test `dimension_anomalies`.
Configuration for the tests `dimension_anomalies`, `column_anomalies` and `all_columns_anomalies`.
The test counts rows grouped by given column / columns / valid select sql expression.
Under `dimensions` you can configure the group by expression.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
---
title: "exclude_final_results"
sidebarTitle: "exclude_final_results"
---

`exclude_final_results: [SQL where expression on fields value / average]`

Failures in dimension anomaly tests consist of outliers in row counts across all dimensions during the training period.
Some dimensions may contribute metrics that are considered insignificant compared to others, and you may prefer not to receive alerts for them.
With this parameter, you can disregard such failures.

1. `value` - Outlier row count of a dimension during the detection period.
2. `average` - The average rows count of a dimension during the training period.

- _Supported values: valid SQL where expression on the columns value / average_
- _Relevant tests: Dimension anomalies _

<RequestExample>

```yml test
models:
- name: model_name
config:
elementary:
timestamp_column: updated_at
tests:
- elementary.dimension_anomalies:
dimensions:
- device_os
- device_browser
exclude_final_results: 'value > 1000 or average > 10'
```
</RequestExample>
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ No mandatory configuration, however it is highly recommended to configure a `tim
&nbsp;&nbsp;-- elementary.all_columns_anomalies:
&nbsp;&nbsp;&nbsp;&nbsp;<a href="/data-tests/anomaly-detection-configuration/timestamp-column"><font color="#CD7D55">timestamp_column: column name</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="/data-tests/anomaly-detection-configuration/column-anomalies"><font color="#CD7D55">column_anomalies: column monitors list</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="/data-tests/anomaly-detection-configuration/dimensions"><font color="#CD7D55">dimensions: list</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="/data-tests/anomaly-detection-configuration/exclude_prefix"><font color="#CD7D55">exclude_prefix: string</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="/data-tests/anomaly-detection-configuration/exclude_regexp"><font color="#CD7D55">exclude_regexp: regex</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="/data-tests/anomaly-detection-configuration/where-expression"><font color="#CD7D55">where_expression: sql expression</font></a>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ No mandatory configuration, however it is highly recommended to configure a `tim
tests:
&nbsp;&nbsp;-- elementary.column_anomalies:
&nbsp;&nbsp;&nbsp;&nbsp;<a href="/data-tests/anomaly-detection-configuration/column-anomalies"><font color="#CD7D55">column_anomalies: column monitors list</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="/data-tests/anomaly-detection-configuration/dimensions"><font color="#CD7D55">dimensions: list</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="/data-tests/anomaly-detection-configuration/timestamp-column"><font color="#CD7D55">timestamp_column: column name</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="/data-tests/anomaly-detection-configuration/where-expression"><font color="#CD7D55">where_expression: sql expression</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="/data-tests/anomaly-detection-configuration/anomaly-sensitivity"><font color="#CD7D55">anomaly_sensitivity: int</font></a>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ _Required configuration: `dimensions`_
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href="/data-tests/anomaly-detection-configuration/ignore_small_changes"><font color="#CD7D55">spike_failure_percent_threshold: int</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href="/data-tests/anomaly-detection-configuration/ignore_small_changes"><font color="#CD7D55">drop_failure_percent_threshold: int</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="/data-tests/anomaly-detection-configuration/anomaly-exclude-metrics"><font color="#CD7D55">anomaly_exclude_metrics: [SQL expression]</font></a>
&nbsp;&nbsp;&nbsp;&nbsp;<a href="/data-tests/anomaly-detection-configuration/exclude-final-results"><font color="#CD7D55">exclude_final_results: [SQL expression]</font></a>
</code>
</pre>

Expand Down

0 comments on commit 2e0a0a6

Please sign in to comment.