Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add integration tests for the package #29

Merged
merged 4 commits into from
Feb 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ packages:
- ✅: supported
- ✔️: supported not tested
- ❌: not supported
- ℹ️: Opened to contributions ❤️
- ℹ️: Opened to contributions ❤️ (see the integration_tests folder on how to test your adapter)


For latest release, see [https://github.com/AxelThevenot/dbt-assertions/releases](https://github.com/AxelThevenot/dbt-assertions/releases/latest)
Expand All @@ -118,7 +118,7 @@ This package do not have variables.

## Basic Example

Check the [basic_example](models/examples/basic_example) example.
Check the [basic_example](examples/basic_example) example.

## Documentation

Expand Down Expand Up @@ -242,7 +242,7 @@ model:

`[]` represents optional parts. Yes everything is optional but let's see it by examples.

In the [basic test example](./models/examples/basic_test_example/)
In the [basic test example](./examples/basic_test_example/)
you can easily create your test as follows then run your `dbt test` command.

```yml
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,5 @@ WITH
)
SELECT
*,
{{ dbt_assertions.assertions() | indent(4) }},
FROM `final`
{{ dbt_assertions.assertions() | indent(4) }}
FROM final
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
SELECT
site_id,
site_trigram,
open_date,
open_date
FROM {{ ref('dbt_assertions', 'basic_example_d_site') }}
-- Remove bad data: here only sites without ID.
WHERE {{ dbt_assertions.assertions_filter(include_list=['site_id_is_not_null']) }}
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,5 @@ WITH
DATE('2023-01-01') AS open_date
)
SELECT
*,
FROM `final`
*
FROM final
1 change: 1 addition & 0 deletions integration_tests/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
package-lock.yml
9 changes: 9 additions & 0 deletions integration_tests/README.MD
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Integration tests

To test the package for a new adapter:

- in your shell, `cd` to `integration_tests`
- set up the [relevant profile information](https://docs.getdbt.com/docs/core/connect-data-platform/connection-profiles) for the profile name `integration_tests`
- do a `dbt deps` to symlink the package from the repo one (this is only required the first time you run some tests and not every time)
- do a `dbt build` (or `dbt build --target my_target` to specify a given target)
- the dbt command should run successfully, creating the models and running the test as expected ✅
42 changes: 42 additions & 0 deletions integration_tests/dbt_project.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@

# Name your project! Project names should contain only lowercase characters
# and underscores. A good package name should reflect your organization's
# name or the intended use of these models
name: 'dbt_assertions_integration_tests'
version: '1.0.0'
config-version: 2

# This setting configures which "profile" dbt uses for this project.
profile: 'integration_tests'

# These configurations specify where dbt should look for different types of files.
# The `source-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!
model-paths: ["models"]
analysis-paths: ["analysis"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

target-path: "target" # directory which will store compiled SQL files
clean-targets: # directories to be removed by `dbt clean`
- "target"
- "dbt_packages"

# models:
# dbt_assertions_integration_tests:
# # materialize as ephemeral to prevent the fake models from executing, but keep them enabled
# +materialized: ephemeral

# tests:
# dbt_project_evaluator_integration_tests:
# dbt_project_evaluator_schema_tests:
# unique_int_all_dag_relationships_path:
# # Grouping by expressions of type ARRAY is not allowed for BigQuery
# +enabled: "{{ false if target.type in ['bigquery'] else true }}"

# seeds:
# dbt_project_evaluator:
# dbt_project_evaluator_exceptions:
# +enabled: false
25 changes: 25 additions & 0 deletions integration_tests/models/basic_example/basic_example_d_site.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{{
config(alias='d_site', materialized='table')
}}

WITH
final AS (
SELECT
1 AS site_id,
'FRA' AS site_trigram,
DATE('2023-01-01') AS open_date
UNION ALL
SELECT
2 AS site_id,
'France' AS site_trigram,
DATE('2023-01-01') AS open_date
UNION ALL
SELECT
NULL AS site_id,
'Belgium' AS site_trigram,
DATE('2023-01-01') AS open_date
)
SELECT
*,
{{ dbt_assertions.assertions() | indent(4) }}
FROM final
19 changes: 19 additions & 0 deletions integration_tests/models/basic_example/basic_example_d_site.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
version: 2

models:
- name: basic_example_d_site
columns:
- name: site_id
- name: country_trigram
- name: open_date
- name: exceptions
assertions:
site_id_is_not_null:
description: 'Site ID is not null.'
expression: site_id IS NOT NULL

site_trigram_format:
description: 'Site trigram must contain 3 upper digits'
expression: |
LENGTH(site_trigram) = 3
AND site_trigram = UPPER(site_trigram)
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{{
config(alias='downstream_model', materialized='table')
}}

SELECT
site_id,
site_trigram,
open_date
FROM {{ ref('basic_example_d_site') }}
-- Remove bad data: here only sites without ID.
WHERE {{ dbt_assertions.assertions_filter(include_list=['site_id_is_not_null']) }}
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{{
config(alias='d_site_test', materialized='table')
}}

WITH
final AS (
SELECT
1 AS site_id,
'FRA' AS site_trigram,
DATE('2023-01-01') AS open_date
UNION ALL
SELECT
2 AS site_id,
'France' AS site_trigram,
DATE('2023-01-01') AS open_date
UNION ALL
SELECT
NULL AS site_id,
'Belgium' AS site_trigram,
DATE('2023-01-01') AS open_date
)
SELECT
*
FROM final
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
version: 2

models:
- name: basic_test_example_d_site
tests:
- dbt_assertions.generic_assertions:
column: exceptions
include_list:
- site_id_is_not_null
# `re_assert: true` to use only if your assertion's column
# is not computed and saved in your table.
re_assert: true
config:
warn_if: "!=1"
error_if: "!=1"

columns:
- name: site_id
- name: country_trigram
- name: open_date
- name: exceptions
assertions:
site_id_is_not_null:
description: 'Site ID is not null.'
expression: site_id IS NOT NULL

site_trigram_format:
description: 'Site trigram must contain 3 upper digits'
expression: |
LENGTH(site_trigram) = 3
AND site_trigram = UPPER(site_trigram)
2 changes: 2 additions & 0 deletions integration_tests/packages.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
packages:
- local: ../