Skip to content

Commit

Permalink
Merge branch 'current' into update-strict-csp
Browse files Browse the repository at this point in the history
  • Loading branch information
john-rock authored Dec 10, 2024
2 parents 121ed9e + db2d160 commit 959b529
Show file tree
Hide file tree
Showing 18 changed files with 231 additions and 43 deletions.
1 change: 1 addition & 0 deletions website/dbt-versions.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ exports.versions = [
},
{
version: "1.9",
EOLDate: "2025-12-08",
},
{
version: "1.8",
Expand Down
5 changes: 4 additions & 1 deletion website/docs/docs/build/conversion-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ Note that we use the double colon (::) to indicate whether a parameter is nested
| `base_measure:name` | The base conversion event measure. | Required | String |
| `base_measure:fill_nulls_with` | Set the value in your metric definition instead of null (such as zero). | Optional | String |
| `base_measure:join_to_timespine` | Boolean that indicates if the aggregated measure should be joined to the time spine table to fill in missing dates. Default `false`. | Optional | Boolean |
| `base_measure:filter` | Optional `filter` used to apply to the base measure. | Optional | String |
| `conversion_measure` | A list of conversion measure inputs. | Required | Dict |
| `conversion_measure:name` | The base conversion event measure.| Required | String |
| `conversion_measure:fill_nulls_with` | Set the value in your metric definition instead of null (such as zero). | Optional | String |
Expand Down Expand Up @@ -61,6 +62,7 @@ metrics:
name: The name of the measure # Required
fill_nulls_with: Set the value in your metric definition instead of null (such as zero) # Optional
join_to_timespine: true/false # Boolean that indicates if the aggregated measure should be joined to the time spine table to fill in missing dates. Default `false`. # Optional
filter: The filter used to apply to the base measure. # Optional
conversion_measure:
name: The name of the measure # Required
fill_nulls_with: Set the value in your metric definition instead of null (such as zero) # Optional
Expand Down Expand Up @@ -105,12 +107,13 @@ Next, define a conversion metric as follows:
- name: visit_to_buy_conversion_rate_7d
description: "Conversion rate from visiting to transaction in 7 days"
type: conversion
label: Visit to Buy Conversion Rate (7-day window)
label: Visit to buy conversion rate (7-day window)
type_params:
conversion_type_params:
base_measure:
name: visits
fill_nulls_with: 0
filter: {{ Dimension('visits__referrer_id') }} = 'facebook'
conversion_measure:
name: sellers
entity: user
Expand Down
8 changes: 7 additions & 1 deletion website/docs/docs/build/simple.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ Note that we use the double colon (::) to indicate whether a parameter is nested
| `type_params` | The type parameters of the metric. | Required | Dict |
| `measure` | A list of measure inputs. | Required | List |
| `measure:name` | The measure you're referencing. | Required | String |
| `measure:alias` | Optional [`alias`](/reference/resource-configs/alias) to rename the measure. | Optional | String |
| `measure:filter` | Optional `filter` applied to the measure. | Optional | String |
| `measure:fill_nulls_with` | Set the value in your metric definition instead of null (such as zero). | Optional | String |
| `measure:join_to_timespine` | Indicates if the aggregated measure should be joined to the time spine table to fill in missing dates. Default `false`. | Optional | Boolean |

Expand All @@ -38,6 +40,8 @@ metrics:
type_params: # Required
measure:
name: The name of your measure # Required
alias: The alias applied to the measure. # Optional
filter: The filter applied to the measure. # Optional
fill_nulls_with: Set value instead of null (such as zero) # Optional
join_to_timespine: true/false # Boolean that indicates if the aggregated measure should be joined to the time spine table to fill in missing dates. # Optional

Expand Down Expand Up @@ -65,9 +69,11 @@ If you've already defined the measure using the `create_metric: true` parameter,
name: customers # The measure you are creating a proxy of.
fill_nulls_with: 0
join_to_timespine: true
alias: customer_count
filter: {{ Dimension('customer__customer_total') }} >= 20
- name: large_orders
description: "Order with order values over 20."
type: SIMPLE
type: simple
label: Large orders
type_params:
measure:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,14 @@ To set up [personal access tokens (PATs)](https://help.tableau.com/current/serve
<Lightbox src="/img/docs/cloud-integrations/auto-exposures/cloud-integration-details.jpg" title="Enter the details for the exposure connection."/>
4. Select the collections you want to include for auto exposures.

dbt Cloud automatically imports and syncs any workbook within the selected collections. New additions to the collections will be added to the lineage in dbt Cloud during the next automatic sync (usually once per day).
<Lightbox src="/img/docs/cloud-integrations/auto-exposures/cloud-select-collections.jpg" title="Select the collections you want to include for auto exposures."/>

:::info
dbt Cloud automatically imports and syncs any workbook within the selected collections. New additions to the collections will be added to the lineage in dbt Cloud during the next sync (automatically once per day).

dbt Cloud immediately starts a sync when you update the selected collections list, capturing new workbooks and removing irrelevant ones.
:::

5. Click **Save**.

dbt Cloud imports everything in the collection(s) and you can continue to view them in Explorer. For more information on how to view and use auto-exposures, refer to [View auto-exposures from dbt Explorer](/docs/collaborate/auto-exposures) page.
Expand Down
15 changes: 3 additions & 12 deletions website/docs/docs/cloud/git/connect-gitlab.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Connecting your GitLab account to dbt Cloud provides convenience and another lay
- Clone repos using HTTPS rather than SSH.
- Carry GitLab user permissions through to dbt Cloud or dbt Cloud CLI's git actions.
- Trigger [Continuous integration](/docs/deploy/continuous-integration) builds when merge requests are opened in GitLab.
- GitLab automatically registers a webhook in your GitLab repository to enable seamless integration with dbt Cloud.

The steps to integrate GitLab in dbt Cloud depend on your plan. If you are on:
- the Developer or Team plan, read these [instructions](#for-dbt-cloud-developer-and-team-tiers).
Expand Down Expand Up @@ -114,20 +115,10 @@ If your GitLab account is not connected, you’ll see "No connected account". Se

Once you approve authorization, you will be redirected to dbt Cloud, and you should see your connected account. You're now ready to start developing in the dbt Cloud IDE or dbt Cloud CLI.


## Troubleshooting

### Errors when importing a repository on dbt Cloud project set up
If you do not see your repository listed, double-check that:
- Your repository is in a Gitlab group you have access to. dbt Cloud will not read repos associated with a user.

If you do see your repository listed, but are unable to import the repository successfully, double-check that:
- You are a maintainer of that repository. Only users with maintainer permissions can set up repository connections.

If you imported a repository using the dbt Cloud native integration with GitLab, you should be able to see the clone strategy is using a `deploy_token`. If it's relying on an SSH key, this means the repository was not set up using the native GitLab integration, but rather using the generic git clone option. The repository must be reconnected in order to get the benefits described above.

## FAQs

<FAQ path="Troubleshooting/gitlab-webhook"/>
<FAQ path="Troubleshooting/error-importing-repo"/>
<FAQ path="Git/gitignore"/>
<FAQ path="Git/gitlab-authentication"/>
<FAQ path="Git/gitlab-selfhosted"/>
Expand Down
3 changes: 3 additions & 0 deletions website/docs/docs/dbt-versions/release-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,12 @@ Release notes are grouped by month for both multi-tenant and virtual private clo

## December 2024

- **Fix**: [The dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) now respects the BigQuery [`execution_project` attribute](/docs/core/connect-data-platform/bigquery-setup#execution-project), including for exports.
- **New**: [Model notifications](/docs/deploy/model-notifications) are now generally available in dbt Cloud. These notifications alert model owners through email about any issues encountered by models and tests as soon as they occur while running a job.
- **New**: You can now use your [Azure OpenAI key](/docs/cloud/account-integrations?ai-integration=azure#ai-integrations) (available in beta) to use dbt Cloud features like [dbt Copilot](/docs/cloud/dbt-copilot) and [Ask dbt](/docs/cloud-integrations/snowflake-native-app) . Additionally, you can use your own [OpenAI API key](/docs/cloud/account-integrations?ai-integration=openai#ai-integrations) or use [dbt Labs-managed OpenAI](/docs/cloud/account-integrations?ai-integration=dbtlabs#ai-integrations) key. Refer to [AI integrations](/docs/cloud/account-integrations#ai-integrations) for more information.
- **New**: The [`hard_deletes`](/reference/resource-configs/hard-deletes) config gives you more control on how to handle deleted rows from the source. Supported options are `ignore` (default), `invalidate` (replaces the legacy `invalidate_hard_deletes=true`), and `new_record`. Note that `new_record` will create a new metadata column in the snapshot table.


## November 2024
- **Enhancement**: Trust signal icons in dbt Explorer are now available for Exposures, providing a quick view of data health while browsing resources. To view trust signal icons, go to dbt Explorer and click **Exposures** under the **Resource** tab. Refer to [Trust signal for resources](/docs/collaborate/explore-projects#trust-signals-for-resources) for more info.
- **Bug**: Identified and fixed an error with Semantic Layer queries that take longer than 10 minutes to complete.
Expand Down
3 changes: 3 additions & 0 deletions website/docs/docs/deploy/ci-jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,8 @@ To validate _all_ semantic nodes in your project, add the following command to d

## Troubleshooting

<FAQ path="Troubleshooting/gitlab-webhook"/>

<DetailsToggle alt_header="Temporary schemas aren't dropping">
If your temporary schemas aren't dropping after a PR merges or closes, this typically indicates one of these issues:
- You have overridden the <code>generate_schema_name</code> macro and it isn't using <code>dbt_cloud_pr_</code> as the prefix.
Expand All @@ -201,6 +203,7 @@ A macro is creating a schema but there are no dbt models writing to that schema.

</DetailsToggle>


<DetailsToggle alt_header="Error messages that refer to schemas from previous PRs">

If you receive a schema-related error message referencing a <i>previous</i> PR, this is usually an indicator that you are not using a production job for your deferral and are instead using <i>self</i>. If the prior PR has already been merged, the prior PR's schema may have been dropped by the time the CI job for the current PR is kicked off.
Expand Down
20 changes: 7 additions & 13 deletions website/docs/docs/deploy/model-notifications.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,37 +3,31 @@ title: "Model notifications"
description: "While a job is running, receive email notifications in real time about any issues with your models and tests. "
---

# Model notifications <Lifecycle status="beta" />

Set up dbt to notify the appropriate model owners through email about issues as soon as they occur, while the job is still running. Model owners can specify which statuses to receive notifications about:

- `Success` and `Fails` for models
- `Warning`, `Success`, and `Fails` for tests

With model-level notifications, model owners can be the first ones to know about issues before anyone else (like the stakeholders).

:::info Beta feature

This feature is currently available in [beta](/docs/dbt-versions/product-lifecycles#dbt-cloud) to a limited group of users and is gradually being rolled out. If you're in the beta, please contact the Support team at [email protected] for assistance or questions.

:::

To be timely and keep the number of notifications to a reasonable amount when multiple models or tests trigger them, dbt observes the following guidelines when notifying the owners:

- Send a notification to each unique owner/email during a job run about any models (with status of failure/success) or tests (with status of warning/failure/success). Each owner receives only one notification, the initial one.
- Don't send any notifications about subsequent models or tests while a dbt job is still running.
- At the end of a job run, each owner receives a notification, for each of the statuses they specified to be notified about, with a list of models and tests that have that status.
- No notifications sent about subsequent models or tests while a dbt job is still running.
- Each owner/user who subscribes to notifications for one or more statuses (like failure, success, warning) will receive only _one_ email notification at the end of the job run.
- The email includes a consolidated list of all models or tests that match the statuses the user subscribed to, instead of sending separate emails for each status.

Create configuration YAML files in your project for dbt to send notifications about the status of your models and tests.

## Prerequisites
- Your dbt Cloud administrator has [enabled the appropriate account setting](#enable-access-to-model-notifications) for you.
- Your environment(s) must be on a [release track](/docs/dbt-versions/cloud-release-tracks) instead of a legacy dbt Core version.


## Configure groups

Define your groups in any .yml file in your [models directory](/reference/project-configs/model-paths). For example:
Define your groups in any `.yml` file in your [models directory](/reference/project-configs/model-paths). Each group must have a single email address specified &mdash; multiple email fields or lists aren't supported.

The following example shows how to define groups in a `groups.yml` file.

<File name='models/groups.yml'>

Expand Down Expand Up @@ -116,6 +110,6 @@ Provide dbt Cloud account members the ability to configure and receive alerts ab
To use model-level notifications, your dbt Cloud account must have access to the feature. Ask your dbt Cloud administrator to enable this feature for account members by following these steps:

1. Navigate to **Notification settings** from your profile name in the sidebar (lower left-hand side).
1. From **Email notications**, enable the setting **Enable group/owner notifications on models** under the **Model notifications** section. Then, specify which statuses to receive notifications about (Success, Warning, and/or Fails).
1. From **Email notifications**, enable the setting **Enable group/owner notifications on models** under the **Model notifications** section. Then, specify which statuses to receive notifications about (Success, Warning, and/or Fails).

<Lightbox src="/img/docs/dbt-cloud/example-enable-model-notifications.png" title="Example of the setting Enable group/owner notifications on models" />
2 changes: 1 addition & 1 deletion website/docs/docs/deploy/monitor-jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ This portion of our documentation will go over dbt Cloud's various capabilities
- [Run visibility](/docs/deploy/run-visibility) &mdash; View your run history to help identify where improvements can be made to scheduled jobs.
- [Retry jobs](/docs/deploy/retry-jobs) &mdash; Rerun your errored jobs from start or the failure point.
- [Job notifications](/docs/deploy/job-notifications) &mdash; Receive email or Slack notifications when a job run succeeds, encounters warnings, fails, or is canceled.
- [Model notifications](/docs/deploy/model-notifications)<Lifecycle status="beta"/> &mdash; Receive email notifications about any issues encountered by your models and tests as soon as they occur while running a job.
- [Model notifications](/docs/deploy/model-notifications) &mdash; Receive email notifications about any issues encountered by your models and tests as soon as they occur while running a job.
- [Webhooks](/docs/deploy/webhooks) &mdash; Use webhooks to send events about your dbt jobs' statuses to other systems.
- [Leverage artifacts](/docs/deploy/artifacts) &mdash; dbt Cloud generates and saves artifacts for your project, which it uses to power features like creating docs for your project and reporting freshness of your sources.
- [Source freshness](/docs/deploy/source-freshness) &mdash; Monitor data governance by enabling snapshots to capture the freshness of your data sources.
Expand Down
14 changes: 14 additions & 0 deletions website/docs/faqs/Troubleshooting/error-importing-repo.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
title: Errors importing a repository on dbt Cloud project set up
description: "Errors importing a repository on dbt Cloud project set up"
sidebar_label: 'Errors importing a repository on dbt Cloud project set up'
id: error-importing-repo
---

If you don't see your repository listed, double-check that:
- Your repository is in a Gitlab group you have access to. dbt Cloud will not read repos associated with a user.

If you do see your repository listed, but are unable to import the repository successfully, double-check that:
- You are a maintainer of that repository. Only users with maintainer permissions can set up repository connections.

If you imported a repository using the dbt Cloud native integration with GitLab, you should be able to see if the clone strategy is using a `deploy_token`. If it's relying on an SSH key, this means the repository was not set up using the native GitLab integration, but rather using the generic git clone option. The repository must be reconnected in order to get the benefits described above.
19 changes: 19 additions & 0 deletions website/docs/faqs/Troubleshooting/gitlab-webhook.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
title: Unable to trigger a CI job with GitLab
description: "Unable to trigger a CI job"
sidebar_label: 'Unable to trigger a CI job'
id: gitlab-webhook
---

When you connect dbt Cloud to a GitLab repository, GitLab automatically registers a webhook in the background, viewable under the repository settings. This webhook is also used to trigger [CI jobs](/docs/deploy/ci-jobs) when you push to the repository.

If you're unable to trigger a CI job, this usually indicates that the webhook registration is missing or incorrect.

To resolve this issue, navigate to the repository settings in GitLab and view the webhook registrations by navigating to GitLab --> **Settings** --> **Webhooks**.

Some things to check:

- The webhook registration is enabled in GitLab.
- The webhook registration is configured with the correct URL and secret.

If you're still experiencing this issue, reach out to the Support team at [email protected] and we'll be happy to help!
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ sidebar: "Indirect selection"

import IndirSelect from '/snippets/_indirect-selection-definitions.md';

Use the `--indirect_selection` flag to `dbt test` or `dbt build` to configure which tests to run for the nodes you specify. You can set this as a CLI flag or an environment variable. In dbt Core, you can also configure user configurations in [YAML selectors](/reference/node-selection/yaml-selectors) or in the `flags:` block of `dbt_project.yml`, which sets project-level flags.
Use the `--indirect-selection` flag to `dbt test` or `dbt build` to configure which tests to run for the nodes you specify. You can set this as a CLI flag or an environment variable. In dbt Core, you can also configure user configurations in [YAML selectors](/reference/node-selection/yaml-selectors) or in the `flags:` block of `dbt_project.yml`, which sets project-level flags.

When all flags are set, the order of precedence is as follows. Refer to [About global configs](/reference/global-configs/about-global-configs) for more details:

Expand Down
Loading

0 comments on commit 959b529

Please sign in to comment.