Skip to content

Commit

Permalink
Merge branch 'current' into patch-1
Browse files Browse the repository at this point in the history
  • Loading branch information
mirnawong1 authored Dec 13, 2024
2 parents 7b07a59 + 4705cdd commit 6c808bd
Show file tree
Hide file tree
Showing 24 changed files with 327 additions and 107 deletions.
12 changes: 6 additions & 6 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@
* @dbt-labs/product-docs

# Adapter & Package Development Docs
/website/docs/docs/supported-data-platforms.md @dbt-labs/product-docs @dataders
/website/docs/reference/warehouse-setups @dbt-labs/product-docs @dataders
/website/docs/docs/supported-data-platforms.md @dbt-labs/product-docs @amychen1776
/website/docs/reference/warehouse-setups @dbt-labs/product-docs @amychen1776
# `resource-configs` contains more than just warehouse setups
/website/docs/reference/resource-configs/*-configs.md @dbt-labs/product-docs @dataders
/website/docs/guides/advanced/adapter-development @dbt-labs/product-docs @dataders @dbeatty10
/website/docs/reference/resource-configs/*-configs.md @dbt-labs/product-docs @amychen1776
/website/docs/guides/advanced/adapter-development @dbt-labs/product-docs @amychen1776

/website/docs/guides/building-packages @dbt-labs/product-docs @amychen1776 @dataders @dbeatty10
/website/docs/guides/creating-new-materializations @dbt-labs/product-docs @dataders @dbeatty10
/website/docs/guides/building-packages @dbt-labs/product-docs @amychen1776
/website/docs/guides/creating-new-materializations @dbt-labs/product-docs

# Require approval from the Multicell team when making
# changes to the public facing migration documentation.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ id: 5-how-we-style-our-yaml

- 2️⃣ Indents should be two spaces
- ➡️ List items should be indented
- 🔠 List items with a single entry can be a string. For example, `'select': 'other_user'`, but it's best practice to provide the argument as an explicit list. For example, `'select': ['other_user']`
- 🆕 Use a new line to separate list items that are dictionaries where appropriate
- 📏 Lines of YAML should be no longer than 80 characters.
- 🛠️ Use the [dbt JSON schema](https://github.com/dbt-labs/dbt-jsonschema) with any compatible IDE and a YAML formatter (we recommend [Prettier](https://prettier.io/)) to validate your YAML files and format them automatically.
Expand Down
110 changes: 60 additions & 50 deletions website/docs/community/resources/oss-expectations.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion website/docs/docs/build/incremental-microbatch.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ Several configurations are relevant to microbatch models, and some are required:
| [`begin`](/reference/resource-configs/begin) | The "beginning of time" for the microbatch model. This is the starting point for any initial or full-refresh builds. For example, a daily-grain microbatch model run on `2024-10-01` with `begin = '2023-10-01` will process 366 batches (it's a leap year!) plus the batch for "today." | N/A | Date | Required |
| [`batch_size`](/reference/resource-configs/batch-size) | The granularity of your batches. Supported values are `hour`, `day`, `month`, and `year` | N/A | String | Required |
| [`lookback`](/reference/resource-configs/lookback) | Process X batches prior to the latest bookmark to capture late-arriving records. | `1` | Integer | Optional |
| [`concurrent_batches`](/reference/resource-properties/concurrent_batches) | An override for whether batches run concurrently (at the same time) or sequentially (one after the other). | `None` | Boolean | Optional |
| [`concurrent_batches`](/reference/resource-properties/concurrent_batches) | Overrides dbt's auto detect for running batches concurrently (at the same time). Read more about [configuring concurrent batches](/docs/build/incremental-microbatch#configure-concurrent_batches). Setting to <br />* `true` runs batches concurrently (in parallel). <br />* `false` runs batches sequentially (one after the other). | `None` | Boolean | Optional |

<Lightbox src="/img/docs/building-a-dbt-project/microbatch/event_time.png" title="The event_time column configures the real-world time of this record"/>

Expand Down
1 change: 1 addition & 0 deletions website/docs/docs/build/unit-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ Starting in dbt Core v1.8, we have introduced an additional type of test to dbt
- We currently only support adding unit tests to models in your _current_ project.
- We currently _don't_ support unit testing models that use the [`materialized view`](/docs/build/materializations#materialized-view) materialization.
- We currently _don't_ support unit testing models that use recursive SQL.
- We currently _don't_ support unit testing models that use introspective queries.
- If your model has multiple versions, by default the unit test will run on *all* versions of your model. Read [unit testing versioned models](/reference/resource-properties/unit-testing-versions) for more information.
- Unit tests must be defined in a YML file in your [`models/` directory](/reference/project-configs/model-paths).
- Table names must be aliased in order to unit test `join` logic.
Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/cloud/about-cloud/about-dbt-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ dbt Cloud's [flexible plans](https://www.getdbt.com/pricing/) and features make

<Card
title="dbt Cloud IDE"
body="The IDE is the easiest and most efficient way to develop dbt models, allowing you to build, test, run, and version control your dbt projects directly from your browser. Use dbt Copilot, a powerful AI engine that automatically generates documentation, tests, and semantic models."
body="The IDE is the easiest and most efficient way to develop dbt models, allowing you to build, test, run, and version control your dbt projects directly from your browser. Use dbt Copilot, a powerful AI engine that automatically generates code, documentation, tests, and semantic models."
link="/docs/cloud/dbt-cloud-ide/develop-in-the-cloud"
icon="dbt-bit"/>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ The dbt Cloud integrated development environment (IDE) is a single web-based int
The dbt Cloud IDE offers several [keyboard shortcuts](/docs/cloud/dbt-cloud-ide/keyboard-shortcuts) and [editing features](/docs/cloud/dbt-cloud-ide/ide-user-interface#editing-features) for faster and efficient development and governance:

- Syntax highlighting for SQL &mdash; Makes it easy to distinguish different parts of your code, reducing syntax errors and enhancing readability.
- AI copilot &mdash; Use [dbt Copilot](/docs/cloud/dbt-copilot), a powerful AI engine that can generate documentation, tests, and semantic models for your dbt SQL models.
- AI copilot &mdash; Use [dbt Copilot](/docs/cloud/dbt-copilot), a powerful AI engine that can [generate code](/docs/cloud/use-dbt-copilot#generate-and-edit-code) using natural language, and [generate documentation](/docs/build/documentation), [tests](/docs/build/data-tests), and [semantic models](/docs/build/semantic-models) for you with the click of a button.
- Auto-completion &mdash; Suggests table names, arguments, and column names as you type, saving time and reducing typos.
- Code [formatting and linting](/docs/cloud/dbt-cloud-ide/lint-format) &mdash; Helps standardize and fix your SQL code effortlessly.
- Navigation tools &mdash; Easily move around your code, jump to specific lines, find and replace text, and navigate between project files.
Expand Down
6 changes: 4 additions & 2 deletions website/docs/docs/cloud/dbt-copilot.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,12 @@ pagination_prev: null

# About dbt Copilot <Lifecycle status='beta'/>

dbt Copilot is a powerful artificial intelligence (AI) engine that's fully integrated into your dbt Cloud experience and designed to accelerate your analytics workflows. dbt Copilot embeds AI-driven assistance across every stage of the analytics development life cycle (ADLC), empowering data practitioners to deliver data products faster, improve data quality, and enhance data accessibility. With automatic code generation, you can let the AI engine generate the [documentation](/docs/build/documentation), [tests](/docs/build/data-tests), and [semantic models](/docs/build/semantic-models) for you.
dbt Copilot is a powerful artificial intelligence (AI) engine that's fully integrated into your dbt Cloud experience and designed to accelerate your analytics workflows. dbt Copilot embeds AI-driven assistance across every stage of the analytics development life cycle (ADLC), empowering data practitioners to deliver data products faster, improve data quality, and enhance data accessibility.

With automatic code generation, let dbt Copilot [generate code](/docs/cloud/use-dbt-copilot#generate-and-edit-code) using natural language, and [generate documentation](/docs/build/documentation), [tests](/docs/build/data-tests), and [semantic models](/docs/build/semantic-models) for you with the click of a button.

:::tip Beta feature
dbt Copilot is designed to _help_ developers generate documentation, tests, and semantic models in dbt Cloud. It's available in beta, in the dbt Cloud IDE only.
dbt Copilot is designed to _help_ developers generate documentation, tests, and semantic models, as well as [code](/docs/cloud/use-dbt-copilot#generate-and-edit-code) using natural language, in dbt Cloud. It's available in beta, in the dbt Cloud IDE only.

To use dbt Copilot, you must have an active [dbt Cloud Enterprise account](https://www.getdbt.com/pricing) and either agree to use dbt Labs' OpenAI key or provide your own Open AI API key. [Register here](https://docs.google.com/forms/d/e/1FAIpQLScPjRGyrtgfmdY919Pf3kgqI5E95xxPXz-8JoVruw-L9jVtxg/viewform) or reach out to the Account Team if you're interested in joining the private beta.
:::
Expand Down
63 changes: 57 additions & 6 deletions website/docs/docs/cloud/use-dbt-copilot.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,73 @@
---
title: "Use dbt Copilot"
sidebar_label: "Use dbt Copilot"
description: "Use the dbt Copilot AI engine to generate documentation, tests, and semantic models from scratch, giving you the flexibility to modify or fix generated code."
description: "Use dbt Copilot to generate documentation, tests, semantic models, and sql code from scratch, giving you the flexibility to modify or fix generated code."
---

# Use dbt Copilot <Lifecycle status='beta'/>

Use dbt Copilot to generate documentation, tests, and semantic models from scratch, giving you the flexibility to modify or fix generated code. To access and use this AI engine:
Use dbt Copilot to generate documentation, tests, semantic models, and code from scratch, giving you the flexibility to modify or fix generated code.

1. Navigate to the dbt Cloud IDE and select a SQL model file under the **File Explorer**.
This page explains how to use dbt Copilot to:

2. In the **Console** section (under the **File Editor**), click **dbt Copilot** to view the available AI options.
- [Generate resources](#generate-resources) &mdash; Save time by using dbt Copilot’s generation button to generate documentation, tests, and semantic model files during your development.
- [Generate and edit code](#generate-and-edit-code) &mdash; Use natural language prompts to generate SQL code from scratch or to edit existing SQL file by using keyboard shortcuts or highlighting code.

## Generate resources

Generate documentation, tests, and semantic models resources with the click-of-a-button using dbt Copilot, saving you time. To access and use this AI feature:

1. Navigate to the dbt Cloud IDE and select a SQL model file under the **File Explorer**.
2. In the **Console** section (under the **File Editor**), click **dbt Copilot** to view the available AI options.
3. Select the available options to generate the YAML config: **Generate Documentation**, **Generate Tests**, or **Generate Semantic Model**.
- To generate multiple YAML configs for the same model, click each option separately. dbt Copilot intelligently saves the YAML config in the same file.

4. Verify the AI-generated code. You can update or fix the code as needed.

5. Click **Save As**. You should see the file changes under the **Version control** section.

<Lightbox src="/img/docs/dbt-cloud/cloud-ide/dbt-copilot-doc.gif" width="100%" title="Example of using dbt Copilot to generate documentation in the IDE" />

## Generate and edit code <Lifecycle status='beta'/>

dbt Copilot also allows you to generate SQL code directly within the SQL file in the dbt Cloud IDE, using natural language prompts. This means you can rewrite or add specific portions of the SQL file without needing to edit the entire file.

This intelligent AI tool streamlines SQL development by reducing errors, scaling effortlessly with complexity, and saving valuable time. dbt Copilot's [prompt window](#use-the-prompt-window), accessible by keyboard shortcut, handles repetitive or complex SQL generation effortlessly so you can focus on high-level tasks.

Use Copilot's prompt window for use cases like:

- Writing advanced transformations
- Performing bulk edits efficiently
- Crafting complex patterns like regex

### Use the prompt window

Access dbt Copilot's AI prompt window using the keyboard shortcut Cmd+B (Mac) or Ctrl+B (Windows) to:

#### 1. Generate SQL from scratch
- Use the keyboard shortcuts Cmd+B (Mac) or Ctrl+B (Windows) to generate SQL from scratch.
- Enter your instructions to generate SQL code tailored to your needs using natural language.
- Ask dbt Copilot to fix the code or add a specific portion of the SQL file.

<Lightbox src="/img/docs/dbt-cloud/cloud-ide/copilot-sql-generation-prompt.jpg" width="90%" title="dbt Copilot's prompt window accessible by keyboard shortcut Cmd+B (Mac) or Ctrl+B (Windows)" />

#### 2. Edit existing SQL code
- Highlight a section of SQL code and press Cmd+B (Mac) or Ctrl+B (Windows) to open the prompt window for editing.
- Use this to refine or modify specific code snippets based on your needs.
- Ask dbt Copilot to fix the code or add a specific portion of the SQL file.

#### 3. Review changes with the diff view to quickly assess the impact of the changes before making changes
- When a suggestion is generated, Copilot displays a visual "diff" view to help you compare the proposed changes with your existing code:
- **Green**: Means new code that will be added if you accept the suggestion.
- **Red**: Highlights existing code that will be removed or replaced by the suggested changes.

#### 4. Accept or reject suggestions
- **Accept**: If the generated SQL meets your requirements, click the **Accept** button to apply the changes directly to your `.sql` file directly in the IDE.
- **Reject**: If the suggestion don’t align with your request/prompt, click **Reject** to discard the generated SQL without making changes and start again.

#### 5. Regenerate code
- To regenerate, press the **Escape** button on your keyboard (or click the Reject button in the popup). This will remove the generated code and puts your cursor back into the prompt text area.
- Update your prompt and press **Enter** to try another generation. Press **Escape** again to close the popover entirely.

Once you've accepted a suggestion, you can continue to use the prompt window to generate additional SQL code and commit your changes to the branch.

<Lightbox src="/img/docs/dbt-cloud/cloud-ide/copilot-sql-generation.gif" width="100%" title="Edit existing SQL code using dbt Copilot's prompt window accessible by keyboard shortcut Cmd+B (Mac) or Ctrl+B (Windows)" />

3 changes: 2 additions & 1 deletion website/docs/docs/community-adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ Community adapters are adapter plugins contributed and maintained by members of

| Data platforms (click to view setup guide) |||
| ------------------------------------------ | -------------------------------- | ------------------------------------- |
| [Clickhouse](/docs/core/connect-data-platform/clickhouse-setup) | [Databend Cloud](/docs/core/connect-data-platform/databend-setup) | [Doris & SelectDB](/docs/core/connect-data-platform/doris-setup) |
| [Clickhouse](/docs/core/connect-data-platform/clickhouse-setup) | [CrateDB](/docs/core/connect-data-platform/cratedb-setup)
| [Databend Cloud](/docs/core/connect-data-platform/databend-setup) | [Doris & SelectDB](/docs/core/connect-data-platform/doris-setup) |
| [DuckDB](/docs/core/connect-data-platform/duckdb-setup) | [Exasol Analytics](/docs/core/connect-data-platform/exasol-setup) | [Extrica](/docs/core/connect-data-platform/extrica-setup) |
| [Hive](/docs/core/connect-data-platform/hive-setup) | [IBM DB2](/docs/core/connect-data-platform/ibmdb2-setup) | [Impala](/docs/core/connect-data-platform/impala-setup) |
| [Infer](/docs/core/connect-data-platform/infer-setup) | [iomete](/docs/core/connect-data-platform/iomete-setup) | [MindsDB](/docs/core/connect-data-platform/mindsdb-setup) |
Expand Down
62 changes: 62 additions & 0 deletions website/docs/docs/core/connect-data-platform/cratedb-setup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
title: "CrateDB setup"
description: "Read this guide to learn about the CrateDB data platform setup in dbt."
id: "cratedb-setup"
meta:
maintained_by: Crate.io, Inc.
authors: 'CrateDB maintainers'
github_repo: 'crate/dbt-cratedb2'
pypi_package: 'dbt-cratedb2'
min_core_version: 'v1.0.0'
cloud_support: Not Supported
min_supported_version: 'n/a'
slack_channel_name: 'Community Forum'
slack_channel_link: 'https://community.cratedb.com/'
platform_name: 'CrateDB'
config_page: '/reference/resource-configs/no-configs'
---

import SetUpPages from '/snippets/_setup-pages-intro.md';

<SetUpPages meta={frontMatter.meta}/>


[CrateDB] is compatible with PostgreSQL, so its dbt adapter strongly depends on
dbt-postgres, documented at [PostgreSQL profile setup].

CrateDB targets are configured exactly the same way, see also [PostgreSQL
configuration], with just a few things to consider which are special to
CrateDB. Relevant details are outlined at [using dbt with CrateDB],
which also includes up-to-date information.


## Profile configuration

CrateDB targets should be set up using a configuration like this minimal sample
of settings in your [`profiles.yml`] file.

<File name='~/.dbt/profiles.yml'>

```yaml
cratedb_analytics:
target: dev
outputs:
dev:
type: cratedb
host: [clustername].aks1.westeurope.azure.cratedb.net
port: 5432
user: [username]
pass: [password]
dbname: crate # Do not change this value. CrateDB's only catalog is `crate`.
schema: doc # Define the schema name. CrateDB's default schema is `doc`.
```
</File>
[CrateDB]: https://cratedb.com/database
[PostgreSQL configuration]: https://docs.getdbt.com/reference/resource-configs/postgres-configs
[PostgreSQL profile setup]: https://docs.getdbt.com/docs/core/connect-data-platform/postgres-setup
[`profiles.yml`]: https://docs.getdbt.com/docs/core/connect-data-platform/profiles.yml
[using dbt with CrateDB]: https://cratedb.com/docs/guide/integrate/dbt/
8 changes: 1 addition & 7 deletions website/docs/docs/core/connect-data-platform/dremio-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,10 +60,6 @@ Next, configure the profile for your project.

When you initialize a project, you create one of these three profiles. You must configure it before trying to connect to Dremio Cloud or Dremio Software.

## Profiles

When you initialize a project, you create one of these three profiles. You must configure it before trying to connect to Dremio Cloud or Dremio Software.

* Profile for Dremio Cloud
* Profile for Dremio Software with Username/Password Authentication
* Profile for Dremio Software with Authentication Through a Personal Access Token
Expand Down Expand Up @@ -149,9 +145,7 @@ For descriptions of the configurations in these profiles, see [Configurations](#
</TabItem>
</Tabs>
## Configurations
### Configurations Common to Profiles for Dremio Cloud and Dremio Software
## Configurations Common to Profiles for Dremio Cloud and Dremio Software
| Configuration | Required? | Default Value | Description |
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Upgrading to v1.8 (latest)"
title: "Upgrading to v1.8"
id: upgrading-to-v1.8
description: New features and changes in dbt Core v1.8
displayed_sidebar: "docs"
Expand Down
1 change: 1 addition & 0 deletions website/docs/docs/deploy/retry-jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ If your dbt job run completed with a status of **Error**, you can rerun it from

- You have a [dbt Cloud account](https://www.getdbt.com/signup).
- You must be using [dbt version](/docs/dbt-versions/upgrade-dbt-version-in-cloud) 1.6 or newer.
- dbt can successfully parse the project and generate a [manifest](/reference/artifacts/manifest-json)
- The most recent run of the job hasn't completed successfully. The latest status of the run is **Error**.
- The job command that failed in the run must be one that supports the [retry command](/reference/commands/retry).

Expand Down
Loading

0 comments on commit 6c808bd

Please sign in to comment.