Skip to content

Commit

Permalink
Adding configurable delimiters (#4265)
Browse files Browse the repository at this point in the history
Closes #4126 
Closes #4273

## What are you changing in this pull request and why?

* Adding delimiter to https://docs.getdbt.com/reference/seed-configs
* Adding new
https://docs.getdbt.com/reference/resource-configs/delimiter

## Checklist
<!--
Uncomment if you're publishing docs for a prerelease version of dbt
(delete if not applicable):
- [ ] Add versioning components, as described in [Versioning
Docs](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-entire-pages)
- [ ] Add a note to the prerelease version [Migration
Guide](https://github.com/dbt-labs/docs.getdbt.com/tree/current/website/docs/guides/migration/versions)
-->
- [ ] Review the [Content style
guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md)
and [About
versioning](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version)
so my content adheres to these guidelines.
- [ ] Add a checklist item for anything that needs to happen before this
PR is merged, such as "needs technical review" or "change base branch."

Adding new pages (delete if not applicable):
- [ ] Add page to `website/sidebars.js`
- [ ] Provide a unique filename for the new page

Removing or renaming existing pages (delete if not applicable):
- [ ] Remove page from `website/sidebars.js`
- [ ] Add an entry `website/static/_redirects`
- [ ] [Ran link
testing](https://github.com/dbt-labs/docs.getdbt.com#running-the-cypress-tests-locally)
to update the links that point to the deleted page

---------

Co-authored-by: mirnawong1 <[email protected]>
Co-authored-by: Grace Goheen <[email protected]>
Co-authored-by: Matt Shaver <[email protected]>
  • Loading branch information
4 people authored Oct 13, 2023
1 parent 5b00716 commit 82bc1df
Show file tree
Hide file tree
Showing 4 changed files with 131 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ This is a relatively small behavior change, but worth calling out in case you no

### MetricFlow enhancements

- Automatically create metrics on measures with `create_metric: true`.
- Automatically create metrics on measures with [`create_metric: true`](/docs/build/semantic-models).
- Optional [`label`](/docs/build/semantic-models) in semantic_models, measures, dimensions and entities.
- New configurations for semantic models - [enable/disable](/reference/resource-configs/enabled), [group](/reference/resource-configs/group), and [meta](/reference/resource-configs/meta).
- Support `fill_nulls_with` and `join_to_timespine` for metric nodes.
Expand All @@ -59,6 +59,6 @@ dbt Core v1.5 introduced model governance which we're continuing to refine. v1.
### Quick hits

With these quick hits, you can now:
- Configure a `delimiter` for a seed file.
- Configure a [`delimiter`](/reference/resource-configs/delimiter) for a seed file.
- Use packages with the same git repo and unique subdirectory.
- Access the `date_spine` macro directly from dbt-core (moved over from dbt-utils).
126 changes: 126 additions & 0 deletions website/docs/reference/resource-configs/delimiter.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
---
resource_types: [seeds]
datatype: <string>
default_value: ","
---

## Definition

You can use this optional seed configuration to customize how you separate values in a [seed](/docs/build/seeds) by providing the one-character string.

* The delimiter defaults to a comma when not specified.
* Explicitly set the `delimiter` configuration value if you want seed files to use a different delimiter, such as "|" or ";".

:::info New in 1.7!

Delimiter is new functionality available beginning with dbt Core v1.7.

:::


## Usage

Specify a delimiter in your `dbt_project.yml` file to customize the global separator for all seed values:

<File name='dbt_project.yml'>

```yml
seeds:
<project_name>:
+delimiter: "|" # default project delimiter for seeds will be "|"
<seed_subdirectory>:
+delimiter: "," # delimiter for seeds in seed_subdirectory will be ","
```
</File>
Or use a custom delimiter to override the values for a specific seed:
<File name='seeds/properties.yml'>
```yml
version: 2

seeds:
- name: <seed_name>
config:
delimiter: "|"
```
</File>
## Examples
For a project with:
* `name: jaffle_shop` in the `dbt_project.yml` file
* `seed-paths: ["seeds"]` in the `dbt_project.yml` file

### Use a custom delimiter to override global values

You can set a default behavior for all seeds with an exception for one seed, `seed_a`, which uses a comma:

<File name='dbt_project.yml'>

```yml
seeds:
jaffle_shop:
+delimiter: "|" # default delimiter for seeds in jaffle_shop project will be "|"
seed_a:
+delimiter: "," # delimiter for seed_a will be ","
```

</File>

Your corresponding seed files would be formatted like this:

<File name='seeds/my_seed.csv'>

```text
col_a|col_b|col_c
1|2|3
4|5|6
...
```

</File>

<File name='seeds/seed_a.csv'>

```text
name,id
luna,1
doug,2
...
```

</File>

Or you can configure custom behavior for one seed. The `country_codes` uses the ";" delimiter:

<File name='seeds/properties.yml'>

```yml
version: 2
seeds:
- name: country_codes
config:
delimiter: ";"
```

</File>

The `country_codes` seed file would be formatted like this:

<File name='seeds/country_codes.csv'>

```text
country_code;country_name
US;United States
CA;Canada
GB;United Kingdom
...
```

</File>
2 changes: 2 additions & 0 deletions website/docs/reference/seed-configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ seeds:
[<resource-path>](/reference/resource-configs/resource-path):
[+](/reference/resource-configs/plus-prefix)[quote_columns](/reference/resource-configs/quote_columns): true | false
[+](/reference/resource-configs/plus-prefix)[column_types](/reference/resource-configs/column_types): {column_name: datatype}
[+](/reference/resource-configs/plus-prefix)[delimiter](/reference/resource-configs/delimiter): <string>

```

Expand All @@ -43,6 +44,7 @@ seeds:
config:
[quote_columns](/reference/resource-configs/quote_columns): true | false
[column_types](/reference/resource-configs/column_types): {column_name: datatype}
[delimiter](/reference/resource-configs/grants): <string>

```

Expand Down
1 change: 1 addition & 0 deletions website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -694,6 +694,7 @@ const sidebarSettings = {
"reference/seed-properties",
"reference/seed-configs",
"reference/resource-configs/column_types",
"reference/resource-configs/delimiter",
"reference/resource-configs/quote_columns",
],
},
Expand Down

0 comments on commit 82bc1df

Please sign in to comment.