Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding configurable delimiters #4265

Merged
merged 34 commits into from
Oct 13, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
1586627
Adding delimiter
runleonarun Oct 12, 2023
9ed18a9
adding delimiter page
runleonarun Oct 12, 2023
8bd37ef
Apply suggestions from code review
runleonarun Oct 12, 2023
4ce3545
Apply suggestions from code review
runleonarun Oct 12, 2023
f233765
Apply suggestions from code review
runleonarun Oct 12, 2023
52a6916
adding examles
runleonarun Oct 12, 2023
4a75514
Update website/docs/reference/resource-configs/delimiter.md
runleonarun Oct 12, 2023
ad16abc
Update website/docs/reference/resource-configs/delimiter.md
runleonarun Oct 12, 2023
ffa3c81
Apply suggestions from code review
runleonarun Oct 12, 2023
bc8b6ec
Update website/docs/reference/resource-configs/delimiter.md
runleonarun Oct 12, 2023
6f7e161
adding examles
runleonarun Oct 12, 2023
7eacd47
Merge branch 'current' into delimiter-adding
runleonarun Oct 12, 2023
87576e6
Apply suggestions from code review
runleonarun Oct 12, 2023
3d27d14
Apply suggestions from code review
runleonarun Oct 12, 2023
bec181a
Update delimiter.md
runleonarun Oct 12, 2023
cde62d0
Update delimiter.md
runleonarun Oct 12, 2023
cd69a56
adding versioning note
runleonarun Oct 12, 2023
f4e8e71
fixing sentence
runleonarun Oct 12, 2023
111472d
addign link to migration guide
runleonarun Oct 12, 2023
03be195
fixing note title
runleonarun Oct 12, 2023
29a2306
Apply suggestions from code review
runleonarun Oct 12, 2023
b0b1607
Merge branch 'current' into delimiter-adding
runleonarun Oct 12, 2023
bc638a9
addign to examples
runleonarun Oct 12, 2023
c739349
Merge branch 'delimiter-adding' of github.com:dbt-labs/docs.getdbt.co…
runleonarun Oct 12, 2023
462f4c7
fixing wording adding csv
runleonarun Oct 12, 2023
211a847
Merge branch 'current' into delimiter-adding
matthewshaver Oct 12, 2023
e5f602f
Apply suggestions from code review
runleonarun Oct 13, 2023
3042c76
Update website/docs/reference/resource-configs/delimiter.md
runleonarun Oct 13, 2023
7ce357d
Update delimiter.md
runleonarun Oct 13, 2023
4eccbfc
Merge branch 'current' into delimiter-adding
runleonarun Oct 13, 2023
5670118
Merge branch 'current' into delimiter-adding
runleonarun Oct 13, 2023
e1283c2
Merge branch 'current' into delimiter-adding
runleonarun Oct 13, 2023
6fbffaa
Update delimiter.md
runleonarun Oct 13, 2023
03fb18a
Update delimiter.md
runleonarun Oct 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ This is a relatively small behavior change, but worth calling out in case you no

### MetricFlow enhancements

- Automatically create metrics on measures with `create_metric: true`.
- Automatically create metrics on measures with [`create_metric: true`](/docs/build/semantic-models).
- Optional [`label`](/docs/build/semantic-models) in semantic_models, measures, dimensions and entities.
- New configurations for semantic models - [enable/disable](/reference/resource-configs/enabled), [group](/reference/resource-configs/group), and [meta](/reference/resource-configs/meta).
- Support `fill_nulls_with` and `join_to_timespine` for metric nodes.
Expand All @@ -59,6 +59,6 @@ dbt Core v1.5 introduced model governance which we're continuing to refine. v1.
### Quick hits

With these quick hits, you can now:
- Configure a `delimiter` for a seed file.
- Configure a [`delimiter`](/reference/resource-configs/delimiter) for a seed file.
- Use packages with the same git repo and unique subdirectory.
- Access the `date_spine` macro directly from dbt-core (moved over from dbt-utils).
126 changes: 126 additions & 0 deletions website/docs/reference/resource-configs/delimiter.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
---
resource_types: [seeds]
datatype: <string>
default_value: ","
---

## Definition

You can use this optional seed configuration to customize how you separate values in a [seed](/docs/build/seeds) by providing the one-character string.

runleonarun marked this conversation as resolved.
Show resolved Hide resolved
* The delimiter defaults to a comma when not specified.
* Explicitly set the `delimiter` configuration value if you want seed files to use a different delimiter, such as "|" or ";".

runleonarun marked this conversation as resolved.
Show resolved Hide resolved
:::info New in 1.7!

Delimiter is new functionality available beginning with dbt Core v1.7.

:::


## Usage

Specify a delimiter in your `dbt_project.yml` file to customize the global separator for all seed values:

<File name='dbt_project.yml'>

```yml
seeds:
<project_name>:
+delimiter: "|" # default project delimiter for seeds will be "|"
<seed_subdirectory>:
+delimiter: "," # delimiter for seeds in seed_subdirectory will be ","
```

</File>

runleonarun marked this conversation as resolved.
Show resolved Hide resolved

Or use a custom delimiter to override the values for a specific seed:

<File name='seeds/properties.yml'>

```yml
version: 2

seeds:
- name: <seed_name>
config:
delimiter: "|"
```

</File>

## Examples
For a project with:

* `name: jaffle_shop` in the `dbt_project.yml` file
* `seed-paths: ["seeds"]` in the `dbt_project.yml` file

### Use a custom delimiter to override global values

You can set a default behavior for all seeds with an exception for one seed, `seed_a`, which uses a comma:

<File name='dbt_project.yml'>

```yml
seeds:
jaffle_shop:
+delimiter: "|" # default delimiter for seeds in jaffle_shop project will be "|"
seed_a:
+delimiter: "," # delimiter for seed_a will be ","
```

</File>

Your corresponding seed files would be formatted like this:

<File name='seeds/my_seed.csv'>

```text
col_a|col_b|col_c
1|2|3
4|5|6
...
```

</File>

<File name='seeds/seed_a.csv'>

```text
name,id
luna,1
doug,2
...
```

</File>

Or you can configure custom behavior for one seed. The `country_codes` uses the ";" delimiter:

<File name='seeds/properties.yml'>

```yml
version: 2

seeds:
- name: country_codes
config:
delimiter: ";"
```

</File>

The `country_codes` seed file would be formatted like this:

runleonarun marked this conversation as resolved.
Show resolved Hide resolved
<File name='seeds/country_codes.csv'>

```text
country_code;country_name
US;United States
CA;Canada
GB;United Kingdom
...
```

</File>
2 changes: 2 additions & 0 deletions website/docs/reference/seed-configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ seeds:
[<resource-path>](/reference/resource-configs/resource-path):
[+](/reference/resource-configs/plus-prefix)[quote_columns](/reference/resource-configs/quote_columns): true | false
[+](/reference/resource-configs/plus-prefix)[column_types](/reference/resource-configs/column_types): {column_name: datatype}
[+](/reference/resource-configs/plus-prefix)[delimiter](/reference/resource-configs/delimiter): <string>

```

Expand All @@ -43,6 +44,7 @@ seeds:
config:
[quote_columns](/reference/resource-configs/quote_columns): true | false
[column_types](/reference/resource-configs/column_types): {column_name: datatype}
[delimiter](/reference/resource-configs/grants): <string>

```

Expand Down
1 change: 1 addition & 0 deletions website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -694,6 +694,7 @@ const sidebarSettings = {
"reference/seed-properties",
"reference/seed-configs",
"reference/resource-configs/column_types",
"reference/resource-configs/delimiter",
"reference/resource-configs/quote_columns",
],
},
Expand Down
Loading