From 1586627980ab6fb57be57dc6ffcdfd0941eb64e8 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Wed, 11 Oct 2023 21:27:37 -0700 Subject: [PATCH 01/27] Adding delimiter --- website/docs/reference/seed-configs.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/website/docs/reference/seed-configs.md b/website/docs/reference/seed-configs.md index d74f414cbfe..429aa9444ae 100644 --- a/website/docs/reference/seed-configs.md +++ b/website/docs/reference/seed-configs.md @@ -23,6 +23,7 @@ seeds: [](/reference/resource-configs/resource-path): [+](/reference/resource-configs/plus-prefix)[quote_columns](/reference/resource-configs/quote_columns): true | false [+](/reference/resource-configs/plus-prefix)[column_types](/reference/resource-configs/column_types): {column_name: datatype} + [+](/reference/resource-configs/plus-prefix)[delimiter](/reference/resource-configs/delimiter): ``` @@ -43,6 +44,7 @@ seeds: config: [quote_columns](/reference/resource-configs/quote_columns): true | false [column_types](/reference/resource-configs/column_types): {column_name: datatype} + [delimiter](/reference/resource-configs/grants): ``` From 9ed18a98b15bafc1bdcf318de6565f672c8986ed Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Wed, 11 Oct 2023 21:52:54 -0700 Subject: [PATCH 02/27] adding delimiter page --- .../reference/resource-configs/delimiter.md | 88 +++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 website/docs/reference/resource-configs/delimiter.md diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md new file mode 100644 index 00000000000..654f5f07aed --- /dev/null +++ b/website/docs/reference/resource-configs/delimiter.md @@ -0,0 +1,88 @@ +--- +resource_types: [seeds] +datatype: +--- + +## Description + +Optionally specify a delimiter for [seed](/docs/build/seeds), by providing a string value. The defaults to comma. + +Without specifying this, dbt will infer the datatype based on the column values in your seed file. + +## Usage +Specify column types in your `dbt_project.yml` file: + + + +```yml +seeds: + jaffle_shop: + country_codes: + +column_types: + country_code: varchar(2) + country_name: varchar(32) + +delimiter: "," +``` + + + + + +Or: + + + +```yml +version: 2 + +seeds: + - name: country_codes + config: + column_types: + country_code: varchar(2) + country_name: varchar(32) + delimiter: "," +``` + + + +If you have previously run `dbt seed`, you'll need to run `dbt seed --full-refresh` for the changes to take effect. + +Note that you will need to use the fully directory path of a seed when configuring `delimiter`. For example, for a seed file at `seeds/marketing/utm_mappings.csv`, you will need to configure it like so: + + + +```yml +seeds: + jaffle_shop: + marketing: + utm_mappings: + +delimiter: + ... + +``` + + + +## Examples + +### Use a varchar column type to preserve leading zeros in a zipcode +(Note: preservation of leading zeros works for v0.16.0 onwards) + + +```yml +seeds: + jaffle_shop: # you must include the project name + warehouse_locations: + +delimiter: "," +``` + + + +## Recommendation + +Use this configuration only when required. Otherwise you can omit this configuration. + +## Troubleshooting + +The `delimiter` configuration should contain a value when set and not be empty. This value should not be set to a semicolon. From 8bd37ef88cfbad0ab9c61d5783e9e1376d7cb1b8 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Wed, 11 Oct 2023 22:02:02 -0700 Subject: [PATCH 03/27] Apply suggestions from code review --- website/docs/reference/resource-configs/delimiter.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 654f5f07aed..6eacf0b4b0e 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -5,9 +5,8 @@ datatype: ## Description -Optionally specify a delimiter for [seed](/docs/build/seeds), by providing a string value. The defaults to comma. +Optionally specify a custom delimiter for [seed](/docs/build/seeds) by providing a string value. The delimiter defaults to comma. -Without specifying this, dbt will infer the datatype based on the column values in your seed file. ## Usage Specify column types in your `dbt_project.yml` file: From 4ce3545bdf8d7c46081cb0748a5442d2bf9e18d9 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Wed, 11 Oct 2023 22:10:29 -0700 Subject: [PATCH 04/27] Apply suggestions from code review --- website/docs/reference/resource-configs/delimiter.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 6eacf0b4b0e..f51e2c5743f 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -84,4 +84,4 @@ Use this configuration only when required. Otherwise you can omit this configura ## Troubleshooting -The `delimiter` configuration should contain a value when set and not be empty. This value should not be set to a semicolon. +The `delimiter` configuration should contain a value when set and not be empty. From f233765bce3dc21d3984637098524ce512946ce9 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Wed, 11 Oct 2023 22:17:46 -0700 Subject: [PATCH 05/27] Apply suggestions from code review --- website/docs/reference/resource-configs/delimiter.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index f51e2c5743f..4d9bf46f444 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -65,8 +65,7 @@ seeds: ## Examples -### Use a varchar column type to preserve leading zeros in a zipcode -(Note: preservation of leading zeros works for v0.16.0 onwards) +### Use a comma delimiter ```yml From 52a69161cbd15793d5a3004335e85c063a009ef0 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Wed, 11 Oct 2023 22:36:32 -0700 Subject: [PATCH 06/27] adding examles --- .../reference/resource-configs/delimiter.md | 61 ++++++++----------- 1 file changed, 26 insertions(+), 35 deletions(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 4d9bf46f444..cd06c92a427 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -1,26 +1,23 @@ --- resource_types: [seeds] datatype: +default_value: "," --- -## Description +## Definition -Optionally specify a custom delimiter for [seed](/docs/build/seeds) by providing a string value. The delimiter defaults to comma. +An optional seed configuration, used to delimit [seed](/docs/build/seeds) by providing a string value. The delimiter defaults to comma. ## Usage -Specify column types in your `dbt_project.yml` file: +Specify delimiter in your `dbt_project.yml` file: ```yml seeds: - jaffle_shop: - country_codes: - +column_types: - country_code: varchar(2) - country_name: varchar(32) - +delimiter: "," + +quote_columns: False + +delimiter: "," ``` @@ -35,52 +32,46 @@ Or: version: 2 seeds: - - name: country_codes - config: - column_types: - country_code: varchar(2) - country_name: varchar(32) - delimiter: "," + quote_columns: False + delimiter: "," ``` -If you have previously run `dbt seed`, you'll need to run `dbt seed --full-refresh` for the changes to take effect. +## Examples +For a project with: -Note that you will need to use the fully directory path of a seed when configuring `delimiter`. For example, for a seed file at `seeds/marketing/utm_mappings.csv`, you will need to configure it like so: +* `name: jaffle_shop` in the `dbt_project.yml` file +* `seed-paths: ["seeds"]` in the `dbt_project.yml` file +### Use a comma delimiter ```yml seeds: - jaffle_shop: - marketing: - utm_mappings: - +delimiter: - ... - + jaffle_shop: # you must include the project name + mappings: + +delimiter: "," ``` -## Examples +Or -### Use a comma delimiter - + ```yml +version: 2 + seeds: - jaffle_shop: # you must include the project name - warehouse_locations: - +delimiter: "," + - name: mappings + config: + delimiter: "," ``` -## Recommendation - -Use this configuration only when required. Otherwise you can omit this configuration. - -## Troubleshooting +## Recommended configuration -The `delimiter` configuration should contain a value when set and not be empty. +* Explicitly set this value if using seed files. +* The `delimiter` configuration should contain a value and not be empty. From 4a75514f413bed327f916d06b90ff3a20410ff16 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Wed, 11 Oct 2023 22:40:49 -0700 Subject: [PATCH 07/27] Update website/docs/reference/resource-configs/delimiter.md --- website/docs/reference/resource-configs/delimiter.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index cd06c92a427..3b67746efc1 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -6,7 +6,7 @@ default_value: "," ## Definition -An optional seed configuration, used to delimit [seed](/docs/build/seeds) by providing a string value. The delimiter defaults to comma. +An optional seed configuration used to separate values in a [seed](/docs/build/seeds) with a string you provide. The delimiter defaults to comma. ## Usage From ad16abc8a1d7c782a26696052611b411c89e7794 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Wed, 11 Oct 2023 22:41:26 -0700 Subject: [PATCH 08/27] Update website/docs/reference/resource-configs/delimiter.md --- website/docs/reference/resource-configs/delimiter.md | 1 - 1 file changed, 1 deletion(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 3b67746efc1..9ec789c771e 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -23,7 +23,6 @@ seeds: - Or: From ffa3c81306a4cfc05ec8e4382d42a2ee02b04a5b Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Wed, 11 Oct 2023 22:44:05 -0700 Subject: [PATCH 09/27] Apply suggestions from code review --- website/docs/reference/resource-configs/delimiter.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 9ec789c771e..6f0933fb66f 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -10,7 +10,7 @@ An optional seed configuration used to separate values in a [seed](/docs/build/s ## Usage -Specify delimiter in your `dbt_project.yml` file: +Specify delimiter in your `dbt_project.yml` file to globally separate all seed values: From bc8b6ec7442887b451c0545f435237730c814113 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Wed, 11 Oct 2023 22:49:29 -0700 Subject: [PATCH 10/27] Update website/docs/reference/resource-configs/delimiter.md --- website/docs/reference/resource-configs/delimiter.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 6f0933fb66f..a6698198c1f 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -23,7 +23,7 @@ seeds: -Or: +Or only separate values with a delimiter in the seeds/mappings directory: From 6f7e161e55396db829a7bb565e6dce1b4c67e35b Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Wed, 11 Oct 2023 22:57:05 -0700 Subject: [PATCH 11/27] adding examles --- website/docs/reference/resource-configs/delimiter.md | 2 +- website/sidebars.js | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index a6698198c1f..c5378cf59ec 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -24,9 +24,9 @@ seeds: Or only separate values with a delimiter in the seeds/mappings directory: - + ```yml version: 2 diff --git a/website/sidebars.js b/website/sidebars.js index 538575ed0f8..3e9b3379396 100644 --- a/website/sidebars.js +++ b/website/sidebars.js @@ -693,6 +693,7 @@ const sidebarSettings = { "reference/seed-properties", "reference/seed-configs", "reference/resource-configs/column_types", + "reference/resource-configs/delimiter", "reference/resource-configs/quote_columns", ], }, From 87576e6ce96ea09a08ad6b53a2eecfe0d109e700 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Thu, 12 Oct 2023 08:09:37 -0700 Subject: [PATCH 12/27] Apply suggestions from code review Co-authored-by: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Co-authored-by: Grace Goheen <53586774+graciegoheen@users.noreply.github.com> --- website/docs/reference/resource-configs/delimiter.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index c5378cf59ec..2fb57a9378f 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -9,15 +9,18 @@ default_value: "," An optional seed configuration used to separate values in a [seed](/docs/build/seeds) with a string you provide. The delimiter defaults to comma. +* The delimiter defaults to a comma. +* Explicitly set this value if you want seed files to use a different delimiter (default is comma). +* The `delimiter` configuration should contain a value and not be empty. ## Usage -Specify delimiter in your `dbt_project.yml` file to globally separate all seed values: +Specify a delimiter in your `dbt_project.yml` file to override the global separator for all seed values: ```yml seeds: +quote_columns: False - +delimiter: "," + +delimiter: ";" ``` @@ -70,7 +73,5 @@ seeds: -## Recommended configuration -* Explicitly set this value if using seed files. * The `delimiter` configuration should contain a value and not be empty. From 3d27d14fe6077b57a76dd9572ddf4c3fe48f70ea Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Thu, 12 Oct 2023 08:09:54 -0700 Subject: [PATCH 13/27] Apply suggestions from code review Co-authored-by: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> --- website/docs/reference/resource-configs/delimiter.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 2fb57a9378f..86432b6498d 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -6,7 +6,7 @@ default_value: "," ## Definition -An optional seed configuration used to separate values in a [seed](/docs/build/seeds) with a string you provide. The delimiter defaults to comma. +An optional seed configuration used to separate values in a [seed](/docs/build/seeds) with a string you provide. * The delimiter defaults to a comma. From bec181afbb703c2de03ea9cda3683dbf249fb954 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Thu, 12 Oct 2023 09:26:33 -0700 Subject: [PATCH 14/27] Update delimiter.md --- .../reference/resource-configs/delimiter.md | 26 ++++++++++--------- 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 86432b6498d..53f01e80579 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -9,11 +9,13 @@ default_value: "," An optional seed configuration used to separate values in a [seed](/docs/build/seeds) with a string you provide. -* The delimiter defaults to a comma. -* Explicitly set this value if you want seed files to use a different delimiter (default is comma). -* The `delimiter` configuration should contain a value and not be empty. +* The delimiter defaults to a comma when not speicfied. +* Explicitly set The `delimiter` configuration value if you want seed files to use a different delimiter, such as "|" or ";" (default is comma). + + ## Usage -Specify a delimiter in your `dbt_project.yml` file to override the global separator for all seed values: + +Specify a custom delimiter in your `dbt_project.yml` file to override the global separator for all seed values: @@ -26,7 +28,8 @@ seeds: -Or only separate values with a delimiter in the seeds/mappings directory: +Or use a custom delimiter to overrid the values for a specific seed: + @@ -46,14 +49,16 @@ For a project with: * `name: jaffle_shop` in the `dbt_project.yml` file * `seed-paths: ["seeds"]` in the `dbt_project.yml` file -### Use a comma delimiter +### Use a custom delimiter + ```yml seeds: - jaffle_shop: # you must include the project name + jaffle_shop: + +delimiter: "|" mappings: - +delimiter: "," + +delimiter: ";" ``` @@ -68,10 +73,7 @@ version: 2 seeds: - name: mappings config: - delimiter: "," + delimiter: "|" ``` - - -* The `delimiter` configuration should contain a value and not be empty. From cde62d08995311698dfa04c2ef44c58a881895b2 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Thu, 12 Oct 2023 09:54:20 -0700 Subject: [PATCH 15/27] Update delimiter.md --- .../reference/resource-configs/delimiter.md | 27 ++++++++++--------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 53f01e80579..20d2a4d93dc 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -6,8 +6,7 @@ default_value: "," ## Definition -An optional seed configuration used to separate values in a [seed](/docs/build/seeds) with a string you provide. - +An optional seed configuration used to customize how you separate values in a [seed](/docs/build/seeds) with the one-character string you provide. * The delimiter defaults to a comma when not speicfied. * Explicitly set The `delimiter` configuration value if you want seed files to use a different delimiter, such as "|" or ";" (default is comma). @@ -21,24 +20,26 @@ Specify a custom delimiter in your `dbt_project.yml` file to override the global ```yml seeds: - +quote_columns: False - +delimiter: ";" + : + +delimiter: "|" + : + +delimiter: "," ``` -Or use a custom delimiter to overrid the values for a specific seed: +Or use a custom delimiter to override the values for a specific seed: - ```yml version: 2 seeds: - quote_columns: False - delimiter: "," + - name: + config: + delimiter: "|" ``` @@ -49,7 +50,7 @@ For a project with: * `name: jaffle_shop` in the `dbt_project.yml` file * `seed-paths: ["seeds"]` in the `dbt_project.yml` file -### Use a custom delimiter +### Use a custom delimiter to overrride global values @@ -57,13 +58,13 @@ For a project with: seeds: jaffle_shop: +delimiter: "|" - mappings: - +delimiter: ";" + seed_a: + +delimiter: "," ``` -Or +Or for certain seeds: @@ -71,7 +72,7 @@ Or version: 2 seeds: - - name: mappings + - name: seed_a config: delimiter: "|" ``` From cd69a56a7efd8c4c8e5a5052dddcd0ee006c8481 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Thu, 12 Oct 2023 10:43:47 -0700 Subject: [PATCH 16/27] adding versioning note --- website/docs/reference/resource-configs/delimiter.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 20d2a4d93dc..883501e6f9f 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -8,9 +8,15 @@ default_value: "," An optional seed configuration used to customize how you separate values in a [seed](/docs/build/seeds) with the one-character string you provide. -* The delimiter defaults to a comma when not speicfied. +* The delimiter defaults to a comma when not specified. * Explicitly set The `delimiter` configuration value if you want seed files to use a different delimiter, such as "|" or ";" (default is comma). +:::info New to dbt Core 1.7 + +Delimiter is new functionality available in dbt Core v1.7+ + +::: + ## Usage From f4e8e71a7dc93689247c7589aefa2ec7bbfce3d4 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Thu, 12 Oct 2023 10:45:25 -0700 Subject: [PATCH 17/27] fixing sentence --- website/docs/reference/resource-configs/delimiter.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 883501e6f9f..65705421984 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -13,7 +13,7 @@ An optional seed configuration used to customize how you separate values in a [s :::info New to dbt Core 1.7 -Delimiter is new functionality available in dbt Core v1.7+ +Delimiter is new functionality available beginning with dbt Core v1.7. ::: From 111472d95dfd6eb33d040b1be86b9095de2e2f9f Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Thu, 12 Oct 2023 10:54:31 -0700 Subject: [PATCH 18/27] addign link to migration guide --- website/docs/guides/migration/versions/00-upgrading-to-v1.7.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md index ba079e6a0fb..ac9af75f7d8 100644 --- a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md +++ b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md @@ -47,6 +47,6 @@ dbt Core v1.5 introduced model governance which we're continuing to refine. v1. ### Quick hits With these quick hits, you can now: -- Configure a `delimiter` for a seed file. +- Configure a [`delimiter`](/reference/resource-configs/delimiter) for a seed file. - Use packages with the same git repo and unique subdirectory. - Moved the `date_spine` macro from dbt-utils to dbt-core. From 03be195df549d05243651fc6e7bd409ccb036c91 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Thu, 12 Oct 2023 10:57:28 -0700 Subject: [PATCH 19/27] fixing note title --- website/docs/reference/resource-configs/delimiter.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 65705421984..0f2645130b6 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -11,7 +11,7 @@ An optional seed configuration used to customize how you separate values in a [s * The delimiter defaults to a comma when not specified. * Explicitly set The `delimiter` configuration value if you want seed files to use a different delimiter, such as "|" or ";" (default is comma). -:::info New to dbt Core 1.7 +:::info New in 1.7! Delimiter is new functionality available beginning with dbt Core v1.7. From 29a2306fad78372f460b1dd32cefb2c8d3b4f0df Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Thu, 12 Oct 2023 12:26:59 -0700 Subject: [PATCH 20/27] Apply suggestions from code review Co-authored-by: Grace Goheen <53586774+graciegoheen@users.noreply.github.com> --- .../docs/reference/resource-configs/delimiter.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 0f2645130b6..5faeddf1ef3 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -9,7 +9,7 @@ default_value: "," An optional seed configuration used to customize how you separate values in a [seed](/docs/build/seeds) with the one-character string you provide. * The delimiter defaults to a comma when not specified. -* Explicitly set The `delimiter` configuration value if you want seed files to use a different delimiter, such as "|" or ";" (default is comma). +* Explicitly set The `delimiter` configuration value if you want seed files to use a different delimiter, such as "|" or ";". :::info New in 1.7! @@ -27,9 +27,9 @@ Specify a custom delimiter in your `dbt_project.yml` file to override the global ```yml seeds: : - +delimiter: "|" + +delimiter: "|" # default project delimiter for seeds will be "|" : - +delimiter: "," + +delimiter: "," # delimiter for seeds in seed_directory_name will be "," ``` @@ -63,9 +63,9 @@ For a project with: ```yml seeds: jaffle_shop: - +delimiter: "|" + +delimiter: "|" # default delimiter for seeds in jaffle_shop project will be "|" seed_a: - +delimiter: "," + +delimiter: "," # delimiter for seed_a will be "," ``` @@ -78,9 +78,9 @@ Or for certain seeds: version: 2 seeds: - - name: seed_a + - name: seed_b config: - delimiter: "|" + delimiter: ";" ``` From bc638a9cfa18b1362ef034031c1c6a59ef168906 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Thu, 12 Oct 2023 13:49:18 -0700 Subject: [PATCH 21/27] addign to examples --- .../versions/00-upgrading-to-v1.7.md | 2 +- .../reference/resource-configs/delimiter.md | 28 +++++++++++++++---- 2 files changed, 23 insertions(+), 7 deletions(-) diff --git a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md index ac9af75f7d8..b3463f39cbc 100644 --- a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md +++ b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md @@ -22,7 +22,7 @@ dbt Labs is committed to providing backward compatibility for all versions 1.x, ### MetricFlow enhancements -- Automatically create metrics on measures with `create_metric: true`. +- Automatically create metrics on measures with [`create_metric: true`](/docs/build/semantic-models). - Optional [`label`](/docs/build/semantic-models) in semantic_models, measures, dimensions and entities. - New configurations for semantic models - [enable/disable](/reference/resource-configs/enabled), [group](/reference/resource-configs/group), and [meta](/reference/resource-configs/meta). - Support `fill_nulls_with` and `join_to_timespine` for metric nodes. diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 5faeddf1ef3..82958852358 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -28,8 +28,8 @@ Specify a custom delimiter in your `dbt_project.yml` file to override the global seeds: : +delimiter: "|" # default project delimiter for seeds will be "|" - : - +delimiter: "," # delimiter for seeds in seed_directory_name will be "," + : + +delimiter: "," # delimiter for seeds in seed_name will be "," ``` @@ -43,7 +43,7 @@ Or use a custom delimiter to override the values for a specific seed: version: 2 seeds: - - name: + - name: config: delimiter: "|" ``` @@ -56,7 +56,9 @@ For a project with: * `name: jaffle_shop` in the `dbt_project.yml` file * `seed-paths: ["seeds"]` in the `dbt_project.yml` file -### Use a custom delimiter to overrride global values +### Use a custom delimiter to override global values + +You can make the default delimiter "|" for all seeds but have seed_a still use "," like this: @@ -70,7 +72,7 @@ seeds: -Or for certain seeds: +Or you can configure the country_codes seed to use the "|" delimiter: @@ -78,9 +80,23 @@ Or for certain seeds: version: 2 seeds: - - name: seed_b + - name: country_codes config: delimiter: ";" ``` + +Which would result in a CSV that looks like this: + + + +```text +country_code|country_name +US|United States +CA|Canada +GB|United Kingdom +... +``` + + From 462f4c711446e33a2d0a90395a4c9b5e11b52a66 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Thu, 12 Oct 2023 14:02:12 -0700 Subject: [PATCH 22/27] fixing wording adding csv --- website/docs/reference/resource-configs/delimiter.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 82958852358..a15acc36f7c 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -20,7 +20,7 @@ Delimiter is new functionality available beginning with dbt Core v1.7. ## Usage -Specify a custom delimiter in your `dbt_project.yml` file to override the global separator for all seed values: +Specify a delimiter in your `dbt_project.yml` file to customize the global separator for all seed values: @@ -58,7 +58,7 @@ For a project with: ### Use a custom delimiter to override global values -You can make the default delimiter "|" for all seeds but have seed_a still use "," like this: +You can set a default behavior for all seeds with an exception for one seed, `seed_a`, which uses a comma: @@ -72,7 +72,7 @@ seeds: -Or you can configure the country_codes seed to use the "|" delimiter: +Or you can configure custom behavior for one seed. The `country_codes` uses the "|" delimiter: @@ -87,7 +87,7 @@ seeds: -Which would result in a CSV that looks like this: +And results in a CSV like this: From e5f602f07753e4ea7df25886552f6e4c0d2fee88 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Fri, 13 Oct 2023 09:10:50 -0700 Subject: [PATCH 23/27] Apply suggestions from code review Co-authored-by: Grace Goheen <53586774+graciegoheen@users.noreply.github.com> --- .../reference/resource-configs/delimiter.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index a15acc36f7c..f3b3fff4df0 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -9,7 +9,7 @@ default_value: "," An optional seed configuration used to customize how you separate values in a [seed](/docs/build/seeds) with the one-character string you provide. * The delimiter defaults to a comma when not specified. -* Explicitly set The `delimiter` configuration value if you want seed files to use a different delimiter, such as "|" or ";". +* Explicitly set the `delimiter` configuration value if you want seed files to use a different delimiter, such as "|" or ";". :::info New in 1.7! @@ -28,8 +28,8 @@ Specify a delimiter in your `dbt_project.yml` file to customize the global separ seeds: : +delimiter: "|" # default project delimiter for seeds will be "|" - : - +delimiter: "," # delimiter for seeds in seed_name will be "," + : + +delimiter: "," # delimiter for seeds in seed_subdirectory will be "," ``` @@ -72,7 +72,7 @@ seeds: -Or you can configure custom behavior for one seed. The `country_codes` uses the "|" delimiter: +Or you can configure custom behavior for one seed. The `country_codes` uses the ";" delimiter: @@ -87,15 +87,15 @@ seeds: -And results in a CSV like this: +Your corresponding seed files would be formatted like so: ```text -country_code|country_name -US|United States -CA|Canada -GB|United Kingdom +country_code;country_name +US;United States +CA;Canada +GB;United Kingdom ... ``` From 3042c76ffc8b0f8ea23020dca3c710a972d6cbc0 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Fri, 13 Oct 2023 09:11:43 -0700 Subject: [PATCH 24/27] Update website/docs/reference/resource-configs/delimiter.md Co-authored-by: Grace Goheen <53586774+graciegoheen@users.noreply.github.com> --- website/docs/reference/resource-configs/delimiter.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index f3b3fff4df0..e40e164f6a2 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -65,7 +65,7 @@ You can set a default behavior for all seeds with an exception for one seed, `se ```yml seeds: jaffle_shop: - +delimiter: "|" # default delimiter for seeds in jaffle_shop project will be "|" + +delimiter: "|" # default delimiter for seeds in jaffle_shop project will be "|" seed_a: +delimiter: "," # delimiter for seed_a will be "," ``` From 7ce357d10d26e8589265f2128fcc1d2220e523a7 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Fri, 13 Oct 2023 09:38:10 -0700 Subject: [PATCH 25/27] Update delimiter.md --- .../reference/resource-configs/delimiter.md | 22 +++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index e40e164f6a2..317358420a5 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -89,6 +89,28 @@ seeds: Your corresponding seed files would be formatted like so: + + +```text +col_a|col_b|col_c +1|2|3 +4|5|6 +... +``` + + + + + +```text +name,id +luna,1 +doug,2 +... +``` + + + ```text From 6fbffaaa80a4754ee2da3fe93e8a876d7ba84759 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Fri, 13 Oct 2023 16:14:07 -0700 Subject: [PATCH 26/27] Update delimiter.md --- .../reference/resource-configs/delimiter.md | 36 ++++++++++--------- 1 file changed, 19 insertions(+), 17 deletions(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 317358420a5..31782c8cccb 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -72,22 +72,7 @@ seeds: -Or you can configure custom behavior for one seed. The `country_codes` uses the ";" delimiter: - - - -```yml -version: 2 - -seeds: - - name: country_codes - config: - delimiter: ";" -``` - - - -Your corresponding seed files would be formatted like so: +Your corresponding seed files would be formatted like this: @@ -100,7 +85,7 @@ col_a|col_b|col_c - + ```text name,id @@ -111,6 +96,23 @@ doug,2 +Or you can configure custom behavior for one seed. The `country_codes` uses the ";" delimiter: + + + +```yml +version: 2 + +seeds: + - name: country_codes + config: + delimiter: ";" +``` + + + +The `country_codes` seed file would be formatted like this: + ```text From 03fb18acdbf74c10bb8c0a8bbd4db8f93a816131 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Fri, 13 Oct 2023 16:16:03 -0700 Subject: [PATCH 27/27] Update delimiter.md --- website/docs/reference/resource-configs/delimiter.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md index 31782c8cccb..58d6ba8344a 100644 --- a/website/docs/reference/resource-configs/delimiter.md +++ b/website/docs/reference/resource-configs/delimiter.md @@ -6,7 +6,7 @@ default_value: "," ## Definition -An optional seed configuration used to customize how you separate values in a [seed](/docs/build/seeds) with the one-character string you provide. +You can use this optional seed configuration to customize how you separate values in a [seed](/docs/build/seeds) by providing the one-character string. * The delimiter defaults to a comma when not specified. * Explicitly set the `delimiter` configuration value if you want seed files to use a different delimiter, such as "|" or ";".