Add slugify macro to utils, use in pivot macro

CHANGELOG.md

@@ -3,6 +3,10 @@
 
 ## Features
 * Add new `accepted_range` test ([#276](https://github.com/fishtown-analytics/dbt-utils/pull/276) [@joellabes](https://github.com/joellabes))
+* Add `slugify` macro, and use it in the pivot macro. :rotating_light: This macro uses the `re` module, which is only available in dbt v0.19.0+. As a result, this feature introduces a breaking change. ([#314](https://github.com/fishtown-analytics/dbt-utils/pull/314))
+
+## Quality of life
+* Rename "logger" collection of macros to "jinja helpers"
 
 # dbt-utils v0.6.2
 

README.md

@@ -95,7 +95,7 @@ Usage:
 ---
 ### Date/Time
 #### date_spine ([source](macros/datetime/date_spine.sql))
-This macro returns the sql required to build a date spine. The spine will include the `start_date` (if it is aligned to the `datepart`), but it will not include the `end_date`. 
+This macro returns the sql required to build a date spine. The spine will include the `start_date` (if it is aligned to the `datepart`), but it will not include the `end_date`.
 
 Usage:
 ```
@@ -433,9 +433,9 @@ constraint needs to be verified.
 
 
 #### accepted_range ([source](macros/schema_tests/accepted_range.sql))
-This test checks that a column's values fall inside an expected range. Any combination of `min_value` and `max_value` is allowed, and the range can be inclusive or exclusive. Provide a `where` argument to filter to specific records only. 
+This test checks that a column's values fall inside an expected range. Any combination of `min_value` and `max_value` is allowed, and the range can be inclusive or exclusive. Provide a `where` argument to filter to specific records only.
 
-In addition to comparisons to a scalar value, you can also compare to another column's values. Any data type that supports the `>` or `<` operators can be compared, so you could also run tests like checking that all order dates are in the past. 
+In addition to comparisons to a scalar value, you can also compare to another column's values. Any data type that supports the `>` or `<` operators can be compared, so you could also run tests like checking that all order dates are in the past.
 
 Usage:
 ```yaml
@@ -449,19 +449,19 @@ models:
           - dbt_utils.accepted_range:
               min_value: 0
               inclusive: false
-      
+
       - name: account_created_at
         tests:
           - dbt_utils.accepted_range:
               max_value: "getdate()"
               #inclusive is true by default
-      
+
       - name: num_returned_orders
         tests:
           - dbt_utils.accepted_range:
               min_value: 0
               max_value: "num_orders"
-      
+
       - name: num_web_sessions
         tests:
           - dbt_utils.accepted_range:
@@ -744,8 +744,8 @@ Usage:
 ```
 
 ---
-### Logger
-#### pretty_time ([source](macros/logger/pretty_time.sql))
+### Jinja Helpers
+#### pretty_time ([source](macros/jinja_helpers/pretty_time.sql))
 This macro returns a string of the current timestamp, optionally taking a datestring format.
 ```sql
 {#- This will return a string like '14:50:34' -#}
@@ -755,7 +755,7 @@ This macro returns a string of the current timestamp, optionally taking a datest
 {{ dbt_utils.pretty_time(format='%Y-%m-%d %H:%M:%S') }}
 ```
 
-#### pretty_log_format ([source](macros/logger/pretty_log_format.sql))
+#### pretty_log_format ([source](macros/jinja_helpers/pretty_log_format.sql))
 This macro formats the input in a way that will print nicely to the command line when you `log` it.
 ```sql
 {#- This will return a string like:
@@ -764,7 +764,7 @@ This macro formats the input in a way that will print nicely to the command line
 
 {{ dbt_utils.pretty_log_format("my pretty message") }}
 ```
-#### log_info ([source](macros/logger/log_info.sql))
+#### log_info ([source](macros/jinja_helpers/log_info.sql))
 This macro logs a formatted message (with a timestamp) to the command line.
 ```sql
 {{ dbt_utils.log_info("my pretty message") }}
@@ -775,6 +775,40 @@ This macro logs a formatted message (with a timestamp) to the command line.
 11:07:31 + my pretty message
 ```
 
+#### slugify ([source](macros/jinja_helpers/slugify.sql))
+This macro is useful for transforming Jinja strings into "slugs", and can be useful when using a Jinja object as a column name, especially when that Jinja object is not hardcoded.
+
+For this example, let's pretend that we have payment methods in our payments table like `['venmo App', 'ca$h-money']`, which we can't use as a column name due to the spaces and special characters. This macro does its best to strip those out in a sensible way: `['venmo_app',
+'cah_money']`.
+
+```sql
+{%- set payment_methods = dbt_utils.get_column_values(
+    table=ref('raw_payments'),
+    column='payment_method'
+) -%}
+
+select
+order_id,
+{%- for payment_method in payment_methods %}
+sum(case when payment_method = '{{ payment_method }}' then amount end)
+  as {{ slugify(payment_method) }}_amount,
+
+{% endfor %}
+...
+```
+
+```sql
+select
+order_id,
+
+sum(case when payment_method = 'Venmo App' then amount end)
+  as venmo_app_amount,
+
+sum(case when payment_method = 'ca$h money' then amount end)
+  as cah_money_amount,
+...
+```
+
 ### Materializations
 #### insert_by_period ([source](macros/materializations/insert_by_period_materialization.sql))
 `insert_by_period` allows dbt to insert records into a table one period (i.e. day, week) at a time.

dbt_project.yml

@@ -1,7 +1,7 @@
 name: 'dbt_utils'
 version: '0.1.0'
 
-require-dbt-version: [">=0.18.0", "<0.19.0"]
+require-dbt-version: [">=0.10.0", "<0.20.0"]
 config-version: 2
 
 target-path: "target"

integration_tests/models/sql/test_star_aliases.sql

@@ -0,0 +1,16 @@
+
+-- TODO : Should the star macro use a case-insensitive comparison for the `except` field on Snowflake?
+
+{% set aliases = {'FIELD_3':'ALIASED'} if target.type == 'snowflake' else {'field_3':'aliased'} %}
+
+
+with data as (
+
+    select
+        {{ dbt_utils.star(from=ref('data_star'), aliases=aliases) }}
+
+    from {{ ref('data_star') }}
+
+)
+
+select * from data

integration_tests/tests/jinja_helpers/test_slugify.sql

@@ -0,0 +1,7 @@
+{% if dbt_utils.slugify('!Hell0 world-hi') == 'hell0_world_hi' %}
+    {# Return 0 rows for the test to pass #}
+    select 1 limit 0
+{% else %}
+    {# Return >0 rows for the test to fail #}
+    select 1
+{% endif %}

macros/jinja_helpers/slugify.sql

@@ -0,0 +1,12 @@
+{% macro slugify(string) %}
+
+{#- Lower case the string -#}
+{% set string = string | lower %}
+{#- Replace spaces and dashes with underscores -#}
+{% set string = modules.re.sub('[ -]+', '_', string) %}
+{#- Only take letters, numbers, and underscores -#}
+{% set string = modules.re.sub('[^a-z0-9_]+', '', string) %}
+
+{{ return(string) }}
+
+{% endmacro %}

macros/sql/pivot.sql

@@ -50,7 +50,8 @@ Arguments:
                then_value=1,
                else_value=0,
                quote_identifiers=True,
-               distinct=False) %}
+               distinct=False
+           ) %}
   {% for v in values %}
     {{ agg }}(
       {% if distinct %} distinct {% endif %}
@@ -64,7 +65,7 @@ Arguments:
       {% if quote_identifiers %}
             as {{ adapter.quote(prefix ~ v ~ suffix) }}
       {% else %}
-        as {{prefix ~ v ~ suffix }}
+        as ({{ dbt_utils.slugify(prefix ~ v ~ suffix) }})
       {% endif %}
     {% endif %}
     {% if not loop.last %},{% endif %}