Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bool unpivot #305

Merged
merged 4 commits into from
Dec 23, 2020
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Adding some logic to handle unpivoting boolean columns
avishalom authored and clrcrl committed Dec 14, 2020
commit bc80c6585c63aacec7a6d6247a87a93fe0beea29
4 changes: 4 additions & 0 deletions integration_tests/data/sql/data_unpivot_bool.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
customer_id,created_at,status,segment,is_updated
123,2017-01-01,active,tier 1,TRUE
234,2017-02-01,active,tier 3,FALSE
567,2017-03-01,churned,tier 2,TRUE
10 changes: 10 additions & 0 deletions integration_tests/data/sql/data_unpivot_bool_expected.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
customer_id,created_at,prop,val
123,"2017-01-01","segment","tier 1"
123,"2017-01-01","status","active"
123,"2017-01-01","is_updated","true"
234,"2017-02-01","segment","tier 3"
234,"2017-02-01","status","active"
234,"2017-02-01","is_updated","false"
567,"2017-03-01","status","churned"
567,"2017-03-01","is_updated","true"
567,"2017-03-01","segment","tier 2"
5 changes: 5 additions & 0 deletions integration_tests/models/sql/schema.yml
Original file line number Diff line number Diff line change
@@ -96,6 +96,11 @@ models:
- dbt_utils.equality:
compare_model: ref('data_unpivot_expected')

- name: test_unpivot_bool
tests:
- dbt_utils.equality:
compare_model: ref('data_unpivot_bool_expected')

- name: test_star
tests:
- dbt_utils.equality:
33 changes: 33 additions & 0 deletions integration_tests/models/sql/test_unpivot_bool.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@

-- snowflake messes with these tests pretty badly since the
-- output of the macro considers the casing of the source
-- table columns. Using some hacks here to get this to work,
-- but we should consider lowercasing the unpivot macro output
-- at some point in the future for consistency

{% if target.name == 'snowflake' %}
{% set exclude = ['CUSTOMER_ID', 'CREATED_AT'] %}
{% else %}
{% set exclude = ['customer_id', 'created_at'] %}
{% endif %}


select
customer_id,
created_at,
case
when '{{ target.name }}' = 'snowflake' then lower(prop)
else prop
end as prop,
val

from (
{{ dbt_utils.unpivot(
relation=ref('data_unpivot_bool'),
cast_to=dbt_utils.type_string(),
exclude=exclude,
remove='is_updated',
field_name='prop',
value_name='val'
) }}
) as sbq
7 changes: 6 additions & 1 deletion macros/sql/unpivot.sql
Original file line number Diff line number Diff line change
@@ -58,7 +58,12 @@ Arguments:
{%- endfor %}

cast('{{ col.column }}' as {{ dbt_utils.type_string() }}) as {{ field_name }},
cast({{ col.column }} as {{ cast_to }}) as {{ value_name }}
cast( {% if col.data_type == 'boolean' %}
case when {{ col.column }} then 'true' else 'false' end
{% else %}
{{ col.column }}
{% endif %}
as {{ cast_to }}) as {{ value_name }}

from {{ relation }}