Skip to content

Commit b748e5f

Browse files
authored
Remove dependency on dbt-utils (#91)
* Removes dependency on dbt-utils * Update README
1 parent f131d71 commit b748e5f

File tree

8 files changed

+231
-80
lines changed

8 files changed

+231
-80
lines changed

README.md

+54-52
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,6 @@
44

55
Extension package for [**dbt**](https://github.com/dbt-labs/dbt) to handle date logic and calendar functionality.
66

7-
FYI: this package includes [**dbt-utils**](https://github.com/dbt-labs/dbt-utils) so there"s no need to also import dbt-utils in your local project. (In fact, you may get an error if you do.)
8-
97
Include in `packages.yml`
108

119
```yaml
@@ -15,7 +13,13 @@ packages:
1513
# <see https://github.com/calogica/dbt-date/releases/latest> for the latest version tag
1614
```
1715

18-
Note: we no longer include `spark_utils` in this package to avoid versioning conflicts. If you are running this package on non-core (Snowflake, BigQuery, Redshift, Postgres) platforms, you will need to use a package like `spark_utils` to shim macros.
16+
This package supports:
17+
18+
* Postgres
19+
* Snowflake
20+
* BigQuery
21+
22+
For other platforms, you will have to include a shim package for the platform, such as `spark-utils`, or `tsql-utils`.
1923

2024
For example, in `packages.yml`, you will need to include the relevant package:
2125

@@ -43,65 +47,54 @@ vars:
4347
You may specify [any valid timezone string](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) in place of `America/Los_Angeles`.
4448
For example, use `America/New_York` for East Coast Time.
4549

46-
## Integration Tests (Developers Only)
47-
48-
This project contains integration tests for all test macros in a separate `integration_tests` dbt project contained in this repo.
49-
50-
To run the tests:
51-
52-
1. You will need a profile called `integration_tests` in `~/.dbt/profiles.yml` pointing to a writable database. We only support postgres, BigQuery and Snowflake.
53-
2. Then, from within the `integration_tests` folder, run `dbt build` to run the test models in `integration_tests/models/schema_tests/` and run the tests specified in `integration_tests/models/schema_tests/schema.yml`
54-
55-
## Available Tests
56-
5750
## Available Macros
5851

5952
### Date Dimension
6053

61-
- [get_base_dates](#get_base_datesstart_datenone-end_datenone-n_datepartsnone-datepartday)
62-
- [get_date_dimension](#get_date_dimensionstart_date-end_date)
54+
* [get_base_dates](#get_base_datesstart_datenone-end_datenone-n_datepartsnone-datepartday)
55+
* [get_date_dimension](#get_date_dimensionstart_date-end_date)
6356

6457
### Calendar Date
6558

66-
- [convert_timezone](#convert_timezone-column-target_tznone-source_tznone)
67-
- [date_part](#date_partdatepart-date)
68-
- [day_name](#day_namedate-shorttrue)
69-
- [day_of_month](#day_of_monthdate)
70-
- [day_of_week](#day_of_weekdate-isoweektrue)
71-
- [day_of_year](#day_of_yeardate)
72-
- [from_unixtimestamp](#from_unixtimestampepochs-formatseconds)
73-
- [iso_week_end](#iso_week_enddatenone-tznone)
74-
- [iso_week_of_year](#iso_week_of_yeardatenone-tznone)
75-
- [iso_week_start](#iso_week_startdatenone-tznone)
76-
- [last_month_name](#last_month_nameshorttrue-tznone)
77-
- [last_month_number](#last_month_numbertznone)
78-
- [last_month](#last_monthtznone)
79-
- [last_week](#last_weektznone)
80-
- [month_name](#month_namedate-shorttrue-tznone)
81-
- [n_days_ago](#n_days_agon-datenone-tznone)
82-
- [n_days_away](#n_days_awayn-datenone-tznone)
83-
- [n_months_ago](#n_months_agon-tznone)
84-
- [n_months_away](#n_months_awayn-tznone)
85-
- [n_weeks_ago](#n_weeks_agon-tznone)
86-
- [n_weeks_away](#n_weeks_awayn-tznone)
87-
- [next_month_name](#next_month_nameshorttrue-tznone)
88-
- [next_month_number](#next_month_numbertznone)
89-
- [next_month](#next_monthtznone)
90-
- [next_week](#next_weektznone)
91-
- [now](#nowtznone)
92-
- [periods_since](#periods_sincedate_col-period_nameday-tznone)
93-
- [round_timestamp](#round_timestamptimestamp)
94-
- [to_unixtimestamp](#to_unixtimestamptimestamp)
95-
- [today](#todaytznone)
96-
- [tomorrow](#tomorrowdatenone-tznone)
97-
- [week_end](#week_enddatenone-tznone)
98-
- [week_of_year](#week_of_yeardatenone-tznone)
99-
- [week_start](#week_startdatenone-tznone)
100-
- [yesterday](#yesterdaydatenone-tznone)
59+
* [convert_timezone](#convert_timezone-column-target_tznone-source_tznone)
60+
* [date_part](#date_partdatepart-date)
61+
* [day_name](#day_namedate-shorttrue)
62+
* [day_of_month](#day_of_monthdate)
63+
* [day_of_week](#day_of_weekdate-isoweektrue)
64+
* [day_of_year](#day_of_yeardate)
65+
* [from_unixtimestamp](#from_unixtimestampepochs-formatseconds)
66+
* [iso_week_end](#iso_week_enddatenone-tznone)
67+
* [iso_week_of_year](#iso_week_of_yeardatenone-tznone)
68+
* [iso_week_start](#iso_week_startdatenone-tznone)
69+
* [last_month_name](#last_month_nameshorttrue-tznone)
70+
* [last_month_number](#last_month_numbertznone)
71+
* [last_month](#last_monthtznone)
72+
* [last_week](#last_weektznone)
73+
* [month_name](#month_namedate-shorttrue-tznone)
74+
* [n_days_ago](#n_days_agon-datenone-tznone)
75+
* [n_days_away](#n_days_awayn-datenone-tznone)
76+
* [n_months_ago](#n_months_agon-tznone)
77+
* [n_months_away](#n_months_awayn-tznone)
78+
* [n_weeks_ago](#n_weeks_agon-tznone)
79+
* [n_weeks_away](#n_weeks_awayn-tznone)
80+
* [next_month_name](#next_month_nameshorttrue-tznone)
81+
* [next_month_number](#next_month_numbertznone)
82+
* [next_month](#next_monthtznone)
83+
* [next_week](#next_weektznone)
84+
* [now](#nowtznone)
85+
* [periods_since](#periods_sincedate_col-period_nameday-tznone)
86+
* [round_timestamp](#round_timestamptimestamp)
87+
* [to_unixtimestamp](#to_unixtimestamptimestamp)
88+
* [today](#todaytznone)
89+
* [tomorrow](#tomorrowdatenone-tznone)
90+
* [week_end](#week_enddatenone-tznone)
91+
* [week_of_year](#week_of_yeardatenone-tznone)
92+
* [week_start](#week_startdatenone-tznone)
93+
* [yesterday](#yesterdaydatenone-tznone)
10194

10295
## Fiscal Date
10396

104-
- [get_fiscal_periods](#get_fiscal_periodsdates-year_end_month-week_start_day-shift_year1)
97+
* [get_fiscal_periods](#get_fiscal_periodsdates-year_end_month-week_start_day-shift_year1)
10598

10699
## Documentation
107100

@@ -801,3 +794,12 @@ or, optionally, you can override the default timezone:
801794
```sql
802795
{{ dbt_date.yesterday(tz="America/New_York") }} as date_yesterday
803796
```
797+
798+
## Integration Tests (Developers Only)
799+
800+
This project contains integration tests for all test macros in a separate `integration_tests` dbt project contained in this repo.
801+
802+
To run the tests:
803+
804+
1. You will need a profile called `integration_tests` in `~/.dbt/profiles.yml` pointing to a writable database. We only support postgres, BigQuery and Snowflake.
805+
2. Then, from within the `integration_tests` folder, run `dbt build` to run the test models in `integration_tests/models/schema_tests/` and run the tests specified in `integration_tests/models/schema_tests/schema.yml`
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
{% test expression_is_true(model, expression, column_name=None, condition='1=1') %}
2+
{# T-SQL has no boolean data type so we use 1=1 which returns TRUE #}
3+
{# ref https://stackoverflow.com/a/7170753/3842610 #}
4+
{{ return(adapter.dispatch('test_expression_is_true', 'dbt_date_integration_tests')(model, expression, column_name, condition)) }}
5+
{% endtest %}
6+
7+
{% macro default__test_expression_is_true(model, expression, column_name, condition) %}
8+
9+
with meet_condition as (
10+
select * from {{ model }} where {{ condition }}
11+
)
12+
13+
select
14+
*
15+
from meet_condition
16+
{% if column_name is none %}
17+
where not({{ expression }})
18+
{%- else %}
19+
where not({{ column_name }} {{ expression }})
20+
{%- endif %}
21+
22+
{% endmacro %}

integration_tests/models/test_dates.yml

+22-22
Original file line numberDiff line numberDiff line change
@@ -2,50 +2,50 @@ version: 2
22
models:
33
- name: test_dates
44
tests:
5-
- dbt_utils.expression_is_true:
5+
- expression_is_true:
66
expression: "prior_date_day = {{ dbt_date.yesterday('date_day') }}"
7-
- dbt_utils.expression_is_true:
7+
- expression_is_true:
88
expression: "next_date_day = {{ dbt_date.tomorrow('date_day') }}"
9-
- dbt_utils.expression_is_true:
9+
- expression_is_true:
1010
expression: "day_name = {{ dbt_date.day_name('date_day', short=False) }}"
11-
- dbt_utils.expression_is_true:
11+
- expression_is_true:
1212
expression: "day_name_short = {{ dbt_date.day_name('date_day', short=True) }}"
13-
- dbt_utils.expression_is_true:
13+
- expression_is_true:
1414
expression: "day_of_month = {{ dbt_date.day_of_month('date_day') }}"
15-
- dbt_utils.expression_is_true:
15+
- expression_is_true:
1616
expression: "day_of_week = {{ dbt_date.day_of_week('date_day', isoweek=False) }}"
17-
- dbt_utils.expression_is_true:
17+
- expression_is_true:
1818
expression: "iso_day_of_week = {{ dbt_date.day_of_week('date_day', isoweek=True) }}"
19-
- dbt_utils.expression_is_true:
19+
- expression_is_true:
2020
expression: "day_of_year = {{ dbt_date.day_of_year('date_day') }}"
2121

22-
- dbt_utils.expression_is_true:
22+
- expression_is_true:
2323
expression: "week_start_date = {{ dbt_date.week_start('date_day') }}"
24-
- dbt_utils.expression_is_true:
24+
- expression_is_true:
2525
expression: "week_end_date = {{ dbt_date.week_end('date_day') }}"
26-
- dbt_utils.expression_is_true:
26+
- expression_is_true:
2727
expression: "week_of_year = {{ dbt_date.week_of_year('date_day') }}"
28-
- dbt_utils.expression_is_true:
28+
- expression_is_true:
2929
expression: "iso_week_start_date = {{ dbt_date.iso_week_start('date_day') }}"
30-
- dbt_utils.expression_is_true:
30+
- expression_is_true:
3131
expression: "iso_week_end_date = {{ dbt_date.iso_week_end('date_day') }}"
32-
- dbt_utils.expression_is_true:
32+
- expression_is_true:
3333
expression: "iso_week_of_year = {{ dbt_date.iso_week_of_year('date_day') }}"
34-
- dbt_utils.expression_is_true:
34+
- expression_is_true:
3535
expression: "time_stamp_utc = {{ dbt_date.from_unixtimestamp('unix_epoch') }}"
36-
- dbt_utils.expression_is_true:
36+
- expression_is_true:
3737
expression: "unix_epoch = {{ dbt_date.to_unixtimestamp('time_stamp_utc') }}"
38-
- dbt_utils.expression_is_true:
38+
- expression_is_true:
3939
expression: "time_stamp = {{ dbt_date.convert_timezone('time_stamp_utc') }}"
40-
- dbt_utils.expression_is_true:
40+
- expression_is_true:
4141
expression: "time_stamp = {{ dbt_date.convert_timezone('time_stamp_utc', source_tz='UTC') }}"
42-
# - dbt_utils.expression_is_true:
42+
# - expression_is_true:
4343
# expression: "time_stamp_utc = {{ dbt_date.convert_timezone('time_stamp', source_tz='America/Los_Angeles', target_tz='UTC') }}"
44-
# - dbt_utils.expression_is_true:
44+
# - expression_is_true:
4545
# expression: "time_stamp = {{ dbt_date.convert_timezone('time_stamp', source_tz='America/Los_Angeles', target_tz='America/Los_Angeles') }}"
46-
- dbt_utils.expression_is_true:
46+
- expression_is_true:
4747
expression: "rounded_timestamp = {{ dbt_date.round_timestamp('time_stamp') }}"
48-
- dbt_utils.expression_is_true:
48+
- expression_is_true:
4949
expression: "rounded_timestamp_utc = {{ dbt_date.round_timestamp('time_stamp_utc') }}"
5050

5151
columns:

integration_tests/packages.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
packages:
2-
- local: ../
2+
- local: ../

macros/_utils/date_spine.sql

+77
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
{% macro get_intervals_between(start_date, end_date, datepart) -%}
2+
{{ return(adapter.dispatch('get_intervals_between', 'dbt_date')(start_date, end_date, datepart)) }}
3+
{%- endmacro %}
4+
5+
{% macro default__get_intervals_between(start_date, end_date, datepart) -%}
6+
{%- call statement('get_intervals_between', fetch_result=True) %}
7+
8+
select {{ datediff(start_date, end_date, datepart) }}
9+
10+
{%- endcall -%}
11+
12+
{%- set value_list = load_result('get_intervals_between') -%}
13+
14+
{%- if value_list and value_list['data'] -%}
15+
{%- set values = value_list['data'] | map(attribute=0) | list %}
16+
{{ return(values[0]) }}
17+
{%- else -%}
18+
{{ return(1) }}
19+
{%- endif -%}
20+
21+
{%- endmacro %}
22+
23+
24+
25+
26+
{% macro date_spine(datepart, start_date, end_date) %}
27+
{{ return(adapter.dispatch('date_spine', 'dbt_date')(datepart, start_date, end_date)) }}
28+
{%- endmacro %}
29+
30+
{% macro default__date_spine(datepart, start_date, end_date) %}
31+
32+
33+
{# call as follows:
34+
35+
date_spine(
36+
"day",
37+
"to_date('01/01/2016', 'mm/dd/yyyy')",
38+
"dateadd(week, 1, current_date)"
39+
) #}
40+
41+
42+
with rawdata as (
43+
44+
{{
45+
dbt_date.generate_series(
46+
dbt_date.get_intervals_between(start_date, end_date, datepart)
47+
)
48+
}}
49+
50+
),
51+
52+
all_periods as (
53+
54+
select (
55+
{{
56+
dateadd(
57+
datepart,
58+
"row_number() over (order by 1) - 1",
59+
start_date
60+
)
61+
}}
62+
) as date_{{datepart}}
63+
from rawdata
64+
65+
),
66+
67+
filtered as (
68+
69+
select *
70+
from all_periods
71+
where date_{{datepart}} <= {{ end_date }}
72+
73+
)
74+
75+
select * from filtered
76+
77+
{% endmacro %}

macros/_utils/generate_series.sql

+53
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
{% macro get_powers_of_two(upper_bound) %}
2+
{{ return(adapter.dispatch('get_powers_of_two', 'dbt_date')(upper_bound)) }}
3+
{% endmacro %}
4+
5+
{% macro default__get_powers_of_two(upper_bound) %}
6+
7+
{% if upper_bound <= 0 %}
8+
{{ exceptions.raise_compiler_error("upper bound must be positive") }}
9+
{% endif %}
10+
11+
{% for _ in range(1, 100) %}
12+
{% if upper_bound <= 2 ** loop.index %}{{ return(loop.index) }}{% endif %}
13+
{% endfor %}
14+
15+
{% endmacro %}
16+
17+
18+
{% macro generate_series(upper_bound) %}
19+
{{ return(adapter.dispatch('generate_series', 'dbt_date')(upper_bound)) }}
20+
{% endmacro %}
21+
22+
{% macro default__generate_series(upper_bound) %}
23+
24+
{% set n = dbt_date.get_powers_of_two(upper_bound) %}
25+
26+
with p as (
27+
select 0 as generated_number union all select 1
28+
), unioned as (
29+
30+
select
31+
32+
{% for i in range(n) %}
33+
p{{i}}.generated_number * power(2, {{i}})
34+
{% if not loop.last %} + {% endif %}
35+
{% endfor %}
36+
+ 1
37+
as generated_number
38+
39+
from
40+
41+
{% for i in range(n) %}
42+
p as p{{i}}
43+
{% if not loop.last %} cross join {% endif %}
44+
{% endfor %}
45+
46+
)
47+
48+
select *
49+
from unioned
50+
where generated_number <= {{upper_bound}}
51+
order by generated_number
52+
53+
{% endmacro %}

macros/get_base_dates.sql

+2-2
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
with date_spine as
1818
(
1919

20-
{{ dbt_utils.date_spine(
20+
{{ dbt_date.date_spine(
2121
datepart=datepart,
2222
start_date=start_date,
2323
end_date=end_date,
@@ -46,7 +46,7 @@ from
4646
with date_spine as
4747
(
4848

49-
{{ dbt_utils.date_spine(
49+
{{ dbt_date.date_spine(
5050
datepart=datepart,
5151
start_date=start_date,
5252
end_date=end_date,

packages.yml

-3
Original file line numberDiff line numberDiff line change
@@ -1,3 +0,0 @@
1-
packages:
2-
- package: dbt-labs/dbt_utils
3-
version: [">=0.9.0", "<1.0.0"]

0 commit comments

Comments
 (0)