Skip to content

Commit 4554198

Browse files
Claire Carrolldmarts
Claire Carroll
andauthored
Add fewer_rows_then schema test
* Macro and test case * Update fewer_rows_than.sql * Update for dev/0.7.0 changes * Update changelog (skip ci) Co-authored-by: Dan M <47741043+dmarts@users.noreply.github.com>
1 parent d5cdb8e commit 4554198

File tree

6 files changed

+87
-11
lines changed

6 files changed

+87
-11
lines changed

CHANGELOG.md

+1
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
* Make `expression_is_true` work as a column test (code originally in [#226](https://github.com/fishtown-analytics/dbt-utils/pull/226/) from [@elliottohara](https://github.com/elliottohara), merged via [#313])
77
* Allow individual columns in star macro to be aliased (code originally in [#230](https://github.com/fishtown-analytics/dbt-utils/pull/230/) from [@elliottohara](https://github.com/elliottohara), merged via [#245])
88
* Add new schema test, `not_accepted_values` ([#284](https://github.com/fishtown-analytics/dbt-utils/pull/284) [@JavierMonton](https://github.com/JavierMonton))
9+
* Add new schema test, `fewer_rows_than` (code originally in [#221](https://github.com/fishtown-analytics/dbt-utils/pull/230/) from [@dmarts](https://github.com/dmarts), merged via [#343])
910

1011

1112
## Fixes

README.md

+25-11
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,7 @@ Usage:
9595
---
9696
### Date/Time
9797
#### date_spine ([source](macros/datetime/date_spine.sql))
98-
This macro returns the sql required to build a date spine. The spine will include the `start_date` (if it is aligned to the `datepart`), but it will not include the `end_date`.
98+
This macro returns the sql required to build a date spine. The spine will include the `start_date` (if it is aligned to the `datepart`), but it will not include the `end_date`.
9999

100100
Usage:
101101
```
@@ -132,6 +132,20 @@ models:
132132

133133
```
134134

135+
#### fewer_rows_than ([source](macros/schema_tests/fewer_rows_than.sql))
136+
This schema test asserts that this model has fewer rows than the referenced model.
137+
138+
Usage:
139+
```yaml
140+
version: 2
141+
142+
models:
143+
- name: model_name
144+
tests:
145+
- dbt_utils.fewer_rows_than:
146+
compare_model: ref('other_table_name')
147+
```
148+
135149
#### equality ([source](macros/schema_tests/equality.sql))
136150
This schema test asserts the equality of two relations. Optionally specify a subset of columns to compare.
137151
@@ -181,13 +195,13 @@ models:
181195

182196
```
183197

184-
This macro can also be used at the column level. When this is done, the `expression` is evaluated against the column.
198+
This macro can also be used at the column level. When this is done, the `expression` is evaluated against the column.
185199

186200
```yaml
187201
version: 2
188-
models:
202+
models:
189203
- name: model_name
190-
columns:
204+
columns:
191205
- name: col_a
192206
tests:
193207
- dbt_utils.expression_is_true:
@@ -197,7 +211,7 @@ models:
197211
- dbt_utils.expression_is_true:
198212
expression: '= 1'
199213
condition: col_a = 1
200-
214+
201215
```
202216

203217

@@ -361,7 +375,7 @@ models:
361375
upper_bound_column: ended_at
362376
partition_by: customer_id
363377
gaps: required
364-
378+
365379
# test that each customer can have subscriptions that start and end on the same date
366380
- name: subscriptions
367381
tests:
@@ -497,9 +511,9 @@ constraint needs to be verified.
497511

498512

499513
#### accepted_range ([source](macros/schema_tests/accepted_range.sql))
500-
This test checks that a column's values fall inside an expected range. Any combination of `min_value` and `max_value` is allowed, and the range can be inclusive or exclusive. Provide a `where` argument to filter to specific records only.
514+
This test checks that a column's values fall inside an expected range. Any combination of `min_value` and `max_value` is allowed, and the range can be inclusive or exclusive. Provide a `where` argument to filter to specific records only.
501515

502-
In addition to comparisons to a scalar value, you can also compare to another column's values. Any data type that supports the `>` or `<` operators can be compared, so you could also run tests like checking that all order dates are in the past.
516+
In addition to comparisons to a scalar value, you can also compare to another column's values. Any data type that supports the `>` or `<` operators can be compared, so you could also run tests like checking that all order dates are in the past.
503517

504518
Usage:
505519
```yaml
@@ -513,19 +527,19 @@ models:
513527
- dbt_utils.accepted_range:
514528
min_value: 0
515529
inclusive: false
516-
530+
517531
- name: account_created_at
518532
tests:
519533
- dbt_utils.accepted_range:
520534
max_value: "getdate()"
521535
#inclusive is true by default
522-
536+
523537
- name: num_returned_orders
524538
tests:
525539
- dbt_utils.accepted_range:
526540
min_value: 0
527541
max_value: "num_orders"
528-
542+
529543
- name: num_web_sessions
530544
tests:
531545
- dbt_utils.accepted_range:
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
field
2+
1
3+
2
4+
3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
field
2+
1
3+
2
4+
3
5+
4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
with data as (
2+
3+
select * from {{ ref('data_test_fewer_rows_than_table_1') }}
4+
5+
)
6+
7+
select
8+
field
9+
from data
+43
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
{% macro test_fewer_rows_than(model) %}
2+
{{ return(adapter.dispatch('test_fewer_rows_than', packages = dbt_utils._get_utils_namespaces())(model, combination_of_columns, quote_columns, where)) }}
3+
{% endmacro %}
4+
5+
{% macro default__test_fewer_rows_than(model) %}
6+
7+
{% set compare_model = kwargs.get('compare_model', kwargs.get('arg')) %}
8+
9+
with a as (
10+
11+
select count(*) as count_ourmodel from {{ model }}
12+
13+
),
14+
b as (
15+
16+
select count(*) as count_comparisonmodel from {{ compare_model }}
17+
18+
),
19+
counts as (
20+
21+
select
22+
(select count_ourmodel from a) as count_model_with_fewer_rows,
23+
(select count_comparisonmodel from b) as count_model_with_more_rows
24+
25+
),
26+
final as (
27+
28+
select
29+
case
30+
-- fail the test if we have more rows than the reference model and return the row count delta
31+
when count_model_with_fewer_rows > count_model_with_more_rows then (count_model_with_fewer_rows - count_model_with_more_rows)
32+
-- fail the test if they are the same number
33+
when count_model = count_comparison then 1
34+
-- pass the test if the delta is positive (i.e. return the number 0)
35+
else 0
36+
end as row_count_delta
37+
from counts
38+
39+
)
40+
41+
select row_count_delta from final
42+
43+
{% endmacro %}

0 commit comments

Comments
 (0)