Open
Description
Is this a new bug in dbt-core?
- I believe this is a new bug in dbt-core
- I have searched the existing issues, and I could not find an existing issue for this bug
Current Behavior
I wrote a generic test like under tests/generic/column_distinct_count_greather_than_threshold
{% test column_distinct_count_greater_than_threshold(model, column_name, thresholds_table, thresholds_column) %}
WITH t AS (
SELECT *
FROM {{ thresholds_table }}
WHERE prov_code = {{ var('prov_code') }}
LIMIT 1
),
d AS (
SELECT
{{ var('prov_code')}}::INTEGER AS prov_code,
COUNT(DISTINCT {{ column_name }}) AS distinct_count
FROM {{ model }}
)
SELECT * from t
RIGHT JOIN d USING (prov_code)
WHERE coalesce(t.min_threshold,99999999999) > d.distinct_count
{% endtest %}
its then referenced in a source column like:
- name: the_column_name
description: ""
type: string
data_tests:
- column_distinct_count_greater_than_threshold:
thresholds_table: source('raw.devo_by_prov', 'thresholds')
thresholds_column: "min_threshold"
the model is specifying the correct source like:
select * FROM {{ source('raw.devo_by_prov', 'xxxx' ) }}
When I run:
dbt build --select @namespace.model
The test is running, failing or passing accordingly, but the model is always executed, so rows are inserted event when failing.
Expected Behavior
source test should run first in order to skip model generation.
Steps To Reproduce
see above.
Relevant log output
08:09:28 1 of 3 START sql table model bdia_devo.stg_devo_prov__nokia_4g_unpivot ......... [RUN]
08:09:29 1 of 3 OK created sql table model bdia_devo.stg_devo_prov__nokia_4g_unpivot .... [OK in 0.99s]
08:09:29 2 of 3 START test source_column_distinct_count_greater_than_threshold_raw.devo_by_prov_nokia_4g_ECGI__min_threshold__source_raw_devo_by_prov_thresholds_ [RUN]
08:09:30 2 of 3 FAIL 1 source_column_distinct_count_greater_than_threshold_raw.devo_by_prov_nokia_4g_ECGI__min_threshold__source_raw_devo_by_prov_thresholds_ [FAIL 1 in 0.99s]
08:09:30 3 of 3 START sql external model bdia.devo_events_prov_nokia_4g ................. [RUN]
Environment
- OS: python:3.12-slim
- Python:3.12.7
- dbt:
Core:
- installed: 1.9.2
- latest: 1.9.2 - Up to date!
Plugins:
- duckdb: 1.9.2 - Up to date!
Which database adapter are you using with dbt?
other (mention it in "Additional Context")
Additional Context
duckdb adapter
Activity