Skip to content

[Bug] generic data test on source is running after model #11309

Open
@dchimeno

Description

@dchimeno

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

I wrote a generic test like under tests/generic/column_distinct_count_greather_than_threshold

{% test column_distinct_count_greater_than_threshold(model, column_name, thresholds_table, thresholds_column) %}

WITH t AS (
    SELECT *
    FROM {{ thresholds_table }}
    WHERE prov_code = {{ var('prov_code') }}
    LIMIT 1
),
d AS (
    SELECT 
        {{ var('prov_code')}}::INTEGER AS prov_code,
        COUNT(DISTINCT {{ column_name }}) AS distinct_count
    FROM {{ model }}
)
SELECT * from t
RIGHT JOIN d USING (prov_code)
WHERE coalesce(t.min_threshold,99999999999) > d.distinct_count

{% endtest %}

its then referenced in a source column like:

          - name: the_column_name
            description: ""
            type: string
            data_tests:
              - column_distinct_count_greater_than_threshold:
                  thresholds_table: source('raw.devo_by_prov', 'thresholds')
                  thresholds_column: "min_threshold"

the model is specifying the correct source like:

select *  FROM  {{ source('raw.devo_by_prov', 'xxxx' ) }}

When I run:

dbt build --select @namespace.model

The test is running, failing or passing accordingly, but the model is always executed, so rows are inserted event when failing.

Expected Behavior

source test should run first in order to skip model generation.

Steps To Reproduce

see above.

Relevant log output

08:09:28  1 of 3 START sql table model bdia_devo.stg_devo_prov__nokia_4g_unpivot ......... [RUN]
08:09:29  1 of 3 OK created sql table model bdia_devo.stg_devo_prov__nokia_4g_unpivot .... [OK in 0.99s]
08:09:29  2 of 3 START test source_column_distinct_count_greater_than_threshold_raw.devo_by_prov_nokia_4g_ECGI__min_threshold__source_raw_devo_by_prov_thresholds_  [RUN]
08:09:30  2 of 3 FAIL 1 source_column_distinct_count_greater_than_threshold_raw.devo_by_prov_nokia_4g_ECGI__min_threshold__source_raw_devo_by_prov_thresholds_  [FAIL 1 in 0.99s]
08:09:30  3 of 3 START sql external model bdia.devo_events_prov_nokia_4g ................. [RUN]

Environment

- OS: python:3.12-slim
- Python:3.12.7
- dbt:

Core:
  - installed: 1.9.2
  - latest:    1.9.2 - Up to date!

Plugins:
  - duckdb: 1.9.2 - Up to date!

Which database adapter are you using with dbt?

other (mention it in "Additional Context")

Additional Context

duckdb adapter

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdbt testsIssues related to built-in dbt testing functionalitytriage

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions