Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/dbt_ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,4 +91,4 @@ jobs:
run: uv run dbt run --select state:modified,config.materialized:incremental --state ./state

- name: Test modified incremental models (after incremental run)
run: uv run dbt test --select state:modified,config.materialized:incremental --state ./state
run: uv run dbt test --select state:modified,config.materialized:incremental --state ./state
2 changes: 1 addition & 1 deletion .github/workflows/dbt_prod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,4 +41,4 @@ jobs:
run: uv run dbt run --exclude config.materialized:view # views should only run when modified

- name: Test models
run: uv run dbt test --exclude config.materialized:view # views should only be tested when modified
run: uv run dbt test --exclude config.materialized:view # views should only be tested when modified
2 changes: 1 addition & 1 deletion dbt_project.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,4 +37,4 @@ models:
- sql: "{{ optimize_table(this, model.config.materialized) }}"
transaction: true
- sql: "{{ vacuum_table(this, model.config.materialized) }}"
transaction: true
transaction: true
3 changes: 1 addition & 2 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ If this repo was created from the dune-dbt-template, you can pull in updates:

**Set up upstream (one-time):**
```bash
git remote add upstream https://github.com/YOUR_ORG/dune-dbt-template.git
git remote add upstream https://github.com/duneanalytics/dune-dbt-template.git
git fetch upstream
```

Expand All @@ -128,4 +128,3 @@ git push origin main
- Read [Development Workflow](development-workflow.md) to learn the recommended process
- Review [dbt Best Practices](dbt-best-practices.md) for repo-specific patterns
- Check [SQL Style Guide](sql-style-guide.md) for formatting standards

9 changes: 4 additions & 5 deletions docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,16 +121,16 @@ columns:

```bash
# Test all models
dbt test
uv run dbt test

# Test specific model
dbt test --select my_model
uv run dbt test --select my_model

# Test specific model and downstream
dbt test --select my_model+
uv run dbt test --select my_model+

# Run and test in sequence
dbt run --select my_model && dbt test --select my_model
uv run dbt run --select my_model && uv run dbt test --select my_model
```

## Why These Tests?
Expand All @@ -155,4 +155,3 @@ To fix test failures:

- [dbt Best Practices](dbt-best-practices.md) - NULL handling in unique keys
- [CI/CD](cicd.md) - How tests run in GitHub Actions

5 changes: 2 additions & 3 deletions docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,10 +160,10 @@ You must **manually drop the table** before running the full refresh.
**Option 1: Use the provided Python script**
```bash
# Drop a single table
uv run python scripts/drop_table.py --schema your_schema_name --table your_table_name
uv run python scripts/drop_tables.py --schema your_schema_name --table your_table_name

# Drop with target specification
uv run python scripts/drop_table.py --schema your_schema_name --table your_table_name --target dev
uv run python scripts/drop_tables.py --schema your_schema_name --table your_table_name --target dev
```

**Option 2: Use any Trino client**
Expand Down Expand Up @@ -293,4 +293,3 @@ git push
- [Getting Started](getting-started.md) - Initial setup
- [Development Workflow](development-workflow.md) - Development process
- [Testing](testing.md) - Test requirements

2 changes: 1 addition & 1 deletion macros/dune_dbt_overrides/get_custom_schema.sql
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@
{{ target.schema }}__tmp_
{%- endif -%}

{%- endmacro %}
{%- endmacro %}
2 changes: 1 addition & 1 deletion macros/dune_dbt_overrides/optimize_table.sql
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@
{%- if target.name == 'prod' and materialization in ('table', 'incremental') -%}
alter table {{this}} execute optimize
{%- endif -%}
{%- endmacro -%}
{%- endmacro -%}
7 changes: 1 addition & 6 deletions macros/dune_dbt_overrides/source.sql
Original file line number Diff line number Diff line change
@@ -1,10 +1,5 @@
{#
once we change the connection to dune connectivity / api, QES should auto-resolve delta_prod database for sources?
goal: remove this macro override, have the backend handle applying delta_prod database for sources
#}

{% macro source(source_name, table_name, database="delta_prod") %}
{% set rel = builtins.source(source_name, table_name) %}
{% set newrel = rel.replace_path(database=database) %}
{% do return(newrel) %}
{% endmacro %}
{% endmacro %}
2 changes: 1 addition & 1 deletion macros/dune_dbt_overrides/vacuum_table.sql
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@
{%- if target.name == 'prod' and materialization in ('table', 'incremental') -%}
call dune.system.vacuum('{{ this.schema }}', '{{ this.name }}', '7d')
{%- endif -%}
{%- endmacro -%}
{%- endmacro -%}
2 changes: 1 addition & 1 deletion models/templates/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,4 +63,4 @@ models:
- name: block_date
description: "The date of the block"
- name: total_tx_per_block
description: "The total number of transactions per block"
description: "The total number of transactions per block"
2 changes: 1 addition & 1 deletion models/templates/_sources.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ version: 2
sources:
- name: ethereum
tables:
- name: transactions
- name: transactions
2 changes: 1 addition & 1 deletion models/templates/dbt_template_append_incremental_model.sql
Original file line number Diff line number Diff line change
Expand Up @@ -47,4 +47,4 @@ left join {{ this }} as t
and t.block_date >= now() - interval '1' day -- filter target to same time window as source
where
t.block_number is null -- only insert rows that don't already exist
{%- endif %}
{%- endif %}
Original file line number Diff line number Diff line change
@@ -1,11 +1,3 @@
{#
key notes on delete+insert model:
- file_format defaults to delta (TODO: confirm this is dune hive metastore setting)
- when providing file_format config to model, dbt fails on unable to support 'format' property
- incremental_predicates filter the DELETE operation on the target table for better performance
- this limits the rows scanned during the delete phase to match the source filter window
#}

{{ config(
alias = 'dbt_template_delete_insert_incremental_model'
, materialized = 'incremental'
Expand Down Expand Up @@ -33,4 +25,4 @@ where
{%- endif %}
group by
block_number
, block_date
, block_date
8 changes: 1 addition & 7 deletions models/templates/dbt_template_merge_incremental_model.sql
Original file line number Diff line number Diff line change
@@ -1,9 +1,3 @@
{#
key notes on table model:
- file_format defaults to delta (TODO: confirm this is dune hive metastore setting)
- when providing file_format config to model, dbt fails on unable to support 'format' property
#}

{{ config(
alias = 'dbt_template_merge_incremental_model'
, materialized = 'incremental'
Expand Down Expand Up @@ -31,4 +25,4 @@ where
{%- endif %}
group by
block_number
, block_date
, block_date
8 changes: 1 addition & 7 deletions models/templates/dbt_template_table_model.sql
Original file line number Diff line number Diff line change
@@ -1,9 +1,3 @@
{#
key notes on table model:
- file_format defaults to delta (TODO: confirm this is dune hive metastore setting)
- when providing file_format config to model, dbt fails on unable to support 'format' property
#}

{{ config(
alias = 'dbt_template_table_model'
, materialized = 'table'
Expand All @@ -23,4 +17,4 @@ where
block_date >= now() - interval '1' day
group by
block_number
, block_date
, block_date
3 changes: 1 addition & 2 deletions models/templates/dbt_template_view_model.sql
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
}}

select
-- dummy comment to test GH workflows (CI attached to PR, deploy post-merge to main)
block_number
, block_date
, count(1) as total_tx_per_block
Expand All @@ -15,4 +14,4 @@ where
block_date >= now() - interval '1' day
group by
block_number
, block_date
, block_date
2 changes: 1 addition & 1 deletion profiles.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,4 @@ dbt_template:
cert: true
session_properties:
transformations: true
timezone: UTC
timezone: UTC