Add artifact creation and asset tracking to per-node dbt orchestrator#20743
Merged
desertaxle merged 13 commits intomainfrom Feb 18, 2026
Merged
Add artifact creation and asset tracking to per-node dbt orchestrator#20743desertaxle merged 13 commits intomainfrom
desertaxle merged 13 commits intomainfrom
Conversation
desertaxle
requested review from
chrisguidry,
cicdw and
zzstoatzz
as code owners
desertaxle
changed the title
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 38574147fc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
zzstoatzz
approved these changes
Feb 18, 2026
Collaborator
zzstoatzz
left a comment
zzstoatzz
left a comment
There was a problem hiding this comment.
lgtm! a couple non-blocking observations
Base automatically changed from
feat/per-node-dbt-orchestration-phase-8
to
main
February 18, 2026 19:29
Introduces TestStrategy enum (IMMEDIATE, DEFERRED, SKIP) that controls when dbt test and unit_test nodes execute during orchestrated builds. SKIP is the default for backward compatibility. - Add get_test_nodes() and filter_test_nodes() to ManifestParser - Support both NodeType.Test and NodeType.Unit in test collection - Suppress dbt indirect test selection (indirect_selection='empty') in PER_WAVE mode with narrowed TypeError fallback for legacy executors - Exclude test nodes from caching in PER_NODE mode - Add schema.yml fixtures for integration test determinism - Add 48 unit tests and 8 integration tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…me accuracy Adds summary markdown artifacts, run_results.json output, compiled code inclusion, and asset lineage tracking to the per-node dbt orchestrator. Fixes create_markdown_artifact to use _sync=True so it works correctly in async flow contexts, and prefers dbt-reported execution_time over orchestrator-measured duration_seconds in run_results output. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…in PER_WAVE Disable caching for nodes whose upstream dependencies were not executed in the current run (e.g. excluded by select=...), preventing stale downstream results from being reused when an upstream changes separately. In PER_WAVE IMMEDIATE mode, use per-node artifact status to distinguish individually successful models from failed tests in the same wave, so test failures no longer incorrectly propagate as model failures to downstream waves. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add relation_name to DbtNodeCachePolicy key material so that runs targeting different dbt environments (e.g. dev vs prod schemas) produce distinct cache keys and cannot incorrectly skip execution. Always pass --target-path to dbt source freshness using the resolved output target so that a custom settings.target_path is honoured and sources.json is written to the expected location. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…it ref The duplicate _TEST_NODE_TYPES assignment unconditionally accessed NodeType.Unit, breaking module import on dbt-core 1.7.x. Remove the duplicate; the guarded frozenset using _UNIT_TYPE is sufficient. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…strator Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ator Patching a name must target the module that has already imported it. After moving create_markdown_artifact, create_asset_for_node, get_upstream_assets_for_node, and get_compiled_code_for_node from deferred imports to top-level imports in _orchestrator, the patch targets need to reference prefect_dbt.core._orchestrator.* rather than their source modules. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
desertaxle
force-pushed
the
feat/per-node-dbt-orchestration-phase-9
branch
from
f1ba55f to
50a441e
Compare
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds Prefect artifact creation and asset lineage tracking to the per-node dbt orchestrator, along with configurable test execution strategies and several correctness fixes.
Changes
Artifact support (
_artifacts.py)create_summary_markdown: builds a markdown summary ofrun_build()results, posted as a Prefect artifact at the end of each buildcreate_run_results_dict/write_run_results_json: emits a dbt-compatiblerun_results.json(schema v6) for downstream tooling integrationcreate_asset_for_node/get_upstream_assets_for_node: creates PrefectAssetobjects with lineage tracking for models, seeds, and snapshotsget_compiled_code_for_node: optionally attaches compiled SQL to asset descriptionsNew orchestrator options
create_summary_artifact(defaultTrue): posts a markdown build summary to the Prefect UIwrite_run_results(defaultFalse): writesrun_results.jsonto the target directoryinclude_compiled_code(defaultFalse): includes compiled SQL in asset descriptionsAsset lineage
relation_nameare wrapped inMaterializingTask, tracking the materialized database object as a Prefect assetAssetContexton successTest execution strategies (
TestStrategyenum)SKIP(default): tests excluded entirelyIMMEDIATE: tests interleaved with models via Kahn's algorithm — each test runs in the wave after all its parent models completeDEFERRED: all model waves execute first, then a final test waveBug fixes
NodeType.Unitreference that causedAttributeErroron dbt-core 1.7.xPER_WAVEmode — test failures no longer cause downstream model waves to be skippedtarget_pathresolution when an explicitmanifest_pathis providedrelation_nameincluded in cache key so relation renames correctly invalidate the cache