Skip to content

[CT-2850] [Feature] Raise a warning when --state path == --target-path #8160

Closed
@dbeatty10

Description

Is this your first time submitting a feature request?

  • I have read the expectations for open source contributors
  • I have searched the existing issues, and I could not find an existing issue for this feature
  • I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

Mentioned by @aranke here and here:

Something else to note here is that you cannot use the same target directory for both the source and target

You get a very weird error that source and target states are the same, and so it's a no-op.

I ran into this during local development, so think this warning needs to be surfaced loudly.

This happens because

dbt now (over)writes manifest.json during parsing.
[…]
during the subsequent run (step 4) in the reproduction case, it overwrites target/manifest.json, then reads it back in to detect changes, and finds no changes.

from @jtcohen6 in #7790 (comment).

Describe alternatives you've considered

Simple solution that will work:

  1. Write output to tmp directory during dbt run
  2. Copy tmp directory to the output directory at the end of the dbt run

Who will this benefit?

Anyone using state in any form (including deferral, clone) reusing the same artifact directory.

Are you interested in contributing this feature?

Yes

Anything else?

There are 3 possible solutions here, from easiest to hardest:

  1. Raise a warning when the input and output artifact directories are the same
  2. Write to tmp directory as mentioned above
  3. Investigate root cause and fix underlying issue

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions