DbtShellTask

[mhmcdonald]

Thanks for contributing to Prefect!

Please describe your work and make sure your PR:

• adds new tests (if appropriate)
• updates CHANGELOG.md (if appropriate)
• updates docstrings for any new functions or function arguments, including docs/outline.toml for API reference docs (if appropriate)

Note that your PR will not be reviewed unless all three boxes are checked.

What does this PR change?

Adds a DbtShellTask, an extension of the ShellTask, to the task library. This task is intended for working with dbt

Why is this PR important?

The dbt community is very strong and growing in the cloud data warehousing space. There is a lot of potential overlap between the prefect and dbt communities.

[jlowin]

Thanks for the contribution @mhmcdonald! Awesome to see DBT get some first-class representation

[joshmeek]

Weird test failures 🤔all look to be related to logging in some way. I attempted to reproduce using this branch on a fresh conda env but everything was passing for me locally. cc @cicdw any ideas?

[cicdw]

A few requests for test hygene changes -- this is great though! I have a feeling this is going to be a very popular task

[joshmeek]

@mhmcdonald I think we've narrowed down why logging related tests are failing on this branch. It appears that the dbt library modifies global logging settings when imported

https://github.com/fishtown-analytics/dbt/blob/8686ab9a9ddffb63b64d3ee2861f132bf77f54c1/core/dbt/logger.py#L362-L363

# redirect stdlib logging to logbook
_redirect_std_logging()

which in the tests is causing all logs to be redirected elsewhere. Since the dbt library isn't actually used in these tasks and they shell out to using the CLI instead I think we should remove the dependency and instead add some type of CLI check where the importorskip is in order to determine if dbt is installed. Maybe doing something like the following:

from shutil import which
if which("dbt"):
    # good to go

If updated we can see if this resolves the test failures 😄

[codecov]

Codecov Report

Merging #2526 into master will increase coverage by 2.69%.
The diff coverage is 100.00%.

[joshmeek]

🚀

[cicdw]

Awesome!

[cicdw]

clever