Make the DBT dependency optional

[erezsh]

Alternative title: Sudden dependency bloat in data-diff 4.0 and up

Overview of the problem

Version 0.3.2 of data-diff required 11 dependencies total, most of which are already very popular libraries, like "rich", "click", and "dsnparse".

Version 0.4.0 of data-diff introduced 66 new dependencies (!).

This is due to the introduction of the mandatory "dbt" dependency, even for users who don't plan to use the --dbt switch.

Implications of not fixing this

1. Minimum load time for the tool increased from 0.165 seconds to 2.410 seconds (!!)

2. Extra requirements might collide with the requirements of other Python libraries, and make it harder for our non-dbt users to have data-diff installed alongside them.

3. Users who would consider using data-diff as a lightweight tool might be put out by the large number of dependencies.

4. That is also true in regards to being included by default in package managers. For example, in Ubuntu you can apt install python3-lark to install Lark. It would have been much harder to include it if it had many dependencies.

Implications of fixing this

Users that don't already have dbt installed, and run --dbt, will see a message telling them to use pip install data-diff[dbt] which will install dbt.

However, there is absolutely no point in using the --dbt switch if you don't already have dbt installed and configured.

Conclusion

I think there is no reason to keep dbt as a dependency, but very good reasons to make it optional.

[dlawin]

it may be possible to remove the dependency, the dbt package is only used for one method here:

data-diff/data_diff/dbt.py

Line 311 in 584206b

rendered_credentials = ProfileRenderer().render_data(credentials)

to render dbt-jinja {{ env_vars('')}}

[williebsweet]

@erezsh Thoughts on making this part of a guided setup?

In this scenario, someone installs data-diff, then is asked a series of questions that will install relevant dependencies for them, e.g., which databases do they use, do they use dbt, if dbt - what is their prod db and schema, etc.

[erezsh]

@williebsweet Generally, it's a good idea, but it belongs in a separate issue.

In this particular case, I'll reiterate my opinion -

there is absolutely no point in using the --dbt switch if you don't already have dbt installed and configured.

[dlawin]

I noticed numpy was still in the poetry.lock file when incrementing the duckdb version here b939b9a

It's removed now, so that should help with this effort