Feature/first class pg

[drewbanin]

Previously, the only targettype option was "redshift". Now, this type field can be set to "postgres" instead. Everything works almost exactly the same, but this differentiation is used for injecting variables into the compilation context.

In this branch, one new context variable is added, sql_now. In Redshift, this expands to getdate() and in Postgres, it expands to clock_timestamp(). While now() is more canonical (I think?), now() reflects the time of the start of the transaction, whereas clock_timestamp() reflects the wall-clock time when the statement is executed. The primary use case forsql_now` is for pre-and-post-hook audit records, we definitely want wall-clock time.

# profiles.yml
user:
  outputs:
    dev:
      type: postgres
      host: localhost
      user: Drew
....

An example of sql_where (and macros):

# audit_macros.sql
{% set audit_table = ref('audit') %}

{% macro log_run_with_status(status) %}

insert into {{ audit_table }}
    (invocation_id, run_started_at, run_at, model, status)
values
    ('{{ invocation_id }}', '{{ run_started_at }}', {{ sql_now }}, '{{ this.name }}', '{{ status }}')
--                                                        ^
--                                                        |
--                                      sql_now ----------+

{% endmacro %}

Which compiles to:

insert into "analytics"."audit"
    (invocation_id, run_started_at, run_at, model, status)
values
    ('{{ invocation_id }}', '{{ run_started_at }}', clock_timestamp(), 'accounts', 'started');

In sum, this is a pretty minor change, but it has pretty big ramifications for how we think about packages. Curious to hear your thoughts, @jthandy

[jthandy]

I follow the logic and generally am supportive. Here are the things I'm thinking about:

• I feel weird about having a wide open namespace where a bunch of random variables are just floating around. Instead, I would like to think about what a useful eventual namespace would be and begin to build towards that. For instance, I think this might be something like sql.now where the sql namespace contains a bunch of cross-database compatibility functions for sql. I can imagine a large number of other namespaces outside of sql. For instance, invocation could be a namespace, connection could be a namespace--and those are just the things that we're beginning to reference here.
• it's not clear to me how some of these variables are interpolated in compilation and some are interpolated at run time. I feel like there should be some syntactical difference between those, no? They are functionally very different. Or maybe it's intentionally transparent to the user? This is the first time that dbt compile will produce non-runnable SQL.

It might be good to chat about this stuff live.

[jthandy]

Sorry didn't mean to close!

[drewbanin]

Merging this. The macro stuff was more speculative, but the pg support is useful as-is