Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add shim to correct test seed data types in Snowflake #2

Merged
merged 2 commits into from
Aug 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions integration_tests/dbt_project.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ models:
relation: true

vars:
events_table: fullstory_events_integration_tests
dbt_date:time_zone: Etc/UTC
fullstory_replay_host: app.fullstory.com
fullstory_events_types:
Expand Down Expand Up @@ -64,8 +65,9 @@ seeds:
dbt_fullstory_integration_tests:
+enabled: true

# Override column types on all
+column_types:
event_time: datetime
processed_time: datetime
updated_time: datetime
fullstory_events_integration_tests:
+alias: "{% if target.type == 'snowflake' %}fullstory_events_integration_tests_raw{% endif %}"
+column_types:
event_time: datetime
processed_time: datetime
updated_time: datetime
7 changes: 7 additions & 0 deletions integration_tests/models/_snowflake_events_shim.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
version: 2

models:
- name: snowflake_events_shim
config:
enabled: "{{ (target.type == 'snowflake') | as_bool }}"
alias: fullstory_events_integration_tests
13 changes: 13 additions & 0 deletions integration_tests/models/snowflake_events_shim.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
select
event_id,
event_time,
processed_time,
updated_time,
device_id,
session_id,
view_id,
event_type,
parse_json(event_properties) as event_properties,
source_type,
parse_json(source_properties) as source_properties
from {{ ref('fullstory_events_integration_tests') }}
2 changes: 1 addition & 1 deletion models/staging/_events__sources.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ sources:
tables:
- name: events
description: The FullStory events table.
identifier: "{% if target.type == 'bigquery' %}fullstory_events_{{ modules.re.sub('[^a-zA-Z\\d_\\-]', '_', target.name) }}{% else %}events{% endif %}"
identifier: "{% if var('events_table', '') %}{{ var('events_table') }}{% else %}{% if target.type == 'bigquery' %}fullstory_events_{{ modules.re.sub('[^a-zA-Z\\d_\\-]', '_', target.name) }}{% else %}events{% endif %}{% endif %}"
columns:
- name: event_id
- name: device_id
Expand Down