Skip to content

Update Snowflake Setup Guide #6874

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Aug 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 20 additions & 15 deletions src/unify/data-graph/setup-guides/snowflake-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,25 +21,30 @@ Log in to Snowflake with admin privileges to provide Segment Data Graph with the

Segment requires the following settings to connect to your Snowflake warehouse.

<img src="/docs/unify/images/snowflake-setup.png" alt="Connect Snowflake to the Data Graph" width="5888"/>
<img src="/docs/unify/images/snowflake-setup.png" alt="Connect Snowflake to Data Graph" width="5888"/>

- **Account ID**: The Snowflake account ID that uniquely identifies your organization account.
- **Database Name**: The only database that Segment requires write access to in order to create tables for internal bookkeeping. This database is referred to as `segment_connection_db` in the script below.
- **Database**: The only database that Segment requires write access to in order to create tables for internal bookkeeping. This database is referred to as `segment_connection_db` in the script below.
- **Warehouse**: The [warehouse](https://docs.snowflake.com/en/user-guide/warehouses){:target="_blank”} in your Snowflake account that you want to use for Segment to run the SQL queries. This warehouse is referred to as `segment_connection_warehouse` in the script below.
- **Username**: The Snowflake user that Segment uses to run SQL in your warehouse. This user is referred to as `segment_connection_username` in the script below.
- **Password**: The password of the user above. This password is referred to as `segment_connection_password` in the script below.
- **Authentication**: There are 2 supported authentication methods:
1. **Key Pair**: This is the recommended method of authentication. You would need to first create the user and assign it a key pair following the instructions in the [Snowflake docs](https://docs.snowflake.com/en/user-guide/key-pair-auth). Then, follow the Segment docs above to set up Snowflake permissions and set the `segment_connections_username` variable in the SQL script to the user you just created.
2. **Password**: The password of the user above. This password is referred to as `segment_connection_password` in the script below.

## Set up Snowflake credentials

Segment recommends setting up a new Snowflake user and only giving this user permissions to access the required databases and schemas.

### Create Segment user and internal database
### Step 1: Create Segment user and internal database

Use the following steps to set up your Snowflake credentials:
The first step is to create a new Segment role and grant it the appropriate permissions. Run the SQL code block below in your SQL worksheet in Snowflake. It executes the following commands:

- Create a new role and user for Segment Data Graph.
- Create a new role and user for Segment Data Graph. This new role will have access to only the datasets you want to access from the Segment Data Graph.
- Grant the Segment user access to the warehouse of your choice. If you'd like to create a new warehouse, uncomment the SQL below.
- Create a new database for Segment Data Graph. Segment only requires write access to this one database to create a schema for internal bookkeeping, and to store checkpoint tables for the queries that are executed. Segment recommends creating an empty database for this purpose using the script below. This is also the database you'll be required to specify for the "Database Name" when connecting Snowflake with the Segment app.
- Create a new database for Segment Data Graph. **Segment only requires write access to this one database to create a schema for internal bookkeeping, and to store checkpoint tables for the queries that are executed**. Segment recommends creating an empty database for this purpose using the script below. This is also the database you'll be required to specify for the "Database Name" when connecting Snowflake with the Segment app.

> info ""
> The variables specified at the top of the code block with the `SET` command are placeholders and should be updated.

```
-- ********** SET UP THE FOLLOWING WAREHOUSE PERMISSIONS **********
Expand Down Expand Up @@ -91,11 +96,11 @@ GRANT CREATE SCHEMA ON DATABASE identifier($segment_connection_db) TO ROLE iden

```

### Grant access to other databases
### Step 2: Grant read-only access to other databases

Next, give the Segment user **read-only** access to all the other databases you want to use for Data Graph. You must grant access to the Profiles Sync database.
Next, give the Segment role **read-only** access to all the other databases you want to use for Data Graph including the **Profiles Sync database**

Run the SQL query below for **each** database you want to use for Data Graph:
Run the SQL query below for **each** database you want to use for Data Graph. **You may have to re-run this multiple times for each database you want to give access to**.

```

Expand All @@ -117,7 +122,7 @@ GRANT SELECT ON FUTURE MATERIALIZED VIEWS IN DATABASE identifier($linked_read_on

```

### (Optional) Restrict Snowflake schema access
### (Optional) Step 3: Restrict Snowflake schema access

If you want to restrict access to specific [Snowflake schemas and tables](https://docs.snowflake.com/en/user-guide/security-access-control-privileges#table-privileges){:target="_blank”}, run the following commands:

Expand All @@ -143,7 +148,7 @@ GRANT SELECT ON FUTURE MATERIALIZED VIEWS IN SCHEMA identifier($linked_read_only

```

### (If applicable) Update user acccess for Segment Reverse ETL schema
### (If applicable) Step 4: Update user acccess for Segment Reverse ETL schema

> warning ""
> This is only applicable if you choose to use an existing database as the Segment connection database that has also been used for Segment Reverse ETL.
Expand All @@ -157,7 +162,7 @@ Add the Snowflake table permissions by running the following commands:
```
-- If you want to use an existing database that already has Segment Reverse ETL schemas, you’ll need to run some additional steps below to grant the role access to the existing schemas.

SET retl_schema = concat($segment_internal_database,'.__segment_reverse_etl');
SET retl_schema = concat($segment_connection_db,'.__segment_reverse_etl');

GRANT USAGE ON SCHEMA identifier($retl_schema) TO ROLE identifier($segment_connection_role);

Expand All @@ -167,7 +172,7 @@ GRANT SELECT,INSERT,UPDATE,DELETE ON ALL TABLES IN SCHEMA identifier($retl_schem

```

### Confirm permissions
### Step 5: Confirm permissions

To verify you have set up the right permissions for a specific table, log in with the username and password you created for `SEGMENT_CONNECTION_USERNAME` and run the following command to verify the role you created has the correct permissions. If this command succeeds, you should be able to view the respective table.

Expand All @@ -179,6 +184,6 @@ set table_name = 'YOUR_DB.SCHEMA.TABLE';
USE ROLE identifier($segment_connection_role);
USE DATABASE identifier($linked_read_only_database) ;
SHOW SCHEMAS;
SELECT * FROM identifier($table) LIMIT 10;
SELECT * FROM identifier($table_name) LIMIT 10;

```
Binary file modified src/unify/images/snowflake-setup.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading