-
Notifications
You must be signed in to change notification settings - Fork 420
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: Stream on table resource (#3109)
<!-- Feel free to delete comments as you fill this in --> - add `snowflake_stream_on_table` resource - add `LastQueryId` to the context client - add a helper to insert data in a table - this is needed to track data in streams - extract common code to be reused in other stream resources - add a new function to parse a list of schema-level object identifiers - use enums in streams sdk - add a note to CHANGES_BEFORE_V1 explaining support of COPY GRANTS - minor docs fixes <!-- summary of changes --> ## Test Plan <!-- detail ways in which this PR has been tested or needs to be tested --> * [x] acceptance tests <!-- add more below if you think they are relevant --> ## References <!-- issues documentation links, etc --> https://docs.snowflake.com/en/sql-reference/sql/create-stream ## TODO - add remaining resources (external table, stage, view) - rework data source - adjust copy grants description - move baseModel to `ext`
- Loading branch information
1 parent
560ab6b
commit 97fa9b4
Showing
45 changed files
with
2,426 additions
and
108 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,147 @@ | ||
| --- | ||
| page_title: "snowflake_stream_on_table Resource - terraform-provider-snowflake" | ||
| subcategory: "" | ||
| description: |- | ||
| Resource used to manage streams on tables. For more information, check stream documentation https://docs.snowflake.com/en/sql-reference/sql/create-stream. | ||
| --- | ||
|
|
||
| !> **V1 release candidate** This resource was reworked and is a release candidate for the V1. We do not expect significant changes in it before the V1. We will welcome any feedback and adjust the resource if needed. Any errors reported will be resolved with a higher priority. We encourage checking this resource out before the V1 release. Please follow the [migration guide](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/MIGRATION_GUIDE.md#v0960--v0970) to use it. | ||
|
|
||
| # snowflake_stream_on_table (Resource) | ||
|
|
||
| Resource used to manage streams on tables. For more information, check [stream documentation](https://docs.snowflake.com/en/sql-reference/sql/create-stream). | ||
|
|
||
| ## Example Usage | ||
|
|
||
| ```terraform | ||
| resource "snowflake_table" "table" { | ||
| database = "database" | ||
| schema = "schema" | ||
| name = "name" | ||
| column { | ||
| type = "NUMBER(38,0)" | ||
| name = "id" | ||
| } | ||
| } | ||
| # resource with more fields set | ||
| resource "snowflake_stream_on_table" "stream" { | ||
| name = "stream" | ||
| schema = "schema" | ||
| database = "database" | ||
| copy_grants = true | ||
| table = snowflake_table.table.fully_qualified_name | ||
| append_only = "true" | ||
| show_initial_rows = "true" | ||
| at { | ||
| statement = "8e5d0ca9-005e-44e6-b858-a8f5b37c5726" | ||
| } | ||
| comment = "A stream." | ||
| } | ||
| ``` | ||
| -> **Note** Instead of using fully_qualified_name, you can reference objects managed outside Terraform by constructing a correct ID, consult [identifiers guide](https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest/docs/guides/identifiers#new-computed-fully-qualified-name-field-in-resources). | ||
| <!-- TODO(SNOW-1634854): include an example showing both methods--> | ||
|
|
||
| <!-- schema generated by tfplugindocs --> | ||
| ## Schema | ||
|
|
||
| ### Required | ||
|
|
||
| - `database` (String) The database in which to create the stream. Due to technical limitations (read more [here](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/docs/technical-documentation/identifiers_rework_design_decisions.md#known-limitations-and-identifier-recommendations)), avoid using the following characters: `|`, `.`, `(`, `)`, `"` | ||
| - `name` (String) Specifies the identifier for the stream; must be unique for the database and schema in which the stream is created. Due to technical limitations (read more [here](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/docs/technical-documentation/identifiers_rework_design_decisions.md#known-limitations-and-identifier-recommendations)), avoid using the following characters: `|`, `.`, `(`, `)`, `"` | ||
| - `schema` (String) The schema in which to create the stream. Due to technical limitations (read more [here](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/docs/technical-documentation/identifiers_rework_design_decisions.md#known-limitations-and-identifier-recommendations)), avoid using the following characters: `|`, `.`, `(`, `)`, `"` | ||
| - `table` (String) Specifies an identifier for the table the stream will monitor. Due to technical limitations (read more [here](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/docs/technical-documentation/identifiers_rework_design_decisions.md#known-limitations-and-identifier-recommendations)), avoid using the following characters: `|`, `.`, `(`, `)`, `"` | ||
|
|
||
| ### Optional | ||
|
|
||
| - `append_only` (String) Specifies whether this is an append-only stream. Available options are: "true" or "false". When the value is not set in the configuration the provider will put "default" there which means to use the Snowflake default for this value. | ||
| - `at` (Block List, Max: 1) This field specifies that the request is inclusive of any changes made by a statement or transaction with a timestamp equal to the specified parameter. Due to Snowflake limitations, the provider does not detect external changes on this field. External changes for this field won't be detected. In case you want to apply external changes, you can re-create the resource manually using "terraform taint". (see [below for nested schema](#nestedblock--at)) | ||
| - `before` (Block List, Max: 1) This field specifies that the request refers to a point immediately preceding the specified parameter. This point in time is just before the statement, identified by its query ID, is completed. Due to Snowflake limitations, the provider does not detect external changes on this field. External changes for this field won't be detected. In case you want to apply external changes, you can re-create the resource manually using "terraform taint". (see [below for nested schema](#nestedblock--before)) | ||
| - `comment` (String) Specifies a comment for the stream. | ||
| - `copy_grants` (Boolean) Retains the access permissions from the original stream when a new stream is created using the OR REPLACE clause. Use only if the resource is already managed by Terraform. Otherwise, this field is skipped. | ||
| - `show_initial_rows` (String) Specifies whether to return all existing rows in the source table as row inserts the first time the stream is consumed. Available options are: "true" or "false". When the value is not set in the configuration the provider will put "default" there which means to use the Snowflake default for this value. External changes for this field won't be detected. In case you want to apply external changes, you can re-create the resource manually using "terraform taint". | ||
|
|
||
| ### Read-Only | ||
|
|
||
| - `describe_output` (List of Object) Outputs the result of `DESCRIBE STREAM` for the given stream. (see [below for nested schema](#nestedatt--describe_output)) | ||
| - `fully_qualified_name` (String) Fully qualified name of the resource. For more information, see [object name resolution](https://docs.snowflake.com/en/sql-reference/name-resolution). | ||
| - `id` (String) The ID of this resource. | ||
| - `show_output` (List of Object) Outputs the result of `SHOW STREAMS` for the given stream. (see [below for nested schema](#nestedatt--show_output)) | ||
|
|
||
| <a id="nestedblock--at"></a> | ||
| ### Nested Schema for `at` | ||
|
|
||
| Optional: | ||
|
|
||
| - `offset` (String) Specifies the difference in seconds from the current time to use for Time Travel, in the form -N where N can be an integer or arithmetic expression (e.g. -120 is 120 seconds, -30*60 is 1800 seconds or 30 minutes). | ||
| - `statement` (String) Specifies the query ID of a statement to use as the reference point for Time Travel. This parameter supports any statement of one of the following types: DML (e.g. INSERT, UPDATE, DELETE), TCL (BEGIN, COMMIT transaction), SELECT. | ||
| - `stream` (String) Specifies the identifier (i.e. name) for an existing stream on the queried table or view. The current offset in the stream is used as the AT point in time for returning change data for the source object. | ||
| - `timestamp` (String) Specifies an exact date and time to use for Time Travel. The value must be explicitly cast to a TIMESTAMP, TIMESTAMP_LTZ, TIMESTAMP_NTZ, or TIMESTAMP_TZ data type. | ||
|
|
||
|
|
||
| <a id="nestedblock--before"></a> | ||
| ### Nested Schema for `before` | ||
|
|
||
| Optional: | ||
|
|
||
| - `offset` (String) Specifies the difference in seconds from the current time to use for Time Travel, in the form -N where N can be an integer or arithmetic expression (e.g. -120 is 120 seconds, -30*60 is 1800 seconds or 30 minutes). | ||
| - `statement` (String) Specifies the query ID of a statement to use as the reference point for Time Travel. This parameter supports any statement of one of the following types: DML (e.g. INSERT, UPDATE, DELETE), TCL (BEGIN, COMMIT transaction), SELECT. | ||
| - `stream` (String) Specifies the identifier (i.e. name) for an existing stream on the queried table or view. The current offset in the stream is used as the AT point in time for returning change data for the source object. | ||
| - `timestamp` (String) Specifies an exact date and time to use for Time Travel. The value must be explicitly cast to a TIMESTAMP, TIMESTAMP_LTZ, TIMESTAMP_NTZ, or TIMESTAMP_TZ data type. | ||
|
|
||
|
|
||
| <a id="nestedatt--describe_output"></a> | ||
| ### Nested Schema for `describe_output` | ||
|
|
||
| Read-Only: | ||
|
|
||
| - `base_tables` (List of String) | ||
| - `comment` (String) | ||
| - `created_on` (String) | ||
| - `database_name` (String) | ||
| - `invalid_reason` (String) | ||
| - `mode` (String) | ||
| - `name` (String) | ||
| - `owner` (String) | ||
| - `owner_role_type` (String) | ||
| - `schema_name` (String) | ||
| - `source_type` (String) | ||
| - `stale` (String) | ||
| - `stale_after` (String) | ||
| - `table_name` (String) | ||
| - `type` (String) | ||
|
|
||
|
|
||
| <a id="nestedatt--show_output"></a> | ||
| ### Nested Schema for `show_output` | ||
|
|
||
| Read-Only: | ||
|
|
||
| - `base_tables` (List of String) | ||
| - `comment` (String) | ||
| - `created_on` (String) | ||
| - `database_name` (String) | ||
| - `invalid_reason` (String) | ||
| - `mode` (String) | ||
| - `name` (String) | ||
| - `owner` (String) | ||
| - `owner_role_type` (String) | ||
| - `schema_name` (String) | ||
| - `source_type` (String) | ||
| - `stale` (String) | ||
| - `stale_after` (String) | ||
| - `table_name` (String) | ||
| - `type` (String) | ||
|
|
||
| ## Import | ||
|
|
||
| Import is supported using the following syntax: | ||
|
|
||
| ```shell | ||
| terraform import snowflake_stream_on_table.example '"<database_name>"."<schema_name>"."<stream_name>"' | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| terraform import snowflake_stream_on_table.example '"<database_name>"."<schema_name>"."<stream_name>"' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| resource "snowflake_table" "table" { | ||
| database = "database" | ||
| schema = "schema" | ||
| name = "name" | ||
|
|
||
| column { | ||
| type = "NUMBER(38,0)" | ||
| name = "id" | ||
| } | ||
| } | ||
|
|
||
|
|
||
| # resource with more fields set | ||
| resource "snowflake_stream_on_table" "stream" { | ||
| name = "stream" | ||
| schema = "schema" | ||
| database = "database" | ||
|
|
||
| copy_grants = true | ||
| table = snowflake_table.table.fully_qualified_name | ||
| append_only = "true" | ||
| show_initial_rows = "true" | ||
|
|
||
| at { | ||
| statement = "8e5d0ca9-005e-44e6-b858-a8f5b37c5726" | ||
| } | ||
|
|
||
| comment = "A stream." | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
42 changes: 0 additions & 42 deletions
42
pkg/acceptance/bettertestspoc/assert/objectassert/stream_snowflake_gen.go
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
Oops, something went wrong.