Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add stream on directory table #3129

Merged
merged 13 commits into from
Oct 22, 2024
48 changes: 30 additions & 18 deletions MIGRATION_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,28 +7,24 @@ across different versions.
> [!TIP]
> We highly recommend upgrading the versions one by one instead of bulk upgrades.

## v0.96.0 ➞ v0.97.0

### *(new feature)* snowflake_stream_on_table, snowflake_stream_on_external_table resource
## v0.97.0 ➞ v0.98.0

To enhance clarity and functionality, the new resources `snowflake_stream_on_table`, `snowflake_stream_on_external_table` and `snowflake_stream_on_directory_table` have been introduced to replace the previous `snowflake_stream`. Recognizing that the old resource carried multiple responsibilities within a single entity, we opted to divide it into more specialized resources.
### *(new feature)* snowflake_stream_on_directory_table resource
Continuing changes made in v0.96, to enhance clarity and functionality, the new resource `snowflake_stream_on_directory_table` has been introduced to replace the previous `snowflake_stream`. Recognizing that the old resource carried multiple responsibilities within a single entity, we opted to divide it into more specialized resources.
The newly introduced resources are aligned with the latest Snowflake documentation at the time of implementation, and adhere to our [new conventions](#general-changes).
This segregation was based on the object on which the stream is created. The mapping between SQL statements and the resources is the following:
- `ON TABLE <table_name>` -> `snowflake_stream_on_table`
- `ON EXTERNAL TABLE <external_table_name>` -> `snowflake_stream_on_external_table` (this was previously not supported)
- `ON TABLE <table_name>` -> `snowflake_stream_on_table` (added in v0.96)
- `ON EXTERNAL TABLE <external_table_name>` -> `snowflake_stream_on_external_table` (this was previously not supported, added in v0.96)
- `ON STAGE <stage_name>` -> `snowflake_stream_on_directory_table`

The resources for streams on directory tables and streams on views will be implemented in the future releases.

To use the new `stream_on_table`, change the old `stream` from
To use the new `stream_on_directory_table`, change the old `stream` from
```terraform
resource "snowflake_stream" "stream" {
name = "stream"
schema = "schema"
database = "database"

on_table = snowflake_table.table.fully_qualified_name
append_only = true
on_stage = snowflake_stage.stage.fully_qualified_name

comment = "A stream."
}
Expand All @@ -37,27 +33,42 @@ resource "snowflake_stream" "stream" {
to

```terraform
resource "snowflake_stream_on_table" "stream" {
resource "snowflake_stream_on_directory_table" "stream" {
name = "stream"
schema = "schema"
database = "database"

table = snowflake_table.table.fully_qualified_name
append_only = "true"
stage = snowflake_stage.stage.fully_qualified_name

comment = "A stream."
}
```

Then, follow our [Resource migration guide](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/docs/technical-documentation/resource_migration.md).

To use the new `stream_on_directory_table`, change the old `stream` from

## v0.96.0 ➞ v0.97.0
sfc-gh-jcieslak marked this conversation as resolved.
Show resolved Hide resolved

### *(new feature)* snowflake_stream_on_table, snowflake_stream_on_external_table resource

To enhance clarity and functionality, the new resources `snowflake_stream_on_table`, `snowflake_stream_on_external_table` and `snowflake_stream_on_directory_table` have been introduced to replace the previous `snowflake_stream`. Recognizing that the old resource carried multiple responsibilities within a single entity, we opted to divide it into more specialized resources.
The newly introduced resources are aligned with the latest Snowflake documentation at the time of implementation, and adhere to our [new conventions](#general-changes).
This segregation was based on the object on which the stream is created. The mapping between SQL statements and the resources is the following:
- `ON TABLE <table_name>` -> `snowflake_stream_on_table`
- `ON EXTERNAL TABLE <external_table_name>` -> `snowflake_stream_on_external_table` (this was previously not supported)
- `ON STAGE <stage_name>` -> `snowflake_stream_on_directory_table`
sfc-gh-asawicki marked this conversation as resolved.
Show resolved Hide resolved

The resources for streams on directory tables and streams on views will be implemented in the future releases.

To use the new `stream_on_table`, change the old `stream` from
```terraform
resource "snowflake_stream" "stream" {
name = "stream"
schema = "schema"
database = "database"

on_stage = snowflake_stage.stage.fully_qualified_name
on_table = snowflake_table.table.fully_qualified_name
append_only = true

comment = "A stream."
}
Expand All @@ -66,12 +77,13 @@ resource "snowflake_stream" "stream" {
to

```terraform
resource "snowflake_stream_on_directory_table" "stream" {
resource "snowflake_stream_on_table" "stream" {
name = "stream"
schema = "schema"
database = "database"

stage = snowflake_stage.stage.fully_qualified_name
table = snowflake_table.table.fully_qualified_name
append_only = "true"

comment = "A stream."
}
Expand Down
3 changes: 2 additions & 1 deletion docs/resources/stream_on_directory_table.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: |-
Resource used to manage streams on directory tables. For more information, check stream documentation https://docs.snowflake.com/en/sql-reference/sql/create-stream.
---

!> **V1 release candidate** This resource was reworked and is a release candidate for the V1. We do not expect significant changes in it before the V1. We will welcome any feedback and adjust the resource if needed. Any errors reported will be resolved with a higher priority. We encourage checking this resource out before the V1 release. Please follow the [migration guide](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/MIGRATION_GUIDE.md#v0960--v0970) to use it.
!> **V1 release candidate** This resource was reworked and is a release candidate for the V1. We do not expect significant changes in it before the V1. We will welcome any feedback and adjust the resource if needed. Any errors reported will be resolved with a higher priority. We encourage checking this resource out before the V1 release. Please follow the [migration guide](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/MIGRATION_GUIDE.md#v0970--v0980) to use it.

# snowflake_stream_on_directory_table (Resource)

Expand Down Expand Up @@ -72,6 +72,7 @@ resource "snowflake_stream_on_directory_table" "stream" {
- `fully_qualified_name` (String) Fully qualified name of the resource. For more information, see [object name resolution](https://docs.snowflake.com/en/sql-reference/name-resolution).
- `id` (String) The ID of this resource.
- `show_output` (List of Object) Outputs the result of `SHOW STREAMS` for the given stream. (see [below for nested schema](#nestedatt--show_output))
- `stale` (Boolean) Indicated if the stream is stale. When Terraform detects that the stream is stale, the stream is recreated with `CREATE OR REPLACE`. Read more on stream staleness in Snowflake [docs](https://docs.snowflake.com/en/user-guide/streams-intro#data-retention-period-and-staleness).

<a id="nestedatt--describe_output"></a>
### Nested Schema for `describe_output`
Expand Down
1 change: 1 addition & 0 deletions docs/resources/stream_on_external_table.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ resource "snowflake_stream_on_external_table" "stream" {
- `fully_qualified_name` (String) Fully qualified name of the resource. For more information, see [object name resolution](https://docs.snowflake.com/en/sql-reference/name-resolution).
- `id` (String) The ID of this resource.
- `show_output` (List of Object) Outputs the result of `SHOW STREAMS` for the given stream. (see [below for nested schema](#nestedatt--show_output))
- `stale` (Boolean) Indicated if the stream is stale. When Terraform detects that the stream is stale, the stream is recreated with `CREATE OR REPLACE`. Read more on stream staleness in Snowflake [docs](https://docs.snowflake.com/en/user-guide/streams-intro#data-retention-period-and-staleness).
sfc-gh-asawicki marked this conversation as resolved.
Show resolved Hide resolved

<a id="nestedblock--at"></a>
### Nested Schema for `at`
Expand Down
1 change: 1 addition & 0 deletions docs/resources/stream_on_table.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ resource "snowflake_stream_on_table" "stream" {
- `fully_qualified_name` (String) Fully qualified name of the resource. For more information, see [object name resolution](https://docs.snowflake.com/en/sql-reference/name-resolution).
- `id` (String) The ID of this resource.
- `show_output` (List of Object) Outputs the result of `SHOW STREAMS` for the given stream. (see [below for nested schema](#nestedatt--show_output))
- `stale` (Boolean) Indicated if the stream is stale. When Terraform detects that the stream is stale, the stream is recreated with `CREATE OR REPLACE`. Read more on stream staleness in Snowflake [docs](https://docs.snowflake.com/en/user-guide/streams-intro#data-retention-period-and-staleness).

<a id="nestedblock--at"></a>
### Nested Schema for `at`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,4 @@ var allResourceSchemaDefs = []ResourceSchemaDef{
name: "StreamOnDirectoryTable",
schema: resources.StreamOnDirectoryTable().Schema,
},
// {
// name: "StreamOnView",
// schema: resources.StreamOnView().Schema,
// },
}

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

This file was deleted.

Loading
Loading