Skip to content

Commit

Permalink
fix: Fix cortex search service (#2904)
Browse files Browse the repository at this point in the history
- fix integration tests
- fix datasource (multiple errors)
- add missing force news
- add missing examples and imports
- add disclaimer
- fix resource logic
- skip resource test (error with no db in session)
- fix datasource acceptance test
- fix docs around grants
- add tests for grants (normal, plural, and ownership)
  • Loading branch information
sfc-gh-asawicki authored Jul 2, 2024
1 parent 43aa89f commit 763d06c
Show file tree
Hide file tree
Showing 30 changed files with 616 additions and 234 deletions.
48 changes: 45 additions & 3 deletions docs/data-sources/cortex_search_services.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,53 @@ description: |-
---

!> **Disclaimer for Cortex Search service** Note that Cortex Search is a Private Preview feature as such, should be used only with non-production data even when using Snowflake's Terraform Provider. Also, note that the Terraform Provider is not covered by Snowflake's support team; the Product and Engineering teams are available for any questions. However, please contact the Cortex Search team for any issues with this object.

# snowflake_cortex_search_services (Data Source)



## Example Usage

```terraform
# Simple usage
data "snowflake_cortex_search_services" "simple" {
}
output "simple_output" {
value = data.snowflake_cortex_search_services.simple.cortex_search_services
}
# Filtering (like)
data "snowflake_cortex_search_services" "like" {
like = "some-name"
}
output "like_output" {
value = data.snowflake_cortex_search_services.like.cortex_search_services
}
# Filtering (starts_with)
data "snowflake_cortex_search_services" "starts_with" {
starts_with = "prefix-"
}
output "starts_with_output" {
value = data.snowflake_cortex_search_services.starts_with.cortex_search_services
}
# Filtering (limit)
data "snowflake_cortex_search_services" "limit" {
limit {
rows = 10
from = "prefix-"
}
}
output "limit_output" {
value = data.snowflake_cortex_search_services.limit.cortex_search_services
}
```

<!-- schema generated by tfplugindocs -->
## Schema
Expand All @@ -23,7 +65,7 @@ description: |-

### Read-Only

- `cortexSearchServices` (List of Object) Holds the output of SHOW CORTEX SEARCH SERVICES. (see [below for nested schema](#nestedatt--cortexSearchServices))
- `cortex_search_services` (List of Object) Holds the output of SHOW CORTEX SEARCH SERVICES. (see [below for nested schema](#nestedatt--cortex_search_services))
- `id` (String) The ID of this resource.

<a id="nestedblock--in"></a>
Expand All @@ -48,8 +90,8 @@ Optional:
- `from` (String) Specifies a **case-sensitive** pattern that is used to match object name. After the first match, the limit on the number of rows will be applied.


<a id="nestedatt--cortexSearchServices"></a>
### Nested Schema for `cortexSearchServices`
<a id="nestedatt--cortex_search_services"></a>
### Nested Schema for `cortex_search_services`

Read-Only:

Expand Down
54 changes: 53 additions & 1 deletion docs/resources/cortex_search_service.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,54 @@ description: |-
---

!> **Disclaimer for Cortex Search service** Note that Cortex Search is a Private Preview feature as such, should be used only with non-production data even when using Snowflake's Terraform Provider. Also, note that the Terraform Provider is not covered by Snowflake's support team; the Product and Engineering teams are available for any questions. However, please contact the Cortex Search team for any issues with this object.

# snowflake_cortex_search_service (Resource)



## Example Usage

```terraform
## Basic
resource "snowflake_database" "test" {
name = "some_database"
}
resource "snowflake_schema" "test" {
database = snowflake_database.test.name
name = "some_schema"
}
resource "snowflake_table" "test" {
database = snowflake_database.test.name
schema = snowflake_schema.test.name
name = "some_table"
change_tracking = true
column {
name = "ID"
type = "NUMBER(38,0)"
}
column {
name = "SOME_TEXT"
type = "VARCHAR"
}
}
resource "snowflake_cortex_search_service" "test" {
depends_on = [snowflake_table.test]
database = snowflake_database.test.name
schema = snowflake_schema.test.name
name = "some_name"
on = "SOME_TEXT"
target_lag = "2 minutes"
warehouse = "some_warehouse"
query = "SELECT SOME_TEXT FROM \"some_database\".\"some_schema\".\"some_table\""
comment = "some comment"
}
```

<!-- schema generated by tfplugindocs -->
## Schema
Expand All @@ -26,9 +69,18 @@ description: |-

### Optional

- `attributes` (List of String) Specifies the list of columns in the base table to enable filtering on when issuing queries to the service.
- `attributes` (Set of String) Specifies the list of columns in the base table to enable filtering on when issuing queries to the service.
- `comment` (String) Specifies a comment for the Cortex search service.

### Read-Only

- `created_on` (String) Creation date for the given Cortex search service.
- `id` (String) The ID of this resource.

## Import

Import is supported using the following syntax:

```shell
terraform import snowflake_cortex_search_service.example 'dbName|schemaName|fileFormatName'
```
6 changes: 3 additions & 3 deletions docs/resources/grant_ownership.md
Original file line number Diff line number Diff line change
Expand Up @@ -264,14 +264,14 @@ Optional:
- `all` (Block List, Max: 1) Configures the privilege to be granted on all objects in either a database or schema. (see [below for nested schema](#nestedblock--on--all))
- `future` (Block List, Max: 1) Configures the privilege to be granted on all objects in either a database or schema. (see [below for nested schema](#nestedblock--on--future))
- `object_name` (String) Specifies the identifier for the object on which you are transferring ownership.
- `object_type` (String) Specifies the type of object on which you are transferring ownership. Available values are: AGGREGATION POLICY | ALERT | AUTHENTICATION POLICY | COMPUTE POOL | CORTEX SEARCH SERVICE | DATA METRIC FUNCTION | DATABASE | DATABASE ROLE | DYNAMIC TABLE | EVENT TABLE | EXTERNAL TABLE | EXTERNAL VOLUME | FAILOVER GROUP | FILE FORMAT | FUNCTION | GIT REPOSITORY | HYBRID TABLE | ICEBERG TABLE | IMAGE REPOSITORY | INTEGRATION | MATERIALIZED VIEW | NETWORK POLICY | NETWORK RULE | PACKAGES POLICY | PIPE | PROCEDURE | MASKING POLICY | PASSWORD POLICY | PROJECTION POLICY | REPLICATION GROUP | ROLE | ROW ACCESS POLICY | SCHEMA | SESSION POLICY | SECRET | SEQUENCE | STAGE | STREAM | TABLE | TAG | TASK | USER | VIEW | WAREHOUSE
- `object_type` (String) Specifies the type of object on which you are transferring ownership. Available values are: AGGREGATION POLICY | ALERT | AUTHENTICATION POLICY | COMPUTE POOL | DATA METRIC FUNCTION | DATABASE | DATABASE ROLE | DYNAMIC TABLE | EVENT TABLE | EXTERNAL TABLE | EXTERNAL VOLUME | FAILOVER GROUP | FILE FORMAT | FUNCTION | GIT REPOSITORY | HYBRID TABLE | ICEBERG TABLE | IMAGE REPOSITORY | INTEGRATION | MATERIALIZED VIEW | NETWORK POLICY | NETWORK RULE | PACKAGES POLICY | PIPE | PROCEDURE | MASKING POLICY | PASSWORD POLICY | PROJECTION POLICY | REPLICATION GROUP | ROLE | ROW ACCESS POLICY | SCHEMA | SESSION POLICY | SECRET | SEQUENCE | STAGE | STREAM | TABLE | TAG | TASK | USER | VIEW | WAREHOUSE

<a id="nestedblock--on--all"></a>
### Nested Schema for `on.all`

Required:

- `object_type_plural` (String) Specifies the type of object in plural form on which you are transferring ownership. Available values are: AGGREGATION POLICIES | ALERTS | AUTHENTICATION POLICIES | COMPUTE POOLS | CORTEX SEARCH SERVICES | DATA METRIC FUNCTIONS | DATABASES | DATABASE ROLES | DYNAMIC TABLES | EVENT TABLES | EXTERNAL TABLES | EXTERNAL VOLUMES | FAILOVER GROUPS | FILE FORMATS | FUNCTIONS | GIT REPOSITORIES | HYBRID TABLES | ICEBERG TABLES | IMAGE REPOSITORIES | INTEGRATIONS | MATERIALIZED VIEWS | NETWORK POLICIES | NETWORK RULES | PACKAGES POLICIES | PIPES | PROCEDURES | MASKING POLICIES | PASSWORD POLICIES | PROJECTION POLICIES | REPLICATION GROUPS | ROLES | ROW ACCESS POLICIES | SCHEMAS | SESSION POLICIES | SECRETS | SEQUENCES | STAGES | STREAMS | TABLES | TAGS | TASKS | USERS | VIEWS | WAREHOUSES. For more information head over to [Snowflake documentation](https://docs.snowflake.com/en/sql-reference/sql/grant-ownership#required-parameters).
- `object_type_plural` (String) Specifies the type of object in plural form on which you are transferring ownership. Available values are: AGGREGATION POLICIES | ALERTS | AUTHENTICATION POLICIES | COMPUTE POOLS | DATA METRIC FUNCTIONS | DATABASES | DATABASE ROLES | DYNAMIC TABLES | EVENT TABLES | EXTERNAL TABLES | EXTERNAL VOLUMES | FAILOVER GROUPS | FILE FORMATS | FUNCTIONS | GIT REPOSITORIES | HYBRID TABLES | ICEBERG TABLES | IMAGE REPOSITORIES | INTEGRATIONS | MATERIALIZED VIEWS | NETWORK POLICIES | NETWORK RULES | PACKAGES POLICIES | PIPES | PROCEDURES | MASKING POLICIES | PASSWORD POLICIES | PROJECTION POLICIES | REPLICATION GROUPS | ROLES | ROW ACCESS POLICIES | SCHEMAS | SESSION POLICIES | SECRETS | SEQUENCES | STAGES | STREAMS | TABLES | TAGS | TASKS | USERS | VIEWS | WAREHOUSES. For more information head over to [Snowflake documentation](https://docs.snowflake.com/en/sql-reference/sql/grant-ownership#required-parameters).

Optional:

Expand All @@ -284,7 +284,7 @@ Optional:

Required:

- `object_type_plural` (String) Specifies the type of object in plural form on which you are transferring ownership. Available values are: AGGREGATION POLICIES | ALERTS | AUTHENTICATION POLICIES | COMPUTE POOLS | CORTEX SEARCH SERVICES | DATA METRIC FUNCTIONS | DATABASES | DATABASE ROLES | DYNAMIC TABLES | EVENT TABLES | EXTERNAL TABLES | EXTERNAL VOLUMES | FAILOVER GROUPS | FILE FORMATS | FUNCTIONS | GIT REPOSITORIES | HYBRID TABLES | ICEBERG TABLES | IMAGE REPOSITORIES | INTEGRATIONS | MATERIALIZED VIEWS | NETWORK POLICIES | NETWORK RULES | PACKAGES POLICIES | PIPES | PROCEDURES | MASKING POLICIES | PASSWORD POLICIES | PROJECTION POLICIES | REPLICATION GROUPS | ROLES | ROW ACCESS POLICIES | SCHEMAS | SESSION POLICIES | SECRETS | SEQUENCES | STAGES | STREAMS | TABLES | TAGS | TASKS | USERS | VIEWS | WAREHOUSES. For more information head over to [Snowflake documentation](https://docs.snowflake.com/en/sql-reference/sql/grant-ownership#required-parameters).
- `object_type_plural` (String) Specifies the type of object in plural form on which you are transferring ownership. Available values are: AGGREGATION POLICIES | ALERTS | AUTHENTICATION POLICIES | COMPUTE POOLS | DATA METRIC FUNCTIONS | DATABASES | DATABASE ROLES | DYNAMIC TABLES | EVENT TABLES | EXTERNAL TABLES | EXTERNAL VOLUMES | FAILOVER GROUPS | FILE FORMATS | FUNCTIONS | GIT REPOSITORIES | HYBRID TABLES | ICEBERG TABLES | IMAGE REPOSITORIES | INTEGRATIONS | MATERIALIZED VIEWS | NETWORK POLICIES | NETWORK RULES | PACKAGES POLICIES | PIPES | PROCEDURES | MASKING POLICIES | PASSWORD POLICIES | PROJECTION POLICIES | REPLICATION GROUPS | ROLES | ROW ACCESS POLICIES | SCHEMAS | SESSION POLICIES | SECRETS | SEQUENCES | STAGES | STREAMS | TABLES | TAGS | TASKS | USERS | VIEWS | WAREHOUSES. For more information head over to [Snowflake documentation](https://docs.snowflake.com/en/sql-reference/sql/grant-ownership#required-parameters).

Optional:

Expand Down
2 changes: 1 addition & 1 deletion docs/resources/grant_privileges_to_account_role.md
Original file line number Diff line number Diff line change
Expand Up @@ -320,7 +320,7 @@ Optional:

Required:

- `object_type_plural` (String) The plural object type of the schema object on which privileges will be granted. Valid values are: ALERTS | AUTHENTICATION POLICIES | CORTEX SEARCH SERVICES | DATA METRIC FUNCTIONS | DYNAMIC TABLES | EVENT TABLES | EXTERNAL TABLES | FILE FORMATS | FUNCTIONS | GIT REPOSITORIES | HYBRID TABLES | ICEBERG TABLES | MATERIALIZED VIEWS | MODELS | NETWORK RULES | PASSWORD POLICIES | PIPES | PROCEDURES | SECRETS | SERVICES | SEQUENCES | STAGES | STREAMS | TABLES | TASKS | VIEWS.
- `object_type_plural` (String) The plural object type of the schema object on which privileges will be granted. Valid values are: ALERTS | AUTHENTICATION POLICIES | DATA METRIC FUNCTIONS | DYNAMIC TABLES | EVENT TABLES | EXTERNAL TABLES | FILE FORMATS | FUNCTIONS | GIT REPOSITORIES | HYBRID TABLES | ICEBERG TABLES | MATERIALIZED VIEWS | MODELS | NETWORK RULES | PASSWORD POLICIES | PIPES | PROCEDURES | SECRETS | SERVICES | SEQUENCES | STAGES | STREAMS | TABLES | TASKS | VIEWS.

Optional:

Expand Down
2 changes: 1 addition & 1 deletion docs/resources/grant_privileges_to_database_role.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ Optional:

Required:

- `object_type_plural` (String) The plural object type of the schema object on which privileges will be granted. Valid values are: ALERTS | AUTHENTICATION POLICIES | CORTEX SEARCH SERVICES | DATA METRIC FUNCTIONS | DYNAMIC TABLES | EVENT TABLES | EXTERNAL TABLES | FILE FORMATS | FUNCTIONS | GIT REPOSITORIES | HYBRID TABLES | ICEBERG TABLES | MATERIALIZED VIEWS | MODELS | NETWORK RULES | PASSWORD POLICIES | PIPES | PROCEDURES | SECRETS | SERVICES | SEQUENCES | STAGES | STREAMS | TABLES | TASKS | VIEWS.
- `object_type_plural` (String) The plural object type of the schema object on which privileges will be granted. Valid values are: ALERTS | AUTHENTICATION POLICIES | DATA METRIC FUNCTIONS | DYNAMIC TABLES | EVENT TABLES | EXTERNAL TABLES | FILE FORMATS | FUNCTIONS | GIT REPOSITORIES | HYBRID TABLES | ICEBERG TABLES | MATERIALIZED VIEWS | MODELS | NETWORK RULES | PASSWORD POLICIES | PIPES | PROCEDURES | SECRETS | SERVICES | SEQUENCES | STAGES | STREAMS | TABLES | TASKS | VIEWS.

Optional:

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Simple usage
data "snowflake_cortex_search_services" "simple" {
}

output "simple_output" {
value = data.snowflake_cortex_search_services.simple.cortex_search_services
}

# Filtering (like)
data "snowflake_cortex_search_services" "like" {
like = "some-name"
}

output "like_output" {
value = data.snowflake_cortex_search_services.like.cortex_search_services
}

# Filtering (starts_with)
data "snowflake_cortex_search_services" "starts_with" {
starts_with = "prefix-"
}

output "starts_with_output" {
value = data.snowflake_cortex_search_services.starts_with.cortex_search_services
}

# Filtering (limit)
data "snowflake_cortex_search_services" "limit" {
limit {
rows = 10
from = "prefix-"
}
}

output "limit_output" {
value = data.snowflake_cortex_search_services.limit.cortex_search_services
}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
terraform import snowflake_cortex_search_service.example 'dbName|schemaName|fileFormatName'
38 changes: 38 additions & 0 deletions examples/resources/snowflake_cortex_search_service/resource.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
## Basic
resource "snowflake_database" "test" {
name = "some_database"
}

resource "snowflake_schema" "test" {
database = snowflake_database.test.name
name = "some_schema"
}

resource "snowflake_table" "test" {
database = snowflake_database.test.name
schema = snowflake_schema.test.name
name = "some_table"
change_tracking = true
column {
name = "ID"
type = "NUMBER(38,0)"
}

column {
name = "SOME_TEXT"
type = "VARCHAR"
}
}

resource "snowflake_cortex_search_service" "test" {
depends_on = [snowflake_table.test]

database = snowflake_database.test.name
schema = snowflake_schema.test.name
name = "some_name"
on = "SOME_TEXT"
target_lag = "2 minutes"
warehouse = "some_warehouse"
query = "SELECT SOME_TEXT FROM \"some_database\".\"some_schema\".\"some_table\""
comment = "some comment"
}
9 changes: 4 additions & 5 deletions pkg/acceptance/helpers/cortex_search_service_client.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,18 +28,17 @@ func (c *CortexSearchServiceClient) client() sdk.CortexSearchServices {

func (c *CortexSearchServiceClient) CreateCortexSearchService(t *testing.T, tableId sdk.SchemaObjectIdentifier) (*sdk.CortexSearchService, func()) {
t.Helper()
return c.CreateCortexSearchServiceWithOptions(t, c.ids.RandomSchemaObjectIdentifier(), c.ids.WarehouseId(), tableId)
return c.CreateCortexSearchServiceWithOptions(t, c.ids.RandomSchemaObjectIdentifier(), c.ids.WarehouseId(), tableId, "some_text_column")
}

func (c *CortexSearchServiceClient) CreateCortexSearchServiceWithOptions(t *testing.T, id sdk.SchemaObjectIdentifier, warehouseId sdk.AccountObjectIdentifier, tableId sdk.SchemaObjectIdentifier) (*sdk.CortexSearchService, func()) {
func (c *CortexSearchServiceClient) CreateCortexSearchServiceWithOptions(t *testing.T, id sdk.SchemaObjectIdentifier, warehouseId sdk.AccountObjectIdentifier, tableId sdk.SchemaObjectIdentifier, column string) (*sdk.CortexSearchService, func()) {
t.Helper()
on := "ID"
targetLag := "2 minutes"
query := fmt.Sprintf(`select "ID" from %s`, tableId.FullyQualifiedName())
query := fmt.Sprintf(`select %s from %s`, column, tableId.FullyQualifiedName())
comment := random.Comment()
ctx := context.Background()

err := c.client().Create(ctx, sdk.NewCreateCortexSearchServiceRequest(id, on, warehouseId, targetLag, query).WithComment(comment))
err := c.client().Create(ctx, sdk.NewCreateCortexSearchServiceRequest(id, column, warehouseId, targetLag, query).WithComment(comment))
require.NoError(t, err)

contextSearchService, err := c.client().ShowByID(ctx, id)
Expand Down
8 changes: 8 additions & 0 deletions pkg/acceptance/helpers/schema_client.go
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,14 @@ func (c *SchemaClient) DropSchemaFunc(t *testing.T, id sdk.DatabaseObjectIdentif
}
}

func (c *SchemaClient) UseDefaultSchema(t *testing.T) {
t.Helper()
ctx := context.Background()

err := c.context.client.Sessions.UseSchema(ctx, c.ids.SchemaId())
require.NoError(t, err)
}

func (c *SchemaClient) UpdateDataRetentionTime(t *testing.T, id sdk.DatabaseObjectIdentifier, days int) func() {
t.Helper()
ctx := context.Background()
Expand Down
12 changes: 12 additions & 0 deletions pkg/acceptance/helpers/table_client.go
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,18 @@ func (c *TableClient) CreateTableWithColumns(t *testing.T, columns []sdk.TableCo
return c.CreateTableWithIdAndColumns(t, c.ids.RandomSchemaObjectIdentifier(), columns)
}

func (c *TableClient) CreateTableWithPredefinedColumns(t *testing.T) (*sdk.Table, func()) {
t.Helper()

columns := []sdk.TableColumnRequest{
*sdk.NewTableColumnRequest("id", "NUMBER"),
*sdk.NewTableColumnRequest("some_text_column", "VARCHAR"),
*sdk.NewTableColumnRequest("some_other_text_column", "VARCHAR"),
}

return c.CreateTableWithIdAndColumns(t, c.ids.RandomSchemaObjectIdentifier(), columns)
}

func (c *TableClient) CreateTableWithIdAndColumns(t *testing.T, id sdk.SchemaObjectIdentifier, columns []sdk.TableColumnRequest) (*sdk.Table, func()) {
t.Helper()
ctx := context.Background()
Expand Down
21 changes: 8 additions & 13 deletions pkg/datasources/cortex_search_services.go
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ var cortexSearchServicesSchema = map[string]*schema.Schema{
},
},
},
"cortexSearchServices": {
"cortex_search_services": {
Type: schema.TypeList,
Computed: true,
Description: "Holds the output of SHOW CORTEX SEARCH SERVICES.",
Expand Down Expand Up @@ -117,10 +117,11 @@ func CortexSearchServices() *schema.Resource {
func ReadCortexSearchServices(d *schema.ResourceData, meta interface{}) error {
client := meta.(*provider.Context).Client
request := sdk.NewShowCortexSearchServiceRequest()
if v, ok := d.GetOk("like"); ok {
like := v.([]interface{})[0].(map[string]interface{})
pattern := like["pattern"].(string)
request.WithLike(sdk.Like{Pattern: sdk.String(pattern)})

if likePattern, ok := d.GetOk("like"); ok {
request.WithLike(sdk.Like{
Pattern: sdk.String(likePattern.(string)),
})
}

if v, ok := d.GetOk("in"); ok {
Expand Down Expand Up @@ -172,20 +173,14 @@ func ReadCortexSearchServices(d *schema.ResourceData, meta interface{}) error {
records := make([]map[string]any, 0, len(dts))
for _, dt := range dts {
record := map[string]any{}
/*
guides on time formatting
https://docs.snowflake.com/en/user-guide/date-time-input-output
https://pkg.go.dev/time
note: format may depend on what the account parameter for TIMESTAMP_OUTPUT_FORMAT is set to. Perhaps we should return this as a string rather than a time.Time?
*/
record["created_on"] = dt.CreatedOn.Format("2006-01-02T16:04:05.000 -0700")
record["created_on"] = dt.CreatedOn.String()
record["name"] = dt.Name
record["database_name"] = dt.DatabaseName
record["schema_name"] = dt.SchemaName
record["comment"] = dt.Comment
records = append(records, record)
}
if err := d.Set("records", records); err != nil {
if err := d.Set("cortex_search_services", records); err != nil {
return err
}
return nil
Expand Down
Loading

0 comments on commit 763d06c

Please sign in to comment.