Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Secrets datasource #3131

Merged
merged 93 commits into from
Oct 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
58d5794
initial secret_def file for sdk
sfc-gh-fbudzynski Sep 12, 2024
f1488ca
naming changes to secret create options
sfc-gh-fbudzynski Sep 12, 2024
752f924
create secrets and alter with two basic tests
sfc-gh-fbudzynski Sep 13, 2024
d7ae12f
unit tests for alter
sfc-gh-fbudzynski Sep 13, 2024
57ca19a
add tests for all four create types
sfc-gh-fbudzynski Sep 13, 2024
8d55a63
rename previous secret to secretReference in common_types.go due to n…
sfc-gh-fbudzynski Sep 16, 2024
9894799
secrets gen with drop, show and desc
sfc-gh-fbudzynski Sep 16, 2024
e4a0827
all unit tests
sfc-gh-fbudzynski Sep 16, 2024
3bb8c85
init for integration tests
sfc-gh-fbudzynski Sep 16, 2024
4130618
added secret to client.go and created secret_client
sfc-gh-fbudzynski Sep 17, 2024
fc85c66
added like and in to showByID for secret
sfc-gh-fbudzynski Sep 17, 2024
aa64511
init for integration tests
sfc-gh-fbudzynski Sep 17, 2024
ab8bee1
secret_def changes
sfc-gh-fbudzynski Sep 17, 2024
3541c70
tests for two methods of creating secret
sfc-gh-fbudzynski Sep 17, 2024
180a7ce
changed refresh token expiry time from string to time.Time
sfc-gh-fbudzynski Sep 18, 2024
7a9ad92
added remaining integration tests
sfc-gh-fbudzynski Sep 19, 2024
0e9cf94
linter
sfc-gh-fbudzynski Sep 19, 2024
deadc65
linting
sfc-gh-fbudzynski Sep 19, 2024
235fbe5
removing comment
sfc-gh-fbudzynski Sep 19, 2024
73eeb63
mid review commit before generating assertions
sfc-gh-fbudzynski Sep 20, 2024
bd0b64f
rebase conflict resolved
sfc-gh-fbudzynski Sep 23, 2024
2050b85
tests pass after changing to []string for oauthScopes
sfc-gh-fbudzynski Sep 23, 2024
59c7b14
readded unit tests
sfc-gh-fbudzynski Sep 23, 2024
df7a8ff
added CreateApiAuthenticationWithRequest to security integration client
sfc-gh-fbudzynski Sep 24, 2024
eed310a
changes to secret after review, all tests pass
sfc-gh-fbudzynski Sep 24, 2024
af2e030
added assert for details
sfc-gh-fbudzynski Sep 24, 2024
4ccc4e7
linter hints applied
sfc-gh-fbudzynski Sep 24, 2024
34983af
updated generator README with validation issue for ConflictingFields …
sfc-gh-fbudzynski Sep 24, 2024
bdf2585
updated after re-review
sfc-gh-fbudzynski Sep 24, 2024
7791f33
test changed to use errMoreThanOneOf
sfc-gh-fbudzynski Sep 25, 2024
1f1fb2b
init commit for secret resource
sfc-gh-fbudzynski Sep 26, 2024
366313f
adding secret common to avoid duplication
sfc-gh-fbudzynski Sep 26, 2024
a383ac4
acc tests pass with usage of secret_common
sfc-gh-fbudzynski Sep 26, 2024
40179ec
Secret with authorization code acceptance test, diff suppress
sfc-gh-fbudzynski Sep 27, 2024
04d3feb
helper function for snowflake datetime for secret
sfc-gh-fbudzynski Sep 27, 2024
fa408fd
check destroy and few cleanups
sfc-gh-fbudzynski Sep 27, 2024
d3e54db
Wrapped OAuth Scopes in separate queryStructField to allow empty list…
sfc-gh-fbudzynski Sep 30, 2024
0e18136
Linter adjustments
sfc-gh-fbudzynski Sep 30, 2024
f60985c
init commit for secret resource
sfc-gh-fbudzynski Sep 26, 2024
8348809
adding secret common to avoid duplication
sfc-gh-fbudzynski Sep 26, 2024
4753096
acc tests pass with usage of secret_common
sfc-gh-fbudzynski Sep 26, 2024
46e2181
Secret with authorization code acceptance test, diff suppress
sfc-gh-fbudzynski Sep 27, 2024
c3f3ea7
helper function for snowflake datetime for secret
sfc-gh-fbudzynski Sep 27, 2024
91269d6
check destroy and few cleanups
sfc-gh-fbudzynski Sep 27, 2024
b91b97c
adding secret with basic authentication flow
sfc-gh-fbudzynski Sep 30, 2024
8de6e84
checking secret with optional oauth_scopes
sfc-gh-fbudzynski Sep 30, 2024
c64a1ed
secret basic authentication flow acceptance tests and fix for parsing…
sfc-gh-fbudzynski Oct 1, 2024
ccdf9f8
Merge remote-tracking branch 'origin/secret-resource' into secret-res…
sfc-gh-fbudzynski Oct 1, 2024
fde6908
removed unwanted duplicats after merge
sfc-gh-fbudzynski Oct 1, 2024
a147b99
All secret flows with acceptance tests and change to handle different…
sfc-gh-fbudzynski Oct 2, 2024
356d98c
linter pre-push changes
sfc-gh-fbudzynski Oct 2, 2024
83ccafc
generated documentation for each secret flow
sfc-gh-fbudzynski Oct 2, 2024
07dc9e2
Merge branch 'main' into secret-resource
sfc-gh-fbudzynski Oct 2, 2024
de99b68
delete unused _ext assert
sfc-gh-fbudzynski Oct 3, 2024
5c6d9ee
self-review cleanup
sfc-gh-fbudzynski Oct 3, 2024
805c61e
docs regenerate
sfc-gh-fbudzynski Oct 3, 2024
044c484
example resource naming fix for secret with generic_string
sfc-gh-fbudzynski Oct 3, 2024
bc43a72
docs fixes
sfc-gh-fbudzynski Oct 3, 2024
4ae36a4
make docs-check error fix
sfc-gh-fbudzynski Oct 3, 2024
4ac851a
Merge branch 'main' into secret-resource
sfc-gh-fbudzynski Oct 4, 2024
4ee377b
initial commit for secrets datasource
sfc-gh-fbudzynski Oct 4, 2024
ac37669
Merge branch 'main' into secret-resource
sfc-gh-fbudzynski Oct 7, 2024
f3cd4b1
Jakub comments resolved
sfc-gh-fbudzynski Oct 8, 2024
e098f69
comments resolved
sfc-gh-fbudzynski Oct 9, 2024
39f3cdd
Merge branch 'main' into secret-resource
sfc-gh-fbudzynski Oct 9, 2024
d5b6db0
added tests to check for external changes to refresh_token_expiry_tim…
sfc-gh-fbudzynski Oct 9, 2024
54a5451
Merge branch 'main' into secrets-datasource
sfc-gh-fbudzynski Oct 9, 2024
e304793
resolved to Arturs comments
sfc-gh-fbudzynski Oct 10, 2024
c1115fa
linter suggestions
sfc-gh-fbudzynski Oct 10, 2024
9dea35c
Merge branch 'main' into secret-resource
sfc-gh-fbudzynski Oct 10, 2024
5638d83
Merge branch 'main' into secrets-datasource
sfc-gh-fbudzynski Oct 10, 2024
92c8d25
linter suggestions
sfc-gh-fbudzynski Oct 10, 2024
bdb7fff
refactor after Kuba re-review comments
sfc-gh-fbudzynski Oct 11, 2024
bd2e172
Merge remote-tracking branch 'origin/secret-resource' into secrets-da…
sfc-gh-fbudzynski Oct 11, 2024
81a8312
linter and pre-push corrections
sfc-gh-fbudzynski Oct 11, 2024
dba8478
changes not to abuse pointer on models
sfc-gh-fbudzynski Oct 14, 2024
40ecd49
linter adjustments
sfc-gh-fbudzynski Oct 14, 2024
1a9a6db
changed username to sensitive, adjusted docs and migration guide to p…
sfc-gh-fbudzynski Oct 14, 2024
7573935
Merge branch 'secret-resource' of github.com:Snowflake-Labs/terraform…
sfc-gh-fbudzynski Oct 14, 2024
57bd2ce
datasource changes after review
sfc-gh-fbudzynski Oct 15, 2024
18f0307
fill more example usage
sfc-gh-fbudzynski Oct 15, 2024
b202fdd
Merge branch 'main' into secret-resource
sfc-gh-fbudzynski Oct 15, 2024
11531cf
re-generated docs
sfc-gh-fbudzynski Oct 15, 2024
9012bb9
changes to secret datasource docs
sfc-gh-fbudzynski Oct 15, 2024
7724d57
linter hint
sfc-gh-fbudzynski Oct 15, 2024
734ec99
fixed naming in tests for usage of resource show_output asserts
sfc-gh-fbudzynski Oct 15, 2024
5e8afaf
removed tests for IN application and resolved comments for review
sfc-gh-fbudzynski Oct 16, 2024
cc04648
Linter checks
sfc-gh-fbudzynski Oct 16, 2024
3ac5fbd
Merge branch 'secret-resource' into secrets-datasource
sfc-gh-fbudzynski Oct 16, 2024
da6223f
Merge branch 'main' into secrets-datasource
sfc-gh-fbudzynski Oct 17, 2024
8276a20
changed schema fully qualified name to proper value in examples and u…
sfc-gh-fbudzynski Oct 17, 2024
b0aed22
Merge branch 'secrets-datasource' of github.com:Snowflake-Labs/terraf…
sfc-gh-fbudzynski Oct 17, 2024
429071a
changed schema fully qualified name to proper value after main merge
sfc-gh-fbudzynski Oct 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 131 additions & 0 deletions docs/data-sources/secrets.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
---
page_title: "snowflake_secrets Data Source - terraform-provider-snowflake"
subcategory: ""
description: |-
Datasource used to get details of filtered secrets. Filtering is aligned with the current possibilities for SHOW SECRETS https://docs.snowflake.com/en/sql-reference/sql/show-secrets query. The results of SHOW and DESCRIBE are encapsulated in one output collection secrets.
---

!> **V1 release candidate** This data source is a release candidate for the V1. We do not expect significant changes in it before the V1. We will welcome any feedback and adjust the data source if needed. Any errors reported will be resolved with a higher priority. We encourage checking this data source out before the V1 release. Please follow the [migration guide](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/MIGRATION_GUIDE.md#v0970--v0980) to use it.

# snowflake_secrets (Data Source)

Datasource used to get details of filtered secrets. Filtering is aligned with the current possibilities for [SHOW SECRETS](https://docs.snowflake.com/en/sql-reference/sql/show-secrets) query. The results of SHOW and DESCRIBE are encapsulated in one output collection `secrets`.
sfc-gh-jcieslak marked this conversation as resolved.
Show resolved Hide resolved

## Example Usage

```terraform
# Simple usage
data "snowflake_secrets" "simple" {
}

output "simple_output" {
value = data.snowflake_secrets.simple.secrets
}

# Filtering (like)
data "snowflake_secrets" "like" {
like = "secret-name"
}

output "like_output" {
value = data.snowflake_secrets.like.secrets
}

# Filtering by prefix (like)
data "snowflake_secrets" "like_prefix" {
like = "prefix%"
}

output "like_prefix_output" {
value = data.snowflake_secrets.like_prefix.secrets
}

# Filtering (in)
data "snowflake_secrets" "in" {
in {
schema = snowflake_schema.test.fully_qualified_name
}
}

output "in_output" {
value = data.snowflake_secrets.in.secrets
}

# Without additional data (to limit the number of calls make for every found secret)
data "snowflake_secrets" "only_show" {
# with_describe is turned on by default and it calls DESCRIBE SECRET for every secret found and attaches its output to secrets.*.describe_output field
with_describe = false
}

output "only_show_output" {
value = data.snowflake_secrets.only_show.secrets
}
```

<!-- schema generated by tfplugindocs -->
## Schema

### Optional

- `in` (Block List, Max: 1) IN clause to filter the list of secrets (see [below for nested schema](#nestedblock--in))
- `like` (String) Filters the output with **case-insensitive** pattern, with support for SQL wildcard characters (`%` and `_`).
- `with_describe` (Boolean) Runs DESC SECRET for each secret returned by SHOW SECRETS. The output of describe is saved to the description field. By default this value is set to true.

### Read-Only

- `id` (String) The ID of this resource.
- `secrets` (List of Object) Holds the aggregated output of all secrets details queries. (see [below for nested schema](#nestedatt--secrets))

<a id="nestedblock--in"></a>
### Nested Schema for `in`

Optional:

- `account` (Boolean) Returns records for the entire account.
- `application` (String) Returns records for the specified application.
- `application_package` (String) Returns records for the specified application package.
- `database` (String) Returns records for the current database in use or for a specified database.
- `schema` (String) Returns records for the current schema in use or a specified schema. Use fully qualified name.


<a id="nestedatt--secrets"></a>
### Nested Schema for `secrets`

Read-Only:

- `describe_output` (List of Object) (see [below for nested schema](#nestedobjatt--secrets--describe_output))
- `show_output` (List of Object) (see [below for nested schema](#nestedobjatt--secrets--show_output))

<a id="nestedobjatt--secrets--describe_output"></a>
### Nested Schema for `secrets.describe_output`

Read-Only:

- `comment` (String)
- `created_on` (String)
- `database_name` (String)
- `integration_name` (String)
- `name` (String)
- `oauth_access_token_expiry_time` (String)
- `oauth_refresh_token_expiry_time` (String)
- `oauth_scopes` (Set of String)
- `owner` (String)
- `schema_name` (String)
- `secret_type` (String)
- `username` (String)


<a id="nestedobjatt--secrets--show_output"></a>
### Nested Schema for `secrets.show_output`

Read-Only:

- `comment` (String)
- `created_on` (String)
- `database_name` (String)
- `name` (String)
- `oauth_scopes` (Set of String)
- `owner` (String)
- `owner_role_type` (String)
- `schema_name` (String)
- `secret_type` (String)
46 changes: 46 additions & 0 deletions examples/data-sources/snowflake_secrets/data-source.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Simple usage
data "snowflake_secrets" "simple" {
}

output "simple_output" {
value = data.snowflake_secrets.simple.secrets
}

# Filtering (like)
data "snowflake_secrets" "like" {
like = "secret-name"
}

output "like_output" {
value = data.snowflake_secrets.like.secrets
}

# Filtering by prefix (like)
data "snowflake_secrets" "like_prefix" {
like = "prefix%"
}

output "like_prefix_output" {
value = data.snowflake_secrets.like_prefix.secrets
}

# Filtering (in)
data "snowflake_secrets" "in" {
in {
schema = snowflake_schema.test.fully_qualified_name
}
}

output "in_output" {
value = data.snowflake_secrets.in.secrets
}

# Without additional data (to limit the number of calls make for every found secret)
data "snowflake_secrets" "only_show" {
# with_describe is turned on by default and it calls DESCRIBE SECRET for every secret found and attaches its output to secrets.*.describe_output field
with_describe = false
}

output "only_show_output" {
value = data.snowflake_secrets.only_show.secrets
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
package resourceshowoutputassert

import (
"testing"

"github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/acceptance/bettertestspoc/assert"
)

// SecretsDatasourceShowOutput is a temporary workaround to have better show output assertions in data source acceptance tests.
func SecretsDatasourceShowOutput(t *testing.T, name string) *SecretShowOutputAssert {
t.Helper()

s := SecretShowOutputAssert{
ResourceAssert: assert.NewDatasourceAssert("data."+name, "show_output", "secrets.0."),
}
s.AddAssertion(assert.ValueSet("show_output.#", "1"))
return &s
}
13 changes: 13 additions & 0 deletions pkg/acceptance/check_destroy.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,19 @@ import (
"github.com/hashicorp/terraform-plugin-testing/terraform"
)

func ComposeCheckDestroy(t *testing.T, resources ...resources.Resource) func(*terraform.State) error {
t.Helper()

return func(s *terraform.State) error {
errs := make([]error, 0)
for _, resource := range resources {
checkFunc := CheckDestroy(t, resource)
errs = append(errs, checkFunc(s))
}
return errors.Join(errs...)
}
}

func CheckDestroy(t *testing.T, resource resources.Resource) func(*terraform.State) error {
t.Helper()
// TODO [SNOW-1653619]: use TestClient() here
Expand Down
142 changes: 142 additions & 0 deletions pkg/datasources/secrets.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
package datasources

import (
"context"

"github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/internal/provider"
"github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/resources"
"github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/schemas"
"github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk"
"github.com/hashicorp/terraform-plugin-sdk/v2/diag"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
)

var secretsSchema = map[string]*schema.Schema{
"with_describe": {
Type: schema.TypeBool,
Optional: true,
Default: true,
Description: "Runs DESC SECRET for each secret returned by SHOW SECRETS. The output of describe is saved to the description field. By default this value is set to true.",
},
"in": {
Type: schema.TypeList,
Optional: true,
Description: "IN clause to filter the list of secrets",
MaxItems: 1,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"account": {
Type: schema.TypeBool,
Optional: true,
Description: "Returns records for the entire account.",
ExactlyOneOf: []string{"in.0.account", "in.0.database", "in.0.schema", "in.0.application", "in.0.application_package"},
},
"database": {
Type: schema.TypeString,
Optional: true,
Description: "Returns records for the current database in use or for a specified database.",
ExactlyOneOf: []string{"in.0.account", "in.0.database", "in.0.schema", "in.0.application", "in.0.application_package"},
ValidateDiagFunc: resources.IsValidIdentifier[sdk.AccountObjectIdentifier](),
},
"schema": {
Type: schema.TypeString,
Optional: true,
Description: "Returns records for the current schema in use or a specified schema. Use fully qualified name.",
ExactlyOneOf: []string{"in.0.account", "in.0.database", "in.0.schema", "in.0.application", "in.0.application_package"},
ValidateDiagFunc: resources.IsValidIdentifier[sdk.DatabaseObjectIdentifier](),
},
"application": {
Type: schema.TypeString,
Optional: true,
Description: "Returns records for the specified application.",
ExactlyOneOf: []string{"in.0.account", "in.0.database", "in.0.schema", "in.0.application", "in.0.application_package"},
ValidateDiagFunc: resources.IsValidIdentifier[sdk.AccountObjectIdentifier](),
},
"application_package": {
Type: schema.TypeString,
Optional: true,
Description: "Returns records for the specified application package.",
ExactlyOneOf: []string{"in.0.account", "in.0.database", "in.0.schema", "in.0.application", "in.0.application_package"},
ValidateDiagFunc: resources.IsValidIdentifier[sdk.AccountObjectIdentifier](),
},
},
},
},
"like": {
Type: schema.TypeString,
Optional: true,
Description: "Filters the output with **case-insensitive** pattern, with support for SQL wildcard characters (`%` and `_`).",
},
"secrets": {
Type: schema.TypeList,
Computed: true,
Description: "Holds the aggregated output of all secrets details queries.",
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
resources.ShowOutputAttributeName: {
Type: schema.TypeList,
Computed: true,
Description: "Holds the output of SHOW SECRETS.",
Elem: &schema.Resource{
Schema: schemas.ShowSecretSchema,
},
},
resources.DescribeOutputAttributeName: {
Type: schema.TypeList,
Computed: true,
Description: "Holds the output of DESCRIBE SECRET.",
Elem: &schema.Resource{
Schema: schemas.DescribeSecretSchema,
},
},
},
},
},
}

func Secrets() *schema.Resource {
return &schema.Resource{
ReadContext: ReadSecrets,
Schema: secretsSchema,
Description: "Datasource used to get details of filtered secrets. Filtering is aligned with the current possibilities for [SHOW SECRETS](https://docs.snowflake.com/en/sql-reference/sql/show-secrets) query. The results of SHOW and DESCRIBE are encapsulated in one output collection `secrets`.",
}
}

func ReadSecrets(ctx context.Context, d *schema.ResourceData, meta any) diag.Diagnostics {
client := meta.(*provider.Context).Client
req := sdk.NewShowSecretRequest()

handleLike(d, &req.Like)
err := handleExtendedIn(d, &req.In)
if err != nil {
return diag.FromErr(err)
}

secrets, err := client.Secrets.Show(ctx, req)
if err != nil {
return diag.FromErr(err)
}
d.SetId("secrets_read")

flattenedSecrets := make([]map[string]any, len(secrets))
for i, secret := range secrets {
secret := secret
var secretDescriptions []map[string]any
if d.Get("with_describe").(bool) {
describeOutput, err := client.Secrets.Describe(ctx, secret.ID())
if err != nil {
return diag.FromErr(err)
}
secretDescriptions = []map[string]any{schemas.SecretDescriptionToSchema(*describeOutput)}
}

flattenedSecrets[i] = map[string]any{
resources.ShowOutputAttributeName: []map[string]any{schemas.SecretToSchema(&secret)},
resources.DescribeOutputAttributeName: secretDescriptions,
}
}
if err := d.Set("secrets", flattenedSecrets); err != nil {
return diag.FromErr(err)
}
return nil
}
Loading
Loading