feat: Masking policy resource v1 (#3078)

<!-- Feel free to delete comments as you fill this in -->
- SDK
  - improve validations
  - add tests
  - improve options parsing (also, get rid of a 3rd party library)
- add a TODO to reuse parsing from row access policies after merging
#3079
- resource
  - use new id handling
  - rename/remove some fields to be consistent with other resources
  - better ACC tests
-  misc
  - adjust helper clients
  - generate config and assertion builders
  - adjust docs
<!-- summary of changes -->

## Test Plan
<!-- detail ways in which this PR has been tested or needs to be tested
-->
* [x] acceptance tests
* [x] integration tests
* [x] unit tests 

## References
<!-- issues documentation links, etc  -->
https://docs.snowflake.com/en/sql-reference/sql/create-masking-policy

## TODO
- add tests for issues
- rework data source

MIGRATION_GUIDE.md

@@ -6,6 +6,58 @@ across different versions.
 
 ## v0.95.0 ➞ v0.96.0
 
+### snowflake_masking_policy resource changes
+New fields:
+  - `show_output` field that holds the response from SHOW MASKING POLICIES.
+  - `describe_output` field that holds the response from DESCRIBE MASKING POLICY.
+
+#### *(breaking change)* Renamed fields in snowflake_masking_policy resource
+Renamed fields:
+  - `masking_expression` to `body`
+Please rename these fields in your configuration files. State will be migrated automatically.
+
+#### *(breaking change)* Removed fields from snowflake_masking_policy resource
+Removed fields:
+- `or_replace`
+- `if_not_exists`
+The value of these field will be removed from the state automatically.
+
+#### *(breaking change)* Adjusted schema of arguments/signature
+The field `signature` is renamed to `arguments` to be consistent with other resources.
+Now, arguments are stored without nested `column` field. Please adjust that in your configs, like in the example below. State is migrated automatically.
+
+The old configuration looks like this:
+```
+  signature {
+    column {
+      name = "val"
+      type = "VARCHAR"
+    }
+  }
+```
+
+The new configuration looks like this:
+```
+  argument {
+    name = "val"
+    type = "VARCHAR"
+  }
+```
+
+#### *(breaking change)* Identifiers related changes
+During [identifiers rework](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/ROADMAP.md#identifiers-rework) we decided to
+migrate resource ids from pipe-separated to regular Snowflake identifiers (e.g. `<database_name>|<schema_name>` -> `"<database_name>"."<schema_name>"`). Importing resources also needs to be adjusted (see [example](https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest/docs/resources/row_access_policy#import)).
+
+Also, we added diff suppress function that prevents Terraform from showing differences, when only quoting is different.
+
+No change is required, the state will be migrated automatically.
+
+#### *(behavior change)* Boolean type changes
+To easily handle three-value logic (true, false, unknown) in provider's configs, type of `exempt_other_policies` was changed from boolean to string.
+
+For more details about default values, please refer to the [changes before v1](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/v1-preparations/CHANGES_BEFORE_V1.md#default-values) document.
+
+
 ### *(breaking change)* resource_monitor resource
 Removed fields:
 - `set_for_account` (will be settable on account resource, right now, the preferred way is to set it through unsafe_execute resource)
@@ -55,12 +107,33 @@ New fields:
 #### *(breaking change)* Renamed fields in snowflake_row_access_policy resource
 Renamed fields:
   - `row_access_expression` to `body`
-  - `signature` to `arguments`
 Please rename these fields in your configuration files. State will be migrated automatically.
 
-#### *(breaking change)* Adjusted behavior of arguments/signature
+#### *(breaking change)* Adjusted schema of arguments/signature
+The field `signature` is renamed to `arguments` to be consistent with other resources.
 Now, arguments are stored as a list, instead of a map. Please adjust that in your configs. State is migrated automatically. Also, this means that order of the items matters and may be adjusted.
 
+
+The old configuration looks like this:
+```
+  signature = {
+    A = "VARCHAR",
+    B = "VARCHAR"
+  }
+```
+
+The new configuration looks like this:
+```
+  argument {
+    name = "A"
+    type = "VARCHAR"
+  }
+  argument {
+    name = "B"
+    type = "VARCHAR"
+  }
+```
+
 Argument names are now case sensitive. All policies created previously in the provider have upper case argument names. If you used lower case before, please adjust your configs. Values in the state will be migrated to uppercase automatically.
 
 #### *(breaking change)* Adjusted behavior on changing name

docs/resources/masking_policy.md

@@ -2,39 +2,77 @@
 page_title: "snowflake_masking_policy Resource - terraform-provider-snowflake"
 subcategory: ""
 description: |-
-  
+  Resource used to manage masking policies. For more information, check masking policies documentation https://docs.snowflake.com/en/sql-reference/sql/create-masking-policy.
 ---
 
 # snowflake_masking_policy (Resource)
 
-
+Resource used to manage masking policies. For more information, check [masking policies documentation](https://docs.snowflake.com/en/sql-reference/sql/create-masking-policy).
 
 ## Example Usage
 
 ```terraform
+# basic resource
 resource "snowflake_masking_policy" "test" {
   name     = "EXAMPLE_MASKING_POLICY"
   database = "EXAMPLE_DB"
   schema   = "EXAMPLE_SCHEMA"
-  signature {
-    column {
-      name = "val"
-      type = "VARCHAR"
-    }
+  argument {
+    name = "ARG1"
+    type = "VARCHAR"
   }
-  masking_expression = <<-EOF
-    case 
-      when current_role() in ('ROLE_A') then 
-        val 
-      when is_role_in_session( 'ROLE_B' ) then 
-        'ABC123'
-      else
-        '******'
-    end
-  EOF
-
+  argument {
+    name = "ARG2"
+    type = "NUMBER"
+  }
+  argument {
+    name = "ARG3"
+    type = "TIMESTAMP_NTZ"
+  }
+  body             = <<-EOF
+  case
+    when current_role() in ('ROLE_A') then
+      ARG1
+    when is_role_in_session( 'ROLE_B' ) then
+      'ABC123'
+    else
+      '******'
+  end
+EOF
   return_data_type = "VARCHAR"
 }
+
+# resource with all fields set
+resource "snowflake_masking_policy" "test" {
+  name     = "EXAMPLE_MASKING_POLICY"
+  database = "EXAMPLE_DB"
+  schema   = "EXAMPLE_SCHEMA"
+  argument {
+    name = "ARG1"
+    type = "VARCHAR"
+  }
+  argument {
+    name = "ARG2"
+    type = "NUMBER"
+  }
+  argument {
+    name = "ARG3"
+    type = "TIMESTAMP_NTZ"
+  }
+  body                  = <<-EOF
+  case
+    when current_role() in ('ROLE_A') then
+      ARG1
+    when is_role_in_session( 'ROLE_B' ) then
+      'ABC123'
+    else
+      '******'
+  end
+EOF
+  return_data_type      = "VARCHAR"
+  exempt_other_policies = "true"
+  comment               = "example masking policy"
+}
 ```
 
 -> **Note** Instead of using fully_qualified_name, you can reference objects managed outside Terraform by constructing a correct ID, consult [identifiers guide](https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest/docs/guides/identifiers#new-computed-fully-qualified-name-field-in-resources).
@@ -45,45 +83,73 @@ resource "snowflake_masking_policy" "test" {
 
 ### Required
 
-- `database` (String) The database in which to create the masking policy.
-- `masking_expression` (String) Specifies the SQL expression that transforms the data.
-- `name` (String) Specifies the identifier for the masking policy; must be unique for the database and schema in which the masking policy is created.
-- `return_data_type` (String) Specifies the data type to return.
-- `schema` (String) The schema in which to create the masking policy.
-- `signature` (Block List, Min: 1, Max: 1) The signature for the masking policy; specifies the input columns and data types to evaluate at query runtime. (see [below for nested schema](#nestedblock--signature))
+- `argument` (Block List, Min: 1) List of the arguments for the masking policy. The first column and its data type always indicate the column data type values to mask or tokenize in the subsequent policy conditions. Note that you can not specify a virtual column as the first column argument in a conditional masking policy. (see [below for nested schema](#nestedblock--argument))
+- `body` (String) Specifies the SQL expression that transforms the data. To mitigate permadiff on this field, the provider replaces blank characters with a space. This can lead to false positives in cases where a change in case or run of whitespace is semantically significant.
+- `database` (String) The database in which to create the masking policy. Due to technical limitations (read more [here](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/docs/technical-documentation/identifiers_rework_design_decisions.md#known-limitations-and-identifier-recommendations)), avoid using the following characters: `|`, `.`, `(`, `)`, `"`
+- `name` (String) Specifies the identifier for the masking policy; must be unique for the database and schema in which the masking policy is created. Due to technical limitations (read more [here](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/docs/technical-documentation/identifiers_rework_design_decisions.md#known-limitations-and-identifier-recommendations)), avoid using the following characters: `|`, `.`, `(`, `)`, `"`
+- `return_data_type` (String) The return data type must match the input data type of the first column that is specified as an input column. For more information about data types, check [Snowflake docs](https://docs.snowflake.com/en/sql-reference/intro-summary-data-types).
+- `schema` (String) The schema in which to create the masking policy. Due to technical limitations (read more [here](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/docs/technical-documentation/identifiers_rework_design_decisions.md#known-limitations-and-identifier-recommendations)), avoid using the following characters: `|`, `.`, `(`, `)`, `"`
 
 ### Optional
 
 - `comment` (String) Specifies a comment for the masking policy.
-- `exempt_other_policies` (Boolean) Specifies whether the row access policy or conditional masking policy can reference a column that is already protected by a masking policy.
-- `if_not_exists` (Boolean) Prevent overwriting a previous masking policy with the same name.
-- `or_replace` (Boolean) Whether to override a previous masking policy with the same name.
+- `exempt_other_policies` (String) Specifies whether the row access policy or conditional masking policy can reference a column that is already protected by a masking policy. Due to Snowflake limitations, when value is chenged, the resource is recreated. Available options are: "true" or "false". When the value is not set in the configuration the provider will put "default" there which means to use the Snowflake default for this value.
 
 ### Read-Only
 
+- `describe_output` (List of Object) Outputs the result of `DESCRIBE MASKING POLICY` for the given masking policy. (see [below for nested schema](#nestedatt--describe_output))
 - `fully_qualified_name` (String) Fully qualified name of the resource. For more information, see [object name resolution](https://docs.snowflake.com/en/sql-reference/name-resolution).
 - `id` (String) The ID of this resource.
+- `show_output` (List of Object) Outputs the result of `SHOW MASKING POLICY` for the given masking policy. (see [below for nested schema](#nestedatt--show_output))
 
-<a id="nestedblock--signature"></a>
-### Nested Schema for `signature`
+<a id="nestedblock--argument"></a>
+### Nested Schema for `argument`
 
 Required:
 
-- `column` (Block List, Min: 1) (see [below for nested schema](#nestedblock--signature--column))
+- `name` (String) The argument name
+- `type` (String) The argument type. VECTOR data types are not yet supported. For more information about data types, check [Snowflake docs](https://docs.snowflake.com/en/sql-reference/intro-summary-data-types).
 
-<a id="nestedblock--signature--column"></a>
-### Nested Schema for `signature.column`
 
-Required:
+<a id="nestedatt--describe_output"></a>
+### Nested Schema for `describe_output`
+
+Read-Only:
+
+- `body` (String)
+- `name` (String)
+- `return_type` (String)
+- `signature` (List of Object) (see [below for nested schema](#nestedobjatt--describe_output--signature))
+
+<a id="nestedobjatt--describe_output--signature"></a>
+### Nested Schema for `describe_output.signature`
+
+Read-Only:
+
+- `name` (String)
+- `type` (String)
+
+
+
+<a id="nestedatt--show_output"></a>
+### Nested Schema for `show_output`
+
+Read-Only:
 
-- `name` (String) Specifies the column name to mask.
-- `type` (String) Specifies the column type to mask.
+- `comment` (String)
+- `created_on` (String)
+- `database_name` (String)
+- `exempt_other_policies` (Boolean)
+- `kind` (String)
+- `name` (String)
+- `owner` (String)
+- `owner_role_type` (String)
+- `schema_name` (String)
 
 ## Import
 
 Import is supported using the following syntax:
 
 ```shell
-# format is database name | schema name | policy name
-terraform import snowflake_masking_policy.example 'dbName|schemaName|policyName'
+terraform import snowflake_row_access_policy.example '"<database_name>"."<schema_name>"."<masking_policy_name>"'
 ```

examples/resources/snowflake_masking_policy/import.sh

@@ -1,2 +1 @@
-# format is database name | schema name | policy name
-terraform import snowflake_masking_policy.example 'dbName|schemaName|policyName'
+terraform import snowflake_row_access_policy.example '"<database_name>"."<schema_name>"."<masking_policy_name>"'

examples/resources/snowflake_masking_policy/resource.tf

@@ -1,23 +1,61 @@
+# basic resource
 resource "snowflake_masking_policy" "test" {
   name     = "EXAMPLE_MASKING_POLICY"
   database = "EXAMPLE_DB"
   schema   = "EXAMPLE_SCHEMA"
-  signature {
-    column {
-      name = "val"
-      type = "VARCHAR"
-    }
+  argument {
+    name = "ARG1"
+    type = "VARCHAR"
   }
-  masking_expression = <<-EOF
-    case 
-      when current_role() in ('ROLE_A') then 
-        val 
-      when is_role_in_session( 'ROLE_B' ) then 
-        'ABC123'
-      else
-        '******'
-    end
-  EOF
-
+  argument {
+    name = "ARG2"
+    type = "NUMBER"
+  }
+  argument {
+    name = "ARG3"
+    type = "TIMESTAMP_NTZ"
+  }
+  body             = <<-EOF
+  case
+    when current_role() in ('ROLE_A') then
+      ARG1
+    when is_role_in_session( 'ROLE_B' ) then
+      'ABC123'
+    else
+      '******'
+  end
+EOF
   return_data_type = "VARCHAR"
 }
+
+# resource with all fields set
+resource "snowflake_masking_policy" "test" {
+  name     = "EXAMPLE_MASKING_POLICY"
+  database = "EXAMPLE_DB"
+  schema   = "EXAMPLE_SCHEMA"
+  argument {
+    name = "ARG1"
+    type = "VARCHAR"
+  }
+  argument {
+    name = "ARG2"
+    type = "NUMBER"
+  }
+  argument {
+    name = "ARG3"
+    type = "TIMESTAMP_NTZ"
+  }
+  body                  = <<-EOF
+  case
+    when current_role() in ('ROLE_A') then
+      ARG1
+    when is_role_in_session( 'ROLE_B' ) then
+      'ABC123'
+    else
+      '******'
+  end
+EOF
+  return_data_type      = "VARCHAR"
+  exempt_other_policies = "true"
+  comment               = "example masking policy"
+}

go.mod

@@ -6,7 +6,6 @@ require (
 	github.com/DATA-DOG/go-sqlmock v1.5.2
 	github.com/avast/retry-go v3.0.0+incompatible
 	github.com/brianvoe/gofakeit/v6 v6.28.0
-	github.com/buger/jsonparser v1.1.1
 	github.com/gookit/color v1.5.4
 	github.com/hashicorp/go-cty v1.4.1-0.20200414143053-d3edf31b6320
 	github.com/hashicorp/go-uuid v1.0.3

go.sum

@@ -75,8 +75,6 @@ github.com/brianvoe/gofakeit/v6 v6.28.0 h1:Xib46XXuQfmlLS2EXRuJpqcw8St6qSZz75OUo
 github.com/brianvoe/gofakeit/v6 v6.28.0/go.mod h1:Xj58BMSnFqcn/fAQeSK+/PLtC5kSb7FJIq4JyGa8vEs=
 github.com/bufbuild/protocompile v0.4.0 h1:LbFKd2XowZvQ/kajzguUp2DC9UEIQhIq77fZZlaQsNA=
 github.com/bufbuild/protocompile v0.4.0/go.mod h1:3v93+mbWn/v3xzN+31nwkJfrEpAUwp+BagBSZWx+TP8=
-github.com/buger/jsonparser v1.1.1 h1:2PnMjfWD7wBILjqQbt530v576A/cAbQvEW9gGIpYMUs=
-github.com/buger/jsonparser v1.1.1/go.mod h1:6RYKKt7H4d4+iWqouImQ9R2FZql3VbhNgx27UK13J/0=
 github.com/cloudflare/circl v1.3.7 h1:qlCDlTPz2n9fu58M0Nh1J/JzcFpfgkFHHX3O35r5vcU=
 github.com/cloudflare/circl v1.3.7/go.mod h1:sRTcRWXGLrKw6yIGJ+l7amYJFfAXbZG0kBSc8r4zxgA=
 github.com/cyphar/filepath-securejoin v0.2.4 h1:Ugdm7cg7i6ZK6x3xDF1oEu1nfkyfH53EtKeQYTC3kyg=

pkg/acceptance/bettertestspoc/assert/objectassert/gen/sdk_object_def.go

@@ -47,6 +47,11 @@ var allStructs = []SdkObjectDef{
 		ObjectType:   sdk.ObjectTypeResourceMonitor,
 		ObjectStruct: sdk.ResourceMonitor{},
 	},
+	{
+		IdType:       "sdk.SchemaObjectIdentifier",
+		ObjectType:   sdk.ObjectTypeMaskingPolicy,
+		ObjectStruct: sdk.MaskingPolicy{},
+	},
 	{
 		IdType:       "sdk.SchemaObjectIdentifier",
 		ObjectType:   sdk.ObjectTypeAuthenticationPolicy,