chore: Enables `mongodbatlas_encryption_at_rest` (Azure) tests to run in CI #2551

maastha · 2024-09-03T20:02:53Z

Description

Enables mongodbatlas_encryption_at_rest and mongodbatlas_encryption_at_rest_private_endpoint Azure tests to run in CI.

Link to any related issue(s): CLOUDP-267663

Type of change:

Bug fix (non-breaking change which fixes an issue). Please, add the "bug" label to the PR.
New feature (non-breaking change which adds functionality). Please, add the "enhancement" label to the PR. A migration guide must be created or updated if the new feature will go in a major version.
Breaking change (fix or feature that would cause existing functionality to not work as expected). Please, add the "breaking change" label to the PR. A migration guide must be created or updated.
This change requires a documentation update
Documentation fix/enhancement

Required Checklist:

I have signed the MongoDB CLA
I have read the contributing guides
I have checked that this change does not generate any credentials and that they are NOT accidentally logged anywhere.
I have added tests that prove my fix is effective or that my feature works per HashiCorp requirements
I have added any necessary documentation (if appropriate)
I have run make fmt and formatted my code
If changes include deprecations or removals I have added appropriate changelog entries.
If changes include removal or addition of 3rd party GitHub actions, I updated our internal document. Reach out to the APIx Integration slack channel to get access to the internal document.

Further comments

…r-msg

…r-ci

maastha · 2024-09-04T10:39:03Z

internal/service/encryptionatrest/resource_migration_test.go

@@ -131,19 +141,16 @@ func TestMigEncryptionAtRest_basicAzure(t *testing.T) {
 		Steps: []resource.TestStep{
 			{
 				ExternalProviders: mig.ExternalProviders(),
-				Config:            acc.ConfigEARAzureKeyVault(projectID, &azureKeyVault, false),
+				Config:            acc.ConfigEARAzureKeyVault(projectID, &azureKeyVault, false, false), // not using data source as it was introduced in 1.19.0


chose to not use data source for migration tests instead of skipping the tests for lower versions

mig.IsProviderVersionAtLeast can be used instead of directly false, you can look for some examples in the code

maastha · 2024-09-04T11:16:22Z

contributing/development-setup.md

  export AZURE_KEY_VAULT_NAME=<YOUR_KEY_VAULT_NAME>
  export AZURE_KEY_IDENTIFIER=<YOUR_KEY_IDENTIFIER>
  export AZURE_TENANT_ID=<YOUR_TENANT_ID>
  export AZURE_DIRECTORY_ID=<YOUR_DIRECTORY_ID>

-  export AZURE_CLIENT_ID_UPDATED=<YOUR_CLIENT_ID_UPDATED>


no longer being used

maastha · 2024-09-04T11:17:34Z

internal/service/encryptionatrest/resource_test.go

@@ -105,25 +106,9 @@ func TestAccEncryptionAtRest_basicAWS(t *testing.T) {
 	})
 }

-func convertToAzureKeyVaultAttrMap(az *admin.AzureKeyVault) map[string]string {


moved function to below, no change

maastha · 2024-09-04T11:24:50Z

internal/service/encryptionatrest/resource_test.go

 	var (
-		projectID = os.Getenv("MONGODB_ATLAS_PROJECT_ID")
+		projectID = os.Getenv("MONGODB_ATLAS_PROJECT_EAR_PE_ID")


Created new Atlas project test-acc-tf-p-keep-ear-private-endpoint with preview flag enabled for this feature

maastha · 2024-09-04T14:38:11Z

internal/service/encryptionatrest/resource_test.go

-	return resource.ComposeAggregateTestCheckFunc(checks...)
-}
-
-func TestAccEncryptionAtRest_azure_requirePrivateNetworking_preview(t *testing.T) {


Had to move this test to encryptionatrestprivateendpoint/resource_test.go since it was updated to use MONGODB_ATLAS_PROJECT_EAR_PE_ID (static atlas project created with PE flag enabled in cloud-dev). Since these tests use resources (project, key vault, etc) common with private endpoint tests, keeping them in the same encryptionatrestprivateendpoint package so tests can run sequentially.

AgustinBettati

LGTM, would just consider making the adjustment now so this test runs correctly in QA

AgustinBettati · 2024-09-04T16:13:01Z

.github/workflows/acceptance-tests.yml

      mongodb_atlas_gov_project_owner_id: ${{ inputs.atlas_cloud_env == 'qa' && vars.MONGODB_ATLAS_GOV_PROJECT_OWNER_ID_QA || vars.MONGODB_ATLAS_GOV_PROJECT_OWNER_ID_DEV }}
      mongodb_atlas_federated_settings_associated_domain: ${{ vars.MONGODB_ATLAS_FEDERATED_SETTINGS_ASSOCIATED_DOMAIN }}
+      mongodb_atlas_project_ear_pe_id: ${{ vars.MONGODB_ATLAS_PROJECT_EAR_PE_ID }}


we will likely need to create another project in QA with the FF enabled, after this adjustment we can to a run of the test group in QA to make sure we already support this correctly.

AgustinBettati

LGTM

lantoli · 2024-09-05T08:58:28Z

.github/workflows/acceptance-tests.yml

@@ -104,3 +110,6 @@ jobs:
      mongodb_atlas_gov_org_id: ${{ inputs.atlas_cloud_env == 'qa' && vars.MONGODB_ATLAS_GOV_ORG_ID_QA || vars.MONGODB_ATLAS_GOV_ORG_ID_DEV }}
      mongodb_atlas_gov_project_owner_id: ${{ inputs.atlas_cloud_env == 'qa' && vars.MONGODB_ATLAS_GOV_PROJECT_OWNER_ID_QA || vars.MONGODB_ATLAS_GOV_PROJECT_OWNER_ID_DEV }}
      mongodb_atlas_federated_settings_associated_domain: ${{ vars.MONGODB_ATLAS_FEDERATED_SETTINGS_ASSOCIATED_DOMAIN }}
+      mongodb_atlas_project_ear_pe_id: ${{ inputs.atlas_cloud_env == 'qa' && vars.MONGODB_ATLAS_PROJECT_EAR_PE_ID_QA || vars.MONGODB_ATLAS_PROJECT_EAR_PE_ID_DEV }}


great that it also works for QA

maastha · 2024-09-05T08:59:28Z

Verified test run in QA - https://github.com/mongodb/terraform-provider-mongodbatlas/actions/runs/10717078915/job/29716005905

lantoli · 2024-09-05T09:00:33Z

internal/service/encryptionatrest/resource_migration_test.go

@@ -46,7 +48,7 @@ func TestMigEncryptionAtRest_basicAWS(t *testing.T) {
 			},
 			{
 				ProtoV6ProviderFactories: acc.TestAccProviderV6Factories,
-				Config:                   testAccMongoDBAtlasEncryptionAtRestConfigAwsKms(projectID, &awsKms),
+				Config:                   testAccMongoDBAtlasEncryptionAtRestConfigAwsKms(projectID, &awsKms, false),
 				ConfigPlanChecks: resource.ConfigPlanChecks{


can we better use mig.TestStepCheckEmptyPlan ?

…o master (#2569) * update sdk dev (#2490) * chore: Creates TF models & interfaces for new `mongodbatlas_encryption_at_rest_private_endpoint` resource (#2493) * chore: Creates TF models & interfaces for new `mongodbatlas_encryption_at_rest_private_endpoint` data source (#2500) * feat: Updates `mongodbatlas_encryption_at_rest` resource to use new `azure_key_vault_config.require_private_networking` field (#2509) * chore: Creates TF models & interfaces for `mongodbatlas_encryption_at_rest_private_endpoints` plural data source (#2502) * feat: Implements `mongodbatlas_encryption_at_rest_private_endpoint` resource (#2512) * wip - implementing CRUD * include changelog entry * small adjustments * supporting state transition logic * implement acceptance test * add unit testing for state transitions * handle return error message if failed status is present * add acceptance test transitioning for public to private network * improve messaging for failed status * fix prechecks * use global const for resource name * avoid hardcoded value * adjust state transition logic for delete * adjusting target version in migration test to 1.19.0 * adjust default refresh to 30 seconds for quicker response * feat: Implements `mongodbatlas_encryption_at_rest_private_endpoint` singular data source (#2527) * implement singular data source * including changelog entry * doc: Updates existing documentation for `mongodbatlas_encryption_at_rest` resource to be auto-generated (#2529) * doc: Include example for new `mongodbatlas_encryption_at_rest_private_endpoint` resource (#2540) * Include example for ear with private endpoint * fix example * adjust readme * Update examples/mongodbatlas_encryption_at_rest_private_endpoint/azure/README.md Co-authored-by: maastha <[email protected]> * Update examples/mongodbatlas_encryption_at_rest_private_endpoint/azure/README.md Co-authored-by: maastha <[email protected]> * add example cli command * make use of variables to make value of resource id more compact --------- Co-authored-by: maastha <[email protected]> * feat: Implements new `mongodbatlas_encryption_at_rest_private_endpoints` data source (#2536) * temporary change to cloud provider access and getting latest sdk * implements plural data source * adapted cloud provider access with latest changes from dev preview * fix unit test * adding changelog entry * add changes to verify plural data source in basic test case * doc adjust to cloud_provider attribute * feat: Implements new `mongodbatlas_encryption_at_rest` singular data source & adds `valid` attribute for cloud provider configs in the resource (#2538) * fix: Adds error message handling to `mongodbatlas_encryption_at_rest_private_endpoint` resource (#2544) * doc: Adds documentation for new `encryption_at_rest_private_endpoint` resource and data sources (#2547) * adding documentation for encryption_at_rest_private_endpoint resource and data sources * align generated docs * minor typo fix * Adjust description of project_id to make it more concise * align note stating feature is available by request as defined in general docs * chore: Adopt latest changes from master into ear private endpoint dev branch to adopt latest SDK (#2549) * test: Reduce instance size and use of provisioned disk iops for test that verifies transition for symmetric to asymmetric configuration (#2503) * doc: Include changelog entries to mention 2 new guides (#2506) * add entry for 2 new guides * add link * chore: Updates examples link in index.md for v1.18.0 release * chore: Updates CHANGELOG.md header for v1.18.0 release * doc: Update Atlas SP db_role_to_execute info. (#2508) * (DOCSP-41590) Updating Atlas SP db_role_to_execute info. * Update docs/resources/stream_connection.md Co-authored-by: kanchana-mongodb <[email protected]> --------- Co-authored-by: kanchana-mongodb <[email protected]> * doc: Contributing Guidelines Updates (#2494) * Contributing Guidelines Updates * Update README.md * Update README.md * Update contributing/README.md Co-authored-by: kyuan-mongodb <[email protected]> --------- Co-authored-by: kyuan-mongodb <[email protected]> * test: Simply migration test checks after 1.18.0 release and adjust version constraint in advanced_cluster examples uing new schema (#2510) * doc: Add references to the terraform modules in the resources documentations (#2513) * add references to the modules in the resources documentations * fix pr comments * chore: Bump hashicorp/setup-terraform from 3.1.1 to 3.1.2 (#2515) Bumps [hashicorp/setup-terraform](https://github.com/hashicorp/setup-terraform) from 3.1.1 to 3.1.2. - [Release notes](https://github.com/hashicorp/setup-terraform/releases) - [Changelog](https://github.com/hashicorp/setup-terraform/blob/main/CHANGELOG.md) - [Commits](hashicorp/setup-terraform@651471c...b9cd54a) --- updated-dependencies: - dependency-name: hashicorp/setup-terraform dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: Add mention of support ticket when opening a pull request (#2507) * Add mention of creating support ticket when opening PR * rephrasing to avoid mention of priority * including suggestion * doc: Updates`mongodbatlas_advanced_cluster` ISS migration guide & resource doc with expected 500 error on update (#2525) * chore: Updates mongodbatlas_advanced_cluster tests to expect temporary SERVICE_UNAVAILABLE error when migrating from old to new schema (#2523) * doc: Fixes wordings in the new advanced_cluster sharding guide. (#2524) * chore: Updates examples link in index.md for v1.18.1 release * chore: Updates CHANGELOG.md header for v1.18.1 release * chore: upgrades go SDK from `v20240805001` to `v20240805002` (#2534) * chore: Updates to Go 1.23 (#2535) * update asdf TF version * update to Go 1.23 * update linter * update golang-ci linter * disable Go telemetry * revert TF change * chore: Bump go.mongodb.org/atlas from 0.36.0 to 0.37.0 (#2532) Bumps [go.mongodb.org/atlas](https://github.com/mongodb/go-client-mongodb-atlas) from 0.36.0 to 0.37.0. - [Release notes](https://github.com/mongodb/go-client-mongodb-atlas/releases) - [Changelog](https://github.com/mongodb/go-client-mongodb-atlas/blob/master/CHANGELOG.md) - [Commits](mongodb/go-client-mongodb-atlas@v0.36.0...v0.37.0) --- updated-dependencies: - dependency-name: go.mongodb.org/atlas dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: Bump github.com/hashicorp/hcl/v2 from 2.21.0 to 2.22.0 (#2530) Bumps [github.com/hashicorp/hcl/v2](https://github.com/hashicorp/hcl) from 2.21.0 to 2.22.0. - [Release notes](https://github.com/hashicorp/hcl/releases) - [Changelog](https://github.com/hashicorp/hcl/blob/main/CHANGELOG.md) - [Commits](hashicorp/hcl@v2.21.0...v2.22.0) --- updated-dependencies: - dependency-name: github.com/hashicorp/hcl/v2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * update asdf TF version to 1.9.5 (#2537) * chore: Changes deprecation message for labels attribute (#2542) * chore: Upgrades go SDK from `v20240805002` to `v20240805003` (#2545) * major version update calling gomajor tool * manual change to reincorporate v20240530005 * reverts temp changes in cloud provider resources, fixes sdk versions in new implementations --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: svc-apix-bot <[email protected]> Co-authored-by: lmkerbey-mdb <[email protected]> Co-authored-by: kanchana-mongodb <[email protected]> Co-authored-by: Zuhair Ahmed <[email protected]> Co-authored-by: kyuan-mongodb <[email protected]> Co-authored-by: rubenVB01 <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: maastha <[email protected]> Co-authored-by: Marco Suma <[email protected]> Co-authored-by: Espen Albert <[email protected]> Co-authored-by: Leo Antoli <[email protected]> Co-authored-by: Oriol <[email protected]> * doc: Adds documentation & examples for `mongodbatlas_encryption_at_rest` singular data source (#2543) * chore: Enables `mongodbatlas_encryption_at_rest` (Azure) tests to run in CI (#2551) * chore: Adds `mongodbatlas_encryption_at_rest_private_endpoint` acceptance test using azapi to approve private endpoint & check ACTIVE status (#2558) * doc: Add user journey considerations in current resource and example documentation (#2559) * minor typo fix * improve initial description in ear * adjust ear docs with mention of azure private link * private link doc adjustments * improve example * improve example * add mention in ear examples about policies * add note on update operation * link adjustments and add header for handling existing clusters * add note on private endpoint * add note in data sources * Update docs/resources/encryption_at_rest_private_endpoint.md Co-authored-by: maastha <[email protected]> * add clarification of preview flag for data sources --------- Co-authored-by: maastha <[email protected]> * update project_ip_addresses action * address doc comment --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Agustin Bettati <[email protected]> Co-authored-by: svc-apix-bot <[email protected]> Co-authored-by: lmkerbey-mdb <[email protected]> Co-authored-by: kanchana-mongodb <[email protected]> Co-authored-by: Zuhair Ahmed <[email protected]> Co-authored-by: kyuan-mongodb <[email protected]> Co-authored-by: rubenVB01 <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Marco Suma <[email protected]> Co-authored-by: Espen Albert <[email protected]> Co-authored-by: Leo Antoli <[email protected]> Co-authored-by: Oriol <[email protected]>

maastha added 18 commits September 2, 2024 11:17

add error message handling & unit test

fd8ba27

Merge branch 'CLOUDP-246459-ear-kms-dev' into CLOUDP-270958-handle-er…

fab4d75

…r-msg

minor

4bd71a8

lint fix

4687f74

update logic

e525d5a

minor

b601396

enable ear tests

8eadad1

Merge branch 'CLOUDP-246459-ear-kms-dev' into CLOUDP-267663-enable-ea…

87f4d81

…r-ci

minor

9a61029

update workflow

dbe74c8

minor

12c7243

minor

7750fa0

fix

241088a

minor

d1d95cd

add preview

3e5c7ab

tmp

442d4b3

enable EAR azure mig tests

76ca07b

Merge branch 'CLOUDP-246459-ear-kms-dev' into CLOUDP-267663-enable-ea…

a9fa82b

…r-ci

maastha commented Sep 4, 2024

View reviewed changes

maastha added 4 commits September 4, 2024 11:46

enable pe tests

dc561f9

minor

175984e

minor

3d30f00

ear pe

88331ec

maastha commented Sep 4, 2024

View reviewed changes

minor

ffcde2a

maastha commented Sep 4, 2024

View reviewed changes

maastha added 3 commits September 4, 2024 14:51

tmp

0b41905

move ear test

cbb97a1

lint fix

db06804

maastha commented Sep 4, 2024

View reviewed changes

maastha marked this pull request as ready for review September 4, 2024 14:40

maastha requested a review from a team as a code owner September 4, 2024 14:40

AgustinBettati reviewed Sep 4, 2024

View reviewed changes

update QA var

1e75193

AgustinBettati approved these changes Sep 5, 2024

View reviewed changes

lantoli reviewed Sep 5, 2024

View reviewed changes

maastha merged commit 2eb7cdb into CLOUDP-246459-ear-kms-dev Sep 5, 2024
57 checks passed

maastha deleted the CLOUDP-267663-enable-ear-ci branch September 5, 2024 09:00

lantoli reviewed Sep 5, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: Enables `mongodbatlas_encryption_at_rest` (Azure) tests to run in CI #2551

chore: Enables `mongodbatlas_encryption_at_rest` (Azure) tests to run in CI #2551

maastha commented Sep 3, 2024 •

edited

Loading

maastha Sep 4, 2024

lantoli Sep 5, 2024 •

edited

Loading

maastha Sep 4, 2024

maastha Sep 4, 2024

maastha Sep 4, 2024

maastha Sep 4, 2024 •

edited

Loading

AgustinBettati left a comment

AgustinBettati Sep 4, 2024

AgustinBettati left a comment

lantoli Sep 5, 2024

maastha commented Sep 5, 2024

lantoli Sep 5, 2024

chore: Enables mongodbatlas_encryption_at_rest (Azure) tests to run in CI #2551

chore: Enables mongodbatlas_encryption_at_rest (Azure) tests to run in CI #2551

Conversation

maastha commented Sep 3, 2024 • edited Loading

Description

Type of change:

Required Checklist:

Further comments

maastha Sep 4, 2024

Choose a reason for hiding this comment

lantoli Sep 5, 2024 • edited Loading

Choose a reason for hiding this comment

maastha Sep 4, 2024

Choose a reason for hiding this comment

maastha Sep 4, 2024

Choose a reason for hiding this comment

maastha Sep 4, 2024

Choose a reason for hiding this comment

maastha Sep 4, 2024 • edited Loading

Choose a reason for hiding this comment

AgustinBettati left a comment

Choose a reason for hiding this comment

AgustinBettati Sep 4, 2024

Choose a reason for hiding this comment

AgustinBettati left a comment

Choose a reason for hiding this comment

lantoli Sep 5, 2024

Choose a reason for hiding this comment

maastha commented Sep 5, 2024

lantoli Sep 5, 2024

Choose a reason for hiding this comment

chore: Enables `mongodbatlas_encryption_at_rest` (Azure) tests to run in CI #2551

chore: Enables `mongodbatlas_encryption_at_rest` (Azure) tests to run in CI #2551

maastha commented Sep 3, 2024 •

edited

Loading

lantoli Sep 5, 2024 •

edited

Loading

maastha Sep 4, 2024 •

edited

Loading