Cloud Provider Access Setup for Datalake

[deaddog]

Terraform CLI and Terraform MongoDB Atlas Provider Version

terraform {
  required_version = "=1.0.0"

  required_providers {
    aws = ">= 3.0.0, < 4.0.0"
    mongodbatlas = {
      source = "mongodb/mongodbatlas"
      version = "0.9.1"
    }
  }
}

Terraform Configuration File

resource "mongodbatlas_cloud_provider_access_setup" "this" {
   project_id = "ATLAS-PROJECT-ID"
   provider_name = "AWS"
}

resource "mongodbatlas_cloud_provider_access_authorization" "this" {
   project_id =  mongodbatlas_cloud_provider_access_setup.this.project_id
   role_id    =  mongodbatlas_cloud_provider_access_setup.this.role_id

   aws = {
      iam_assumed_role_arn = aws_iam_role.this.arn
   }
}

resource "aws_iam_role" "this" {
  name = "atlas_lake"
  max_session_duration = 3600
  assume_role_policy = data.aws_iam_policy_document.atlas_assume_policy
}

# The role also has an attached policy for reading from S3 bucket(s)
# I haven't included it, as the setup "works". We can read from buckets via datalake.

data "aws_iam_policy_document" "atlas_assume_policy" {
  statement {
    sid     = ""
    effect  = "Allow"
    actions = ["sts:AssumeRole"]

    condition {
      test     = "StringEquals"
      variable = "sts:ExternalId"
      values   = [mongodbatlas_cloud_provider_access_setup.this.aws.atlas_assumed_role_external_id]
    }

    principals {
      type        = "AWS"
      identifiers = [mongodbatlas_cloud_provider_access_setup.this.aws.atlas_aws_account_arn]
    }
  }
}

Steps to Reproduce

1. Create AWS IAM role and Atlas provider access using the setup above (terraform apply)
2. Go to Atlas and create new datalake using new role (or updating role on existing datalake)
3. Run terraform plan

Expected Behavior

Terraform should be generate a plan with no changes, as all changes have already been applied.

Actual Behavior

Planning fails with the following message:

error reading cloud provider access feature_usages.0.feature_id: '' expected type 'string', got unconvertible type 'map[string]interface {}'

We only see the error when the role is in use for a datalake. When we remove the role from all datalakes we can plan again.

The datalake is able to access to bucket by assuming the role, but terraform is rendered unusable.

Additional Context

We use the same set of resources for encryption at rest, and don't experience this issue. Unfortunately we can't (yet) create datalakes directly from terraform, but establishing the IAM role along with its policy and Atlas access would simplify the process of creating a lake as we can provision bucket, user/role for applications, role and provider access in a single apply.

[themantissa]

@deaddog we haven't, until recently, done a great deal of testing for data lakes and cloud provider access as we just finished up the PR to support data lakes (#414). @coderGo93 can you take a look at this and see if this is a possible issue we need to address before v1.0.0.

[themantissa]

@deaddog our developer who created #414 confirmed he found the bug as well and corrected it in that PR. It will be out with v1.0.0, which we hope to release within, approximately, the next month.

[deaddog]

@themantissa that's sound great. We'll postpone using the provider access resources for lakes until v1.0.0 is released.

[themantissa]

@deaddog 1.0.0 is now out with Data Lake support. Enjoy!