LakeFormation not clearing default permissions

[automagic]

Describe what happened

Encountering an issue with AWS Lake Formation using Pulumi. Specifically, we are unable to clear the DatabaseDefaultPermissions and TableDefaultPermissions that are already set in the system.

Issue:
Despite setting createDatabaseDefaultPermissions and createTableDefaultPermissions to empty arrays, the permissions are not being cleared in the AWS Lake Formation settings. The expected behavior is that these permissions should be removed or reset to an empty state, but this does not seem to be happening.

According the AWS Document https://docs.aws.amazon.com/lake-formation/latest/dg/change-settings.html
Passing an empty array or omit that setting will revoke that permission, tried the both with: pulumi up – refresh, it can see the diff of the resources, but not applying the changes for the resource.

Sample program

export const dataLakeSettings = new aws.lakeformation.DataLakeSettings(createResourceName("data-lake-settings"), {
    admins: adminAccounts,
    createDatabaseDefaultPermissions: [],
    createTableDefaultPermissions: [],
});

Log output

No response

Affected Resource(s)

aws.lakeformation.DataLakeSettings

Output of pulumi about

CLI
Version 3.127.0
Go Version go1.22.5
Go Compiler gc

Plugins
KIND NAME VERSION
resource aws 6.47.0
language nodejs unknown
resource std 1.7.3

Host
OS Microsoft Windows 10 Enterprise
Version 10.0.19045 Build 19045
Arch x86_64

This project is written in nodejs: executable='C:\Program Files\nodejs\node.exe' version='v20.16.0'

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[t0yv0]

Thank you for reporting this and I am sorry pulumi-aws does not work as expected here! I've taken a pass at diagnosing the issue and it appears the feature of calling PutDataLakeSettings in this way is not yet supported. Per the documentation you referenced, the desired request has a body of:

{
    "DataLakeSettings": {
        "DataLakeAdmins": [
            {
                "DataLakePrincipalIdentifier": "arn:aws:iam::<AccountId>:user/<Username>"
            }
        ],
        "CreateDatabaseDefaultPermissions": [],
        "CreateTableDefaultPermissions": []
    }
}

Unfortunately given the TF heritage of the provider, create_database_default_permissions is defined as a block in TF and is unable to express the distinction between an empty value and a missing value in TF. This also seems to translate to the pulumi-aws projection of the provider.

Running pulumi up given:

const dataLakeSettings = new aws.lakeformation.DataLakeSettings("dls", {
    createTableDefaultPermissions: [],
});

Or else this:

const dataLakeSettings = new aws.lakeformation.DataLakeSettings("dls", {});

Sends this:

POST https://lakeformation.us-west-2.amazonaws.com/PutDataLakeSettings {"DataLakeSettings":{}}

resource "aws_lakeformation_data_lake_settings" "dls" {
  create_database_default_permissions {
    # permissions = ["SELECT", "ALTER", "DROP"]
    # principal   = aws_iam_user.test.arn
  }
}

Results in:

│ Error: creating Lake Formation data lake settings: creating Lake Formation dattDataLakeSettings, https response error StatusCode: 400, RequestID: 3d8c0ec5-362nvalid ARN:ARNs must start with 'arn:':
│
│   with aws_lakeformation_data_lake_settings.dls,
│   on infra.tf line 1, in resource "aws_lakeformation_data_lake_settings" "dls"│    1: resource "aws_lakeformation_data_lake_settings" "dls" {

While this:

resource "aws_lakeformation_data_lake_settings" "dls" {
}

Sends:

2024-08-14T11:40:03.506-0400 [DEBUG] provider.terraform-provider-aws_v5.62.0_x5: HTTP Request Sent: 
rpc.method=PutDataLakeSettings tf_req_id=e2fceb67-91b3-1c43-43fa-3f6545989906
  http.request.body=
  | {"DataLakeSettings":{}}

I'll circle back with the team as time permits to see if this feature can be added in a sensible way.

[wtfzambo]

I'm afraid this is an AWS API issue cause we're having the same exact problem with the AWS CDK. Cc. @rehanvdm