Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"${ARN_PARTITION}" needs to be replaced with "aws" (or equivalent) in several AWS IAM policies #114

Closed
bbreak opened this issue May 13, 2022 · 1 comment

Comments

@bbreak
Copy link

bbreak commented May 13, 2022

Several of the AWS IAM policies published under the public cloudbreak repository now have "${ARN_PARTITION}" replacement prompts included in their Resource references. For example,

"Resource": "arn:aws:s3:::${DATALAKE_BUCKET}"

has been replaced with

"Resource": "arn:${ARN_PARTITION}:s3:::${DATALAKE_BUCKET}"

(I believe this change is related to work required to support GovCloud. )

As a result, environment deployment attempts have started failing with errors like the following:

TASK [cloudera.exe.platform : Create CDP Data Access Policies] *************************************************************************************************************************************************************
Wednesday 11 May 2022  22:53:56 +0000 (0:00:01.692)       0:05:02.625 ********* 
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.errorfactory.MalformedPolicyDocumentException: An error occurred (MalformedPolicyDocument) when calling the CreatePolicy operation: Partition "${ARN_PARTITION}" is not valid for resource "arn:${ARN_PARTITION}:s3:::sup-rp-uet2".
failed: [localhost] (item=sup-rp-logs-policy) => {"__aws_policy_details_item": {"description": "CDP Log Location Access", "key": "log", "name": "sup-rp-logs-policy"}, "ansible_loop_var": "__aws_policy_details_item", "boto3_version": "1.21.40", "botocore_version": "1.24.40", "changed": false, "error": {"code": "MalformedPolicyDocument", "message": "Partition \"${ARN_PARTITION}\" is not valid for resource \"arn:${ARN_PARTITION}:s3:::sup-rp-uet2\".", "type": "Sender"}, "msg": "Couldn't create policy sup-rp-logs-policy: An error occurred (MalformedPolicyDocument) when calling the CreatePolicy operation: Partition \"${ARN_PARTITION}\" is not valid for resource \"arn:${ARN_PARTITION}:s3:::sup-rp-uet2\".", "response_metadata": {"http_headers": {"connection": "close", "content-length": "370", "content-type": "text/xml", "date": "Wed, 11 May 2022 22:54:05 GMT", "x-amzn-requestid": "6d5d6b50-3917-479b-92c6-81618857a0ea"}, "http_status_code": 400, "request_id": "6d5d6b50-3917-479b-92c6-81618857a0ea", "retry_attempts": 0}}
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.errorfactory.MalformedPolicyDocumentException: An error occurred (MalformedPolicyDocument) when calling the CreatePolicy operation: Partition "${ARN_PARTITION}" is not valid for resource "arn:${ARN_PARTITION}:s3:::sup-rp-uet2/ranger/audit/*".
failed: [localhost] (item=sup-rp-audit-policy) => {"__aws_policy_details_item": {"description": "CDP Ranger Audit S3 Access", "key": "ranger_audit_s3", "name": "sup-rp-audit-policy"}, "ansible_loop_var": "__aws_policy_details_item", "boto3_version": "1.21.40", "botocore_version": "1.24.40", "changed": false, "error": {"code": "MalformedPolicyDocument", "message": "Partition \"${ARN_PARTITION}\" is not valid for resource \"arn:${ARN_PARTITION}:s3:::sup-rp-uet2/ranger/audit/*\".", "type": "Sender"}, "msg": "Couldn't create policy sup-rp-audit-policy: An error occurred (MalformedPolicyDocument) when calling the CreatePolicy operation: Partition \"${ARN_PARTITION}\" is not valid for resource \"arn:${ARN_PARTITION}:s3:::sup-rp-uet2/ranger/audit/*\".", "response_metadata": {"http_headers": {"connection": "close", "content-length": "385", "content-type": "text/xml", "date": "Wed, 11 May 2022 22:54:12 GMT", "x-amzn-requestid": "483671df-ad62-48b8-9c53-bcb7a5c44714"}, "http_status_code": 400, "request_id": "483671df-ad62-48b8-9c53-bcb7a5c44714", "retry_attempts": 0}}
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.errorfactory.MalformedPolicyDocumentException: An error occurred (MalformedPolicyDocument) when calling the CreatePolicy operation: Partition "${ARN_PARTITION}" is not valid for resource "arn:${ARN_PARTITION}:s3:::sup-rp-uet2".
failed: [localhost] (item=sup-rp-dladmin-policy) => {"__aws_policy_details_item": {"description": "CDP Datalake Admin S3 Access", "key": "datalake_admin_s3", "name": "sup-rp-dladmin-policy"}, "ansible_loop_var": "__aws_policy_details_item", "boto3_version": "1.21.40", "botocore_version": "1.24.40", "changed": false, "error": {"code": "MalformedPolicyDocument", "message": "Partition \"${ARN_PARTITION}\" is not valid for resource \"arn:${ARN_PARTITION}:s3:::sup-rp-uet2\".", "type": "Sender"}, "msg": "Couldn't create policy sup-rp-dladmin-policy: An error occurred (MalformedPolicyDocument) when calling the CreatePolicy operation: Partition \"${ARN_PARTITION}\" is not valid for resource \"arn:${ARN_PARTITION}:s3:::sup-rp-uet2\".", "response_metadata": {"http_headers": {"connection": "close", "content-length": "370", "content-type": "text/xml", "date": "Wed, 11 May 2022 22:54:20 GMT", "x-amzn-requestid": "06e9eb59-961e-4645-a0fa-3367f4b5132d"}, "http_status_code": 400, "request_id": "06e9eb59-961e-4645-a0fa-3367f4b5132d", "retry_attempts": 0}}
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.errorfactory.MalformedPolicyDocumentException: An error occurred (MalformedPolicyDocument) when calling the CreatePolicy operation: Partition "${ARN_PARTITION}" is not valid for resource "arn:${ARN_PARTITION}:s3:::sup-rp-uet2".
failed: [localhost] (item=sup-rp-storage-policy) => {"__aws_policy_details_item": {"description": "CDP Bucket S3 Access", "key": "bucket_access", "name": "sup-rp-storage-policy"}, "ansible_loop_var": "__aws_policy_details_item", "boto3_version": "1.21.40", "botocore_version": "1.24.40", "changed": false, "error": {"code": "MalformedPolicyDocument", "message": "Partition \"${ARN_PARTITION}\" is not valid for resource \"arn:${ARN_PARTITION}:s3:::sup-rp-uet2\".", "type": "Sender"}, "msg": "Couldn't create policy sup-rp-storage-policy: An error occurred (MalformedPolicyDocument) when calling the CreatePolicy operation: Partition \"${ARN_PARTITION}\" is not valid for resource \"arn:${ARN_PARTITION}:s3:::sup-rp-uet2\".", "response_metadata": {"http_headers": {"connection": "close", "content-length": "370", "content-type": "text/xml", "date": "Wed, 11 May 2022 22:54:26 GMT", "x-amzn-requestid": "d992095d-cf39-42d9-9ed0-3d5f4af3ee53"}, "http_status_code": 400, "request_id": "d992095d-cf39-42d9-9ed0-3d5f4af3ee53", "retry_attempts": 0}}

PLAY RECAP *****************************************************************************************************************************************************************************************************************
localhost                  : ok=212  changed=32   unreachable=0    failed=1    skipped=152  rescued=0    ignored=0   

It will be necessary to replace "${ARN_PARTITION}" with "aws" (or a GovCloud equivalent).

I found that replacement tag in the following IAM policy templates:
aws-cdp-ranger-audit-s3-policy.json
aws-cdp-datalake-admin-s3-policy.json
aws-cdp-bucket-access-policy.json
aws-cdp-dynamodb-policy.json (although this one can probably be ignored)
aws-cdp-log-policy.json

@bbreak
Copy link
Author

bbreak commented May 13, 2022

And now I see that I failed to check issues that are already closed. It appears #112 already addresses this problem.

@bbreak bbreak closed this as completed May 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant