Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

output_plug aws cross account access for cloudwatch or s3 with external_id #3372

Closed
hardwaysun opened this issue Apr 16, 2021 · 4 comments
Closed
Labels

Comments

@hardwaysun
Copy link

hardwaysun commented Apr 16, 2021

When send log to aws cloudwatch from my_account where fluent bit running to user account, without external id it works fine. But with external id , like below
Create role on user account for me to access.
'
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::my_account_id:user/fluentd_role_in_my_account"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": "12345"
}
}
}
]
}
'
So it get arn as arn:aws:iam::user_account_id:role/user_role_allow_log_from_me

Using below output plugin, there are no field in data structure of out_cloudwatch_logs (struct flb_cloudwatch {}) or out_s3 (struct flb_s3 {}) to set the external id,

[OUTPUT]
Name cloudwatch_logs
Match USER_LOG.*
log_stream_name Log-for-user-stream
log_group_name Log-for-user-group
region us-west-1
auto_create_group true
role_arn arn:aws:iam::user_account_id:role/user_role_allow_log_from_me
sts_endpoint https://sts.us-west-1.amazonaws.com

[OUTPUT]
Name s3
Match USER_S3.*
bucket /fluentbit2S3/
region us-west1
use_put_object On
s3_key_format /$TAG[1]/$TAG[3]/%Y/%m/%d/
s3_key_format_tag_delimiters .
total_file_size 5M
upload_timeout 1m
role_arn arn:aws:iam::user_account_id:role/user_role_allow_log_from_me
sts_endpoint https://sts.us-west-1.amazonaws.com

So my question is , for out_cloudwatch_logs and out_s3 or out_kinesis_streams , does cross account with external id supported, if so how can I set it, if not do you planning support it or what is the best solution, like write my own plugin?

By the way looks the out_es supporting it.

one file fluent-bit/plugins/out_es/es.c

902 FLB_CONFIG_MAP_STR, "aws_external_id", NULL, 903 0, FLB_FALSE, 0, 904 "External ID for the AWS IAM Role specified with aws_role_arn" 905. },

@hardwaysun
Copy link
Author

By reading the code, it looks not supported, now the question is do you guys plan to support it soon?

@github-actions
Copy link
Contributor

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

@github-actions github-actions bot added the Stale label May 18, 2021
@github-actions
Copy link
Contributor

This issue was closed because it has been stalled for 5 days with no activity.

@PettitWesley
Copy link
Contributor

Implemented #4626
#4625

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants