-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Please support AWS IAM Instance profiles! #12464
Comments
Yes, I think so. But still, it needs a bit of work in Functionbeat. :) |
@davidmco65 Thanks for creating this issue. When you mentioned |
@kaiyan-sheng No, the default credential provider chain runs through a set of potential credential providers. One of which is the ~/.aws/credentials file, but others include the EC2 Instance Profile - a role that's assigned to the EC2 instance running the beat and would give the beat the permissions it needs to access the resources it needs, i.e., S3 bucket. In our environment, we literally cannot use access key id/secret key, so there is no way to configure the function beat that will allow it to install properly. Thanks! |
I need to be able to specify a role to assume in a child account via a named profile present in the ~/.aws/config file. This is typically passed as https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html |
I also need this but in Metricbeat, should I open a new issue? |
@Thiago-Dantas I'm planning to use this same issue to track for Metricbeat as well! Thanks! |
I have a user who is interested in this feature so they don't have to store creds locally on the ec2 instance. |
@kaiyan-sheng Are you sure this issue can be closed? The original author requested the feature for Functionbeat. However, your PR only adds support in metrcibeat and filebeat. |
No, it does not. Functionbeat depends on the AWS SDK to handle credentials. Thus, it only supports passing those via environment variables. Thank you for reopening. |
If Functionbeat uses AWS SDK, what is the credential provider chain is being used. Wouldn't AWS SDK allow to use Instance profile credentials by default (in case of Default Credential Provider Chain)? |
I guess yes, but we haven't tested it yet. Also, all of the information I was able to found on the subject is for the v1 of |
Hi @kvch, since this issue is related to Functionbeat, is it ok if I assign it to you instead? |
@kaiyan-sheng I've just assinged myself. Thanks for the ping. |
@kvch Any luck with implementing this for functionbeat? Thanks! |
Fellas, i come from the future and this problem still persists. Functionbeat seems to still be using AWS env variable credentials. I am currently playing with Elastic stack using one single node on as private network and i cannot afford to put my credentials there, specially when the list of permissions necessary: https://www.elastic.co/guide/en/beats/functionbeat/current/iam-permissions.html#iam-permissions-cloudwatch Is quite big. Looks like a disaster ready to happen. |
## What does this PR do? This PR makes credential settings when deploying Lambdas to AWS more flexible. New options are introduced: 1. `access_key_id`, `secret_access_key` and/or `session_token` for tokens ```yaml functionbeat.provider.aws.access_key_id: '${AWS_ACCESS_KEY_ID:""}' functionbeat.provider.aws.secret_access_key: '${AWS_SECRET_ACCESS_KEY:""}' functionbeat.provider.aws.session_token: '${AWS_SESSION_TOKEN:""}' ``` 2. `role_arn` for assuming IAM roles ```yaml functionbeat.provider.aws.role_arn: arn:aws:iam::123456789012:role/test-fnb ``` 3. `credential_profile_name` and/or `shared_credential_file` for credential files ```yaml functionbeat.provider.aws.credential_profile_name: fnb-aws functionbeat.provider.aws.shared_credential_file: /etc/functionbeat/aws_credentials ``` ## Why is it important? Credential configuration becomes more flexible and follows the same pattern as in Filebeat and Metricbeat. ## Related issues Based on #17658 Closes #12464 Co-authored-by: Brandon Morelli <[email protected]>
This PR makes credential settings when deploying Lambdas to AWS more flexible. New options are introduced: 1. `access_key_id`, `secret_access_key` and/or `session_token` for tokens ```yaml functionbeat.provider.aws.access_key_id: '${AWS_ACCESS_KEY_ID:""}' functionbeat.provider.aws.secret_access_key: '${AWS_SECRET_ACCESS_KEY:""}' functionbeat.provider.aws.session_token: '${AWS_SESSION_TOKEN:""}' ``` 2. `role_arn` for assuming IAM roles ```yaml functionbeat.provider.aws.role_arn: arn:aws:iam::123456789012:role/test-fnb ``` 3. `credential_profile_name` and/or `shared_credential_file` for credential files ```yaml functionbeat.provider.aws.credential_profile_name: fnb-aws functionbeat.provider.aws.shared_credential_file: /etc/functionbeat/aws_credentials ``` Credential configuration becomes more flexible and follows the same pattern as in Filebeat and Metricbeat. Based on elastic#17658 Closes elastic#12464 Co-authored-by: Brandon Morelli <[email protected]> (cherry picked from commit 5e6558b)
…3386) This PR makes credential settings when deploying Lambdas to AWS more flexible. New options are introduced: 1. `access_key_id`, `secret_access_key` and/or `session_token` for tokens ```yaml functionbeat.provider.aws.access_key_id: '${AWS_ACCESS_KEY_ID:""}' functionbeat.provider.aws.secret_access_key: '${AWS_SECRET_ACCESS_KEY:""}' functionbeat.provider.aws.session_token: '${AWS_SESSION_TOKEN:""}' ``` 2. `role_arn` for assuming IAM roles ```yaml functionbeat.provider.aws.role_arn: arn:aws:iam::123456789012:role/test-fnb ``` 3. `credential_profile_name` and/or `shared_credential_file` for credential files ```yaml functionbeat.provider.aws.credential_profile_name: fnb-aws functionbeat.provider.aws.shared_credential_file: /etc/functionbeat/aws_credentials ``` Credential configuration becomes more flexible and follows the same pattern as in Filebeat and Metricbeat. Based on #17658 Closes #12464 Co-authored-by: Brandon Morelli <[email protected]> (cherry picked from commit 5e6558b)
I can't use the Functionbeat on my EC2 server because it doesn't appear to use the default credential provider chain. Meaning that I have a Role assigned to me EC2 instance that gives it permission to access the bucket specified in the configuration, but the deploy function doesn't see use it and therefore has no permissions to do anything with S3.
I'm trying to deploy the functionbeat from our EC2 server and it's not possible given this missing functionality. I've got my entire ELK stack deployment scripted except for this functionbeat.
The text was updated successfully, but these errors were encountered: