Convert aws_profiles from string to list so that we can have multiple aws accounts for dynamic inventory

[Akasurde]

From @dhikrahashim on Jul 28, 2020 05:37

SUMMARY

We have "aws_ec2" plugin for dynamic inventory, suppose if we have multiple AWS accounts we need to create inventory for each AWS account. If the aws_profile parameter is "list" type we need only one inventory playbook to fetch the inventories. We can use "with_items" or "loop" I guess

ISSUE TYPE

• Feature Idea

COMPONENT NAME

ADDITIONAL INFORMATION

plugin: aws_ec2
aws_profile:
  - new-relic-aws
  - splunk-aws
  - application-aws
  - database-aws

Copied from original issue: ansible/ansible#70928

[ansibullbot]

Files identified in the description:
None

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[next-jesusmanuelnavarro]

Didn't test this myself, but wouldn't dropping multiple inventory files on the same directory, one for each profile, render the same result?

[Porkepix]

Didn't test this myself, but wouldn't dropping multiple inventory files on the same directory, one for each profile, render the same result?

Yes it would, but you'd need to duplicate every single other configuration line in every of those files.

[next-jesusmanuelnavarro]

Didn't test this myself, but wouldn't dropping multiple inventory files on the same directory, one for each profile, render the same result?

Yes it would, but you'd need to duplicate every single other configuration line in every of those files.

I was not implying it to be a solution, just trying to know if that would be a viable workaround.

After all, I think we already know here how to produce a bunch of files that are basically equal except for a token we know in advance.

[goneri]

Thank you @next-jesusmanuelnavarro for the suggestion. The opposite is also true @Porkepix, if we loop on different aws_profile like you suggest, we cannot pass any profile specific configuration.

$ cat us-east-1-aws_ec2.yaml 
plugin: amazon.aws.aws_ec2
strict: True
regions:
- us-east-1
$ cat us-east-2-aws_ec2.yaml 
plugin: amazon.aws.aws_ec2
strict: True
regions:
- us-east-2
$ cat us-west-1-aws_ec2.yaml 
plugin: amazon.aws.aws_ec2
strict: True
regions:
- us-west-1
$ ansible-inventory -i us-east-1-aws_ec2.yaml -i us-east-2-aws_ec2.yaml -i us-west-1-aws_ec2.yaml --list
(...)

[Porkepix]

Thank you @next-jesusmanuelnavarro for the suggestion. The opposite is also true @Porkepix, if we loop on different aws_profile like you suggest, we cannot pass any profile specific configuration.

$ cat us-east-1-aws_ec2.yaml 
plugin: amazon.aws.aws_ec2
strict: True
regions:
- us-east-1
$ cat us-east-2-aws_ec2.yaml 
plugin: amazon.aws.aws_ec2
strict: True
regions:
- us-east-2
$ cat us-west-1-aws_ec2.yaml 
plugin: amazon.aws.aws_ec2
strict: True
regions:
- us-west-1
$ ansible-inventory -i us-east-1-aws_ec2.yaml -i us-east-2-aws_ec2.yaml -i us-west-1-aws_ec2.yaml --list
(...)

Which also means you need to add additional steps and dependencies to your repository for these file generations. I don't think this is the right solution, imho those two solutions should be complementary: have the ability to do this in different files if you want to adapt configurations to different use cases but also to factor many similar cases in a single file without depending on anything else. One solution shouldn't kick the other one out like it seems to be the case here, @goneri

I was well aware of this possibility and would have open the same issue as @Akasurde if I didn't found its one before opening mine.

[goneri]

Hi @Porkepix,

Each new line of code has to be maintained in the long term, especially for the inventory plugin. In addition, @next-jesusmanuelnavarro solution gives more freedom to the user. Also, the other inventory plugins behave the same way, you need one inventory file per account.

[next-jesusmanuelnavarro]

@goneri I didn't think on the way you did it but more taking advantage of the fact that when --inventory points to a directory, all files will be processed at once (all required *-aws_ec2.yaml files would be under inventory-dir/):

ansible-inventory -i inventory-dir/ --graph
ansible-playbook -i inventory-dir/ playbooks/my_playbook.yml

If my examples above work as expected, that is, they produce a single "compact" inventory with the hosts, groups, etc. organized as if transparently belonged to a single AWS account (is up to the user if setting them apart is required, with the help of tags, facts, etc.) then, I'd think that's a nice workaround, since another (previous) playbook (if / as required) could produce exactly the inventory files that would be required in each case out of templates, without the "main" ansible run procedure noticing the difference.

If, in the other hand, my examples above don't work as stated, then I would not consider my suggestion as a valid workaround.