EC2 AMI Creation/Tagging Fails

[watroba]

Summary

When using the ec2_image module with the option for tagging specified, tagging of volumes fails due to the presences of ephemeral volumes in the AMI metadata. Those volumes don't have a populated EBS property nor related snapshots, so tagging fails.

The offending line seems to be in this file at line 525: https://github.com/ansible-collections/amazon.aws/blame/main/plugins/modules/ec2_ami.py

resources_to_tag.append(mapping.get('Ebs').get('SnapshotId'))

Recommend checking if the value returned by mapping.get('Ebs') is not 'None' before attempting to fetch the SnapshotId.

Issue Type

Bug Report

Component Name

ec2_image

Ansible Version

$ ansible --version

2.11.5

Collection Versions

$ ansible-galaxy collection list

# /usr/local/lib/python3.9/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    1.5.0
ansible.netcommon             2.4.0
ansible.posix                 1.3.0
ansible.utils                 2.4.0
ansible.windows               1.7.2
arista.eos                    2.2.0
awx.awx                       19.2.2
azure.azcollection            1.9.0
check_point.mgmt              2.0.0
chocolatey.chocolatey         1.1.0
cisco.aci                     2.0.0
cisco.asa                     2.0.3
cisco.intersight              1.0.17
cisco.ios                     2.4.0
cisco.iosxr                   2.4.0
cisco.meraki                  2.4.2
cisco.mso                     1.2.0
cisco.nso                     1.0.3
cisco.nxos                    2.5.1
cisco.ucs                     1.6.0
cloudscale_ch.cloud           2.2.0
community.aws                 1.5.0
community.azure               1.0.0
community.crypto              1.9.2
community.digitalocean        1.9.0
community.docker              1.9.1
community.fortios             1.0.0
community.general             3.6.0
community.google              1.0.0
community.grafana             1.2.1
community.hashi_vault         1.3.2
community.hrobot              1.1.1
community.kubernetes          1.2.1
community.kubevirt            1.0.0
community.libvirt             1.0.2
community.mongodb             1.3.1
community.mysql               2.1.1
community.network             3.0.0
community.okd                 1.1.2
community.postgresql          1.4.0
community.proxysql            1.2.0
community.rabbitmq            1.1.0
community.routeros            1.2.0
community.skydive             1.0.0
community.sops                1.1.0
community.vmware              1.13.0
community.windows             1.6.0
community.zabbix              1.4.0
containers.podman             1.7.0
cyberark.conjur               1.1.0
cyberark.pas                  1.0.7
dellemc.enterprise_sonic      1.1.0
dellemc.openmanage            3.6.0
dellemc.os10                  1.1.1
dellemc.os6                   1.0.7
dellemc.os9                   1.0.4
f5networks.f5_modules         1.11.0
fortinet.fortimanager         2.1.3
fortinet.fortios              2.1.2
frr.frr                       1.0.3
gluster.gluster               1.0.1
google.cloud                  1.0.2
hetzner.hcloud                1.6.0
hpe.nimble                    1.1.3
ibm.qradar                    1.0.3
infinidat.infinibox           1.2.4
inspur.sm                     1.3.0
junipernetworks.junos         2.5.0
kubernetes.core               1.2.1
mellanox.onyx                 1.0.0
netapp.aws                    21.6.0
netapp.azure                  21.8.1
netapp.cloudmanager           21.9.0
netapp.elementsw              21.6.1
netapp.ontap                  21.10.0
netapp.um_info                21.7.0
netapp_eseries.santricity     1.2.13
netbox.netbox                 3.1.1
ngine_io.cloudstack           2.1.0
ngine_io.exoscale             1.0.0
ngine_io.vultr                1.1.0
openstack.cloud               1.5.0
openvswitch.openvswitch       2.0.0
ovirt.ovirt                   1.6.2
purestorage.flasharray        1.10.0
purestorage.flashblade        1.6.0
sensu.sensu_go                1.12.0
servicenow.servicenow         1.0.6
splunk.es                     1.0.2
t_systems_mms.icinga_director 1.21.1
theforeman.foreman            2.2.0
vyos.vyos                     2.5.1
wti.remote                    1.0.1

# /root/.ansible/collections/ansible_collections
Collection    Version
------------- -------
amazon.aws    2.0.0
community.aws 1.5.0

AWS SDK versions

$ pip show boto boto3 botocore

boto3               1.18.44
botocore           1.21.44 

Configuration

$ ansible-config dump --only-changed

OS / Environment

Python3-Alpine Docker Image on RHEL GitLab Runner

Steps to Reproduce

- name: Create AMI
  ec2_ami:
    instance_id: "{{ infra_artifact.instance_id }}"
    state: present
    region: "{{ manifest.setup.region }}"
    name: "{{ ami_name }}"
    wait: yes
    wait_timeout: "{{ manifest.image.imaging_timeout | default(omit) }}" # default is 1200sec
    tags:
      CreatedBy: "{{ aws_user }}"
    description: "{{ manifest.image.ami_description }}"
  register: created_ami

Expected Results

When executing AMI creation, image should be created with snapshots tagged successfully. Ephemeral volumes should not be tagged (that doesn't work). Recommend ensuring that a volume has EBS data before attempting to fetch the snapshot id.

Actual Results

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: AttributeError: 'NoneType' object has no attribute 'get'
fatal: [localhost]: FAILED! => changed=false 
  module_stderr: |-
    Traceback (most recent call last):
      File "/root/.ansible/tmp/ansible-tmp-1632318788.9606068-145-264789969155637/AnsiballZ_ec2_ami.py", line 100, in <module>
        _ansiballz_main()
      File "/root/.ansible/tmp/ansible-tmp-1632318788.9606068-145-264789969155637/AnsiballZ_ec2_ami.py", line 92, in _ansiballz_main
        invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
      File "/root/.ansible/tmp/ansible-tmp-1632318788.9606068-145-264789969155637/AnsiballZ_ec2_ami.py", line 40, in invoke_module
        runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.ec2_ami', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.ec2_ami', _modlib_path=modlib_path),
      File "/usr/local/lib/python3.9/runpy.py", line 210, in run_module
        return _run_module_code(code, init_globals, run_name, mod_spec)
      File "/usr/local/lib/python3.9/runpy.py", line 97, in _run_module_code
        _run_code(code, mod_globals, init_globals,
      File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/tmp/ansible_ec2_ami_payload_9wbc0zrk/ansible_ec2_ami_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_ami.py", line 773, in <module>
      File "/tmp/ansible_ec2_ami_payload_9wbc0zrk/ansible_ec2_ami_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_ami.py", line 769, in main
      File "/tmp/ansible_ec2_ami_payload_9wbc0zrk/ansible_ec2_ami_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_ami.py", line 525, in create_image
    AttributeError: 'NoneType' object has no attribute 'get'
  module_stdout: ''
  msg: |-
    MODULE FAILURE
    See stdout/stderr for the exact error
  rc: 1

When verifying against a RHEL AMI that does not have ephemeral volumes in the AMI, image creation succeeds. Output of the EBS volumes reference in that image:

PS C:\Users\me> $imgrhel.BlockDeviceMapping

DeviceName Ebs                             NoDevice VirtualName
---------- ---                             -------- -----------
/dev/sda1  Amazon.EC2.Model.EbsBlockDevice

After failing to tag a Windows AMI during creation, which does have ephemeral volumes, I verified that, while the volumes are listed, they do not have any EBS data, so would fail to tag.

DeviceName Ebs                             NoDevice VirtualName
---------- ---                             -------- -----------
/dev/sda1  Amazon.EC2.Model.EbsBlockDevice
xvdf       Amazon.EC2.Model.EbsBlockDevice
xvdca                                               ephemeral0
xvdcb                                               ephemeral1
xvdcc                                               ephemeral2
xvdcd                                               ephemeral3
xvdce                                               ephemeral4
xvdcf                                               ephemeral5
xvdcg                                               ephemeral6
xvdch                                               ephemeral7
xvdci                                               ephemeral8
xvdcj                                               ephemeral9
xvdck                                               ephemeral10
xvdcl                                               ephemeral11
xvdcm                                               ephemeral12
xvdcn                                               ephemeral13
xvdco                                               ephemeral14
xvdcp                                               ephemeral15
xvdcq                                               ephemeral16
xvdcr                                               ephemeral17
xvdcs                                               ephemeral18
xvdct                                               ephemeral19
xvdcu                                               ephemeral20
xvdcv                                               ephemeral21
xvdcw                                               ephemeral22
xvdcx                                               ephemeral23
xvdcy                                               ephemeral24
xvdcz                                               ephemeral25

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[ansibullbot]

Files identified in the description:
None

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[tremble]

Hi @watroba

Thanks for taking the time to report this issue. #520 should fix the issue you're seeing, and updates our integration tests to cover the issue you've hit.

Specifically a7e0f20

If you're able to confirm that #520 fixes your issue, adding a comment on the PR will help the folks reviewing know if it's the right change. (If it doesn't fix the issue, then it'll also let me know there's another issue somewhere).