AWS S3 module Copy function does not seem to work #602

AKSarav · 2021-12-29T06:18:21Z

Summary

I was referring to the Official documentation available on this page

aws_s3 documentation ansible

there was an example for copying objects from one bucket to another

While throws an exception as follows

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: UnboundLocalError: local variable 'tags' referenced before assignment fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756099.59568-98912-227152518535421/AnsiballZ_aws_s3.py\", line 100, in <module>\n _ansiballz_main()\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756099.59568-98912-227152518535421/AnsiballZ_aws_s3.py\", line 92, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756099.59568-98912-227152518535421/AnsiballZ_aws_s3.py\", line 40, in invoke_module\n runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.aws_s3', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.aws_s3', _modlib_path=modlib_path),\n File \"/usr/local/Cellar/[email protected]/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 209, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/local/Cellar/[email protected]/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 96, in _run_module_code\n _run_code(code, mod_globals, init_globals,\n File \"/usr/local/Cellar/[email protected]/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 86, in _run_code\n exec(code, run_globals)\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_8ki221vk/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 1171, in <module>\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_8ki221vk/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 1165, in main\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_8ki221vk/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 757, in copy_object_to_bucket\nUnboundLocalError: local variable 'tags' referenced before assignment\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

I have the latest version of Ansible. Ansible 11

My Boto Versions are above the recommended version on the article

boto3 1.17.58
botocore 1.20.58

Except for mode: copy all other modes seem to work properly.

Is this a bug or documentation is wrong. Can you clarify

here is the complete playbook

---
- name: Copy objects from one S3 bucket to another bucket using Ansible
  hosts: localhost

  tasks:
  - name: Copy Objects from one S3 bucket to another
    amazon.aws.aws_s3:
      bucket: destbucket
      profile: personal
      object: filename
      mode: copy
      copy_src: 
        bucket: sourcebucket
        object: filename
    register: copyresult

  - debug: msg="{{ copyresult.msg }}"

Issue Type

Bug Report

Component Name

amazon.aws.aws_s3

Ansible Version

$ ansible --version
ansible [core 2.11.6]
  config file = /Users/saravananthangaraj/.ansible.cfg
  configured module search path = ['/Users/saravananthangaraj/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/Cellar/ansible/4.8.0/libexec/lib/python3.10/site-packages/ansible
  ansible collection location = /Users/saravananthangaraj/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.10.0 (default, Oct 13 2021, 06:44:31) [Clang 12.0.0 (clang-1200.0.32.29)]
  jinja version = 3.0.2
  libyaml = True

Collection Versions

$ ansible-galaxy collection list
Collection                    Version
----------------------------- -------
amazon.aws                    1.5.1
ansible.netcommon             2.4.0
ansible.posix                 1.3.0
ansible.utils                 2.4.2
ansible.windows               1.7.3
arista.eos                    2.2.0
awx.awx                       19.4.0
azure.azcollection            1.10.0
check_point.mgmt              2.1.1
chocolatey.chocolatey         1.1.0
cisco.aci                     2.1.0
cisco.asa                     2.1.0
cisco.intersight              1.0.17
cisco.ios                     2.5.0
cisco.iosxr                   2.5.0
cisco.meraki                  2.5.0
cisco.mso                     1.2.0
cisco.nso                     1.0.3
cisco.nxos                    2.7.0
cisco.ucs                     1.6.0
cloudscale_ch.cloud           2.2.0
community.aws                 1.5.0
community.azure               1.1.0
community.crypto              1.9.6
community.digitalocean        1.11.0
community.docker              1.10.0
community.fortios             1.0.0
community.general             3.8.1
community.google              1.0.0
community.grafana             1.2.3
community.hashi_vault         1.4.1
community.hrobot              1.2.0
community.kubernetes          1.2.1
community.kubevirt            1.0.0
community.libvirt             1.0.2
community.mongodb             1.3.1
community.mysql               2.3.1
community.network             3.0.0
community.okd                 1.1.2
community.postgresql          1.5.0
community.proxysql            1.3.0
community.rabbitmq            1.1.0
community.routeros            1.2.0
community.skydive             1.0.0
community.sops                1.1.0
community.vmware              1.15.0
community.windows             1.7.0
community.zabbix              1.5.0
containers.podman             1.8.1
cyberark.conjur               1.1.0
cyberark.pas                  1.0.7
dellemc.enterprise_sonic      1.1.0
dellemc.openmanage            3.6.0
dellemc.os10                  1.1.1
dellemc.os6                   1.0.7
dellemc.os9                   1.0.4
f5networks.f5_modules         1.12.0
fortinet.fortimanager         2.1.3
fortinet.fortios              2.1.2
frr.frr                       1.0.3
gluster.gluster               1.0.2
google.cloud                  1.0.2
hetzner.hcloud                1.6.0
hpe.nimble                    1.1.3
ibm.qradar                    1.0.3
infinidat.infinibox           1.2.4
inspur.sm                     1.3.0
junipernetworks.junos         2.6.0
kubernetes.core               1.2.1
mellanox.onyx                 1.0.0
netapp.aws                    21.6.0
netapp.azure                  21.9.0
netapp.cloudmanager           21.11.0
netapp.elementsw              21.6.1
netapp.ontap                  21.12.0
netapp.um_info                21.7.0
netapp_eseries.santricity     1.2.13
netbox.netbox                 3.3.0
ngine_io.cloudstack           2.2.2
ngine_io.exoscale             1.0.0
ngine_io.vultr                1.1.0
openstack.cloud               1.5.1
openvswitch.openvswitch       2.0.2
ovirt.ovirt                   1.6.4
purestorage.flasharray        1.11.0
purestorage.flashblade        1.7.0
sensu.sensu_go                1.12.0
servicenow.servicenow         1.0.6
splunk.es                     1.0.2
t_systems_mms.icinga_director 1.23.0
theforeman.foreman            2.2.0
vyos.vyos                     2.6.0
wti.remote                    1.0.1

AWS SDK versions

$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location: /usr/local/lib/python3.9/site-packages
Requires:
Required-by:
---
Name: boto3
Version: 1.17.58
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: ec2.py
---
Name: botocore
Version: 1.20.58
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed
INVENTORY_ENABLED(/Users/saravananthangaraj/.ansible.cfg) = ['host_list', 'script', 'auto', 'yaml', 'ini', 'toml', 'aws_ec2']

OS / Environment

MAC OS CATALINA 10.15.7

Steps to Reproduce

---
- name: Copy objects from one S3 bucket to another bucket using Ansible
  hosts: localhost

  tasks:
  - name: Copy Objects from one S3 bucket to another
    amazon.aws.aws_s3:
      bucket: destbucket
      profile: personal
      object: filename
      mode: copy
      copy_src: 
        bucket: sourcebucket
        object: filename
    register: copyresult

  - debug: msg="{{ copyresult.msg }}"

Expected Results

The file should be from source bucket to destination bucket

Actual Results

Getting these exceptions

Exception 1

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756980.591985-99892-110059587658594/AnsiballZ_aws_s3.py\", line 100, in <module>\n _ansiballz_main()\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756980.591985-99892-110059587658594/AnsiballZ_aws_s3.py\", line 92, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756980.591985-99892-110059587658594/AnsiballZ_aws_s3.py\", line 40, in invoke_module\n runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.aws_s3', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.aws_s3', _modlib_path=modlib_path),\n File \"/usr/local/Cellar/[email protected]/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 209, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/local/Cellar/[email protected]/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 96, in _run_module_code\n _run_code(code, mod_globals, init_globals,\n File \"/usr/local/Cellar/[email protected]/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 86, in _run_code\n exec(code, run_globals)\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_u1ilxbza/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 1171, in <module>\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_u1ilxbza/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 1157, in main\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_u1ilxbza/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 418, in get_etag\n File \"/usr/local/Cellar/ansible/4.8.0/libexec/lib/python3.10/site-packages/botocore/client.py\", line 388, in _api_call\n return self._make_api_call(operation_name, kwargs)\n File \"/usr/local/Cellar/ansible/4.8.0/libexec/lib/python3.10/site-packages/botocore/client.py\", line 708, in _make_api_call\n raise error_class(parsed_response, operation_name)\nbotocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

Exception 2

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: UnboundLocalError: local variable 'tags' referenced before assignment fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756099.59568-98912-227152518535421/AnsiballZ_aws_s3.py\", line 100, in <module>\n _ansiballz_main()\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756099.59568-98912-227152518535421/AnsiballZ_aws_s3.py\", line 92, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/Users/saravananthangaraj/.ansible/tmp/ansible-tmp-1640756099.59568-98912-227152518535421/AnsiballZ_aws_s3.py\", line 40, in invoke_module\n runpy.run_module(mod_name='ansible_collections.amazon.aws.plugins.modules.aws_s3', init_globals=dict(_module_fqn='ansible_collections.amazon.aws.plugins.modules.aws_s3', _modlib_path=modlib_path),\n File \"/usr/local/Cellar/[email protected]/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 209, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/local/Cellar/[email protected]/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 96, in _run_module_code\n _run_code(code, mod_globals, init_globals,\n File \"/usr/local/Cellar/[email protected]/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py\", line 86, in _run_code\n exec(code, run_globals)\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_8ki221vk/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 1171, in <module>\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_8ki221vk/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 1165, in main\n File \"/var/folders/s0/jddty2fd3bj9q_d5ng9rp5dr0000gn/T/ansible_amazon.aws.aws_s3_payload_8ki221vk/ansible_amazon.aws.aws_s3_payload.zip/ansible_collections/amazon/aws/plugins/modules/aws_s3.py\", line 757, in copy_object_to_bucket\nUnboundLocalError: local variable 'tags' referenced before assignment\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

Code of Conduct

I agree to follow the Ansible Code of Conduct

The text was updated successfully, but these errors were encountered:

Akasurde · 2021-12-29T07:22:00Z

cc @abikouo Could you please take a look at this? Got introduce via #359

acelinkio · 2021-12-29T20:16:42Z

+1. Also encountering this error.

ansible 5.0.1
ansible-core 2.12.1
collection amazon.aws 3.0.0

abikouo · 2022-01-03T11:20:29Z

@AKSarav Thanks for taking the time to report this issue.
The exception is raised when trying to copy a key not existing in the source bucket,
#603 will fix this, could you please try it ?

aws_s3 - fix issue when copy missing key from bucket SUMMARY #602 ISSUE TYPE Bugfix Pull Request COMPONENT NAME aws_s3 ADDITIONAL INFORMATION Reviewed-by: Alina Buzachis <None> Reviewed-by: None <None>

abikouo self-assigned this Jan 3, 2022

abikouo mentioned this issue Jan 3, 2022

aws_s3 - fix issue when copy missing key from bucket #603

Merged

abikouo closed this as completed Jan 7, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AWS S3 module Copy function does not seem to work #602

AWS S3 module Copy function does not seem to work #602

AKSarav commented Dec 29, 2021 •

edited by Akasurde

Loading

Akasurde commented Dec 29, 2021

acelinkio commented Dec 29, 2021

abikouo commented Jan 3, 2022

AWS S3 module Copy function does not seem to work #602

AWS S3 module Copy function does not seem to work #602

Comments

AKSarav commented Dec 29, 2021 • edited by Akasurde Loading

Summary

Issue Type

Component Name

Ansible Version

Collection Versions

AWS SDK versions

Configuration

OS / Environment

Steps to Reproduce

Expected Results

Actual Results

Exception 1

Exception 2

Code of Conduct

Akasurde commented Dec 29, 2021

acelinkio commented Dec 29, 2021

abikouo commented Jan 3, 2022

AKSarav commented Dec 29, 2021 •

edited by Akasurde

Loading