Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s3_bucket module always fails in eu-north-1 #2266

Closed
1 task done
vonschultz opened this issue Aug 29, 2024 · 3 comments · Fixed by #2267
Closed
1 task done

s3_bucket module always fails in eu-north-1 #2266

vonschultz opened this issue Aug 29, 2024 · 3 comments · Fixed by #2267

Comments

@vonschultz
Copy link

Summary

With the introduction of Transfer Acceleration support the s3_bucket module always fail in some regions, including eu-north-1. Transfer Acceleration is only supported in regions listed here: https://docs.aws.amazon.com/AmazonS3/latest/userguide/transfer-acceleration.html. For example, I can use s3_bucket in eu-west-1 without any errors.

This is closely related to #2180, but I've verified that the fix for that issue, #2202, does not solve my problem, possibly because I'm getting MethodNotAllowed instead of UnsupportedArgument.

Issue Type

Bug Report

Component Name

s3_bucket

Ansible Version

$ ansible --version
ansible [core 2.17.3]
  config file = /home/von/.ansible.cfg
  configured module search path = ['/home/von/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/von/src/embedl/ci/provisioning/.tox/ansible/lib/python3.10/site-packages/ansible
  ansible collection location = /home/von/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/von/src/embedl/ci/provisioning/.tox/ansible/bin/ansible
  python version = 3.10.14 (main, Apr  6 2024, 18:45:05) [GCC 9.4.0] (/home/von/src/embedl/ci/provisioning/.tox/ansible/bin/python)
  jinja version = 3.1.4
  libyaml = True

Collection Versions

$ ansible-galaxy collection list

# /home/von/src/embedl/ci/provisioning/.tox/ansible/lib/python3.10/site-packages/ansible_collections
Collection                               Version
---------------------------------------- -------
amazon.aws                               8.1.0  
ansible.netcommon                        6.1.3  
ansible.posix                            1.5.4  
ansible.utils                            4.1.0  
ansible.windows                          2.4.0  
arista.eos                               9.0.0  
awx.awx                                  24.6.1 
azure.azcollection                       2.6.0  
check_point.mgmt                         5.2.3  
chocolatey.chocolatey                    1.5.1  
cisco.aci                                2.10.1 
cisco.asa                                5.0.1  
cisco.dnac                               6.17.1 
cisco.intersight                         2.0.10 
cisco.ios                                8.0.0  
cisco.iosxr                              9.0.0  
cisco.ise                                2.9.3  
cisco.meraki                             2.18.1 
cisco.mso                                2.9.0  
cisco.nxos                               8.1.0  
cisco.ucs                                1.10.0 
cloud.common                             3.0.0  
cloudscale_ch.cloud                      2.4.0  
community.aws                            8.0.0  
community.ciscosmb                       1.0.9  
community.crypto                         2.21.1 
community.digitalocean                   1.26.0 
community.dns                            3.0.3  
community.docker                         3.12.1 
community.general                        9.3.0  
community.grafana                        1.9.1  
community.hashi_vault                    6.2.0  
community.hrobot                         2.0.1  
community.library_inventory_filtering_v1 1.0.1  
community.libvirt                        1.3.0  
community.mongodb                        1.7.6  
community.mysql                          3.9.0  
community.network                        5.0.3  
community.okd                            3.0.1  
community.postgresql                     3.4.1  
community.proxysql                       1.6.0  
community.rabbitmq                       1.3.0  
community.routeros                       2.18.0 
community.sap_libs                       1.4.2  
community.sops                           1.8.2  
community.vmware                         4.5.0  
community.windows                        2.2.0  
community.zabbix                         2.5.1  
containers.podman                        1.15.4 
cyberark.conjur                          1.3.0  
cyberark.pas                             1.0.27 
dellemc.enterprise_sonic                 2.4.0  
dellemc.openmanage                       9.5.0  
dellemc.powerflex                        2.5.0  
dellemc.unity                            2.0.0  
f5networks.f5_modules                    1.30.1 
fortinet.fortimanager                    2.6.0  
fortinet.fortios                         2.3.7  
frr.frr                                  2.0.2  
google.cloud                             1.3.0  
grafana.grafana                          5.4.0  
hetzner.hcloud                           3.1.1  
ibm.qradar                               3.0.0  
ibm.spectrum_virtualize                  2.0.0  
ibm.storage_virtualize                   2.4.1  
ieisystem.inmanage                       2.0.0  
infinidat.infinibox                      1.4.5  
infoblox.nios_modules                    1.6.1  
inspur.ispim                             2.2.3  
inspur.sm                                2.3.0  
junipernetworks.junos                    8.0.0  
kaytus.ksmanage                          1.2.2  
kubernetes.core                          3.2.0  
kubevirt.core                            1.5.0  
lowlydba.sqlserver                       2.3.3  
lvrfrc87.git_acp                         2.2.0  
microsoft.ad                             1.6.0  
netapp.cloudmanager                      21.22.1
netapp.ontap                             22.12.0
netapp.storagegrid                       21.12.0
netapp_eseries.santricity                1.4.0  
netbox.netbox                            3.19.1 
ngine_io.cloudstack                      2.3.0  
ngine_io.exoscale                        1.1.0  
openstack.cloud                          2.2.0  
openvswitch.openvswitch                  2.1.1  
ovirt.ovirt                              3.2.0  
purestorage.flasharray                   1.30.2 
purestorage.flashblade                   1.18.0 
sensu.sensu_go                           1.14.0 
splunk.es                                3.0.0  
t_systems_mms.icinga_director            2.0.1  
telekom_mms.icinga_director              2.1.2  
theforeman.foreman                       4.1.0  
vmware.vmware                            1.4.0  
vmware.vmware_rest                       3.0.1  
vultr.cloud                              1.13.0 
vyos.vyos                                4.1.0  
wti.remote                               1.0.5  

AWS SDK versions

$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location: /home/von/src/embedl/ci/provisioning/.tox/ansible/lib/python3.10/site-packages
Requires: 
Required-by: 
---
Name: boto3
Version: 1.35.8
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /home/von/src/embedl/ci/provisioning/.tox/ansible/lib/python3.10/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: 
---
Name: botocore
Version: 1.35.8
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /home/von/src/embedl/ci/provisioning/.tox/ansible/lib/python3.10/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed
CONFIG_FILE() = /home/von/.ansible.cfg
DEFAULT_HOST_LIST(env: ANSIBLE_INVENTORY) = ['/home/von/src/embedl/ci/provisioning/inventories/production']
DEFAULT_LOAD_CALLBACK_PLUGINS(/home/von/.ansible.cfg) = True
DEFAULT_STDOUT_CALLBACK(/home/von/.ansible.cfg) = yaml
DEFAULT_VAULT_IDENTITY_LIST(env: ANSIBLE_VAULT_IDENTITY_LIST) = ['prod@/home/von/.config/ansible-prod-vault.txt']
EDITOR(env: EDITOR) = emacsclient -t

OS / Environment

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.6 LTS
Release:	20.04
Codename:	focal

Steps to Reproduce

- amazon.aws.s3_bucket:
    name: ansibleissuedebugbucket
    region: eu-north-1
    state: present

Expected Results

Bucket is created, or no changes are made if the bucket already exists.

Actual Results

fatal: [s3.amazonaws.com]: FAILED! => changed=false 
  boto3_version: 1.35.8
  botocore_version: 1.35.8
  error:
    code: MethodNotAllowed
    message: The specified method is not allowed against this resource.
    method: GET
    resource_type: accelerate
  msg: 'Failed to fetch bucket transfer acceleration state: An error occurred (MethodNotAllowed) when calling the GetBucketAccelerateConfiguration operation: The specified method is not allowed against this resource.'
  response_metadata:
    host_id: qjOs6WQzYiENa7r0LypeOzfdSNCiA2c0WyB2+RBjfoTnim74BGo9wORVEYMwemfUIzLbxOx4Vw8=
    http_headers:
      content-type: application/xml
      date: Thu, 29 Aug 2024 14:42:45 GMT
      server: AmazonS3
      transfer-encoding: chunked
      x-amz-id-2: qjOs6WQzYiENa7r0LypeOzfdSNCiA2c0WyB2+RBjfoTnim74BGo9wORVEYMwemfUIzLbxOx4Vw8=
      x-amz-request-id: SR0RTXK1WMF38AB2
    http_status_code: 405
    request_id: SR0RTXK1WMF38AB2
    retry_attempts: 0

Code of Conduct

  • I agree to follow the Ansible Code of Conduct
@tremble
Copy link
Contributor

tremble commented Aug 29, 2024

Thanks for the report, could you test and see if swapping "UnsupportedArgument" for "MethodNotAllowed" (using the fix from #2202) fixes your issue?

@vonschultz
Copy link
Author

Yes, @tremble, I can confirm that changing UnsupportedArgument to MethodNotAllowed in aws/plugins/modules/s3_bucket.py (from #2202) does fix the issue.

@tremble
Copy link
Contributor

tremble commented Aug 29, 2024

@vonschultz There's a slightly different fix in #2267, if you're able to test it and comment over there it would be helpful.

Hopefully we'll be releasing 8.2.0 in the next week or so...

patchback bot pushed a commit that referenced this issue Aug 29, 2024
…er acceleration (#2267)

fixes: #2266
SUMMARY
While #2202 (combined with #2222) fixes the issue for the aws-gov partition, the aws partition throws a different Error (MethodNotAllowed).
I suspect the two different exceptions are a side effect of how AWS implement the Gov-Cloud partition vs the normal partition.
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
s3_bucket
ADDITIONAL INFORMATION
See also: #2266

Reviewed-by: GomathiselviS
(cherry picked from commit db552aa)
softwarefactory-project-zuul bot pushed a commit that referenced this issue Aug 29, 2024
…er acceleration (#2267) (#2269)

This is a backport of PR #2267 as merged into main (db552aa).
fixes: #2266
SUMMARY
While #2202 (combined with #2222) fixes the issue for the aws-gov partition, the aws partition throws a different Error (MethodNotAllowed).
I suspect the two different exceptions are a side effect of how AWS implement the Gov-Cloud partition vs the normal partition.
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
s3_bucket
ADDITIONAL INFORMATION
See also: #2266

Reviewed-by: Mark Chappell
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants