Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
ENH: make it possible to specify "host" option for boto.connect_s3
While trying to crawl dandiarchive bucket with authentication, to fetch also files which are not publicly available, I have ran into <Error><Code>InvalidRequest</Code><Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.< for which discussion was ongoing in 2017: jschneier/django-storages#28 . A workaround which worked for me was to specify host option to boto.connect_s3 to point to the specific region. So with this fix now it would be possible to use it in the provider configuration, e.g. [provider:dandi-s3] url_re = s3://dandiarchive($|/.*) credential = dandi-s3-backup authentication_type = aws-s3 aws-s3_host = s3.us-east-2.amazonaws.com There might be other options we might want to add later on, so I did not store host in the attribute, but right within the dictionary of optional kwargs for connect_s3.
- Loading branch information