Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3Path: support for using access points #228

Open
nickrobinson251 opened this issue Dec 8, 2021 · 3 comments
Open

S3Path: support for using access points #228

nickrobinson251 opened this issue Dec 8, 2021 · 3 comments

Comments

@nickrobinson251
Copy link
Contributor

nickrobinson251 commented Dec 8, 2021

i would like to start accessing an S3 bucket via an "access point"

My code currently uses S3Paths, so i hoped it'd be as simple as changing S3Path("s3://bucket/object") -> S3Path("s3://arn:aws:s3:us-west-2:123456789012:accesspoint/test/object"), but it seems S3Path isn't able to handle access points being used in place of buckets; currently it mangles the "bucket" name:

julia> S3Path("s3://arn:aws:s3:us-west-2:123456789012:accesspoint/test/object")
p"s3://arn/test/object"

julia> ans.bucket
"arn"
@nickrobinson251
Copy link
Contributor Author

a workaround is to use a bucket-style alias for the access point
https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-points-alias.html

@nickrobinson251 nickrobinson251 changed the title S3Path: add support for using access points S3Path: add support for using access point ARNs? Dec 8, 2021
@nickrobinson251 nickrobinson251 changed the title S3Path: add support for using access point ARNs? S3Path: support for using access points Dec 8, 2021
@nickrobinson251
Copy link
Contributor Author

If we decide not to support the access point ARN and instead require the use of a bucket-style alias, then i think this issue can be resolved by just adding documentation about using ARNs with S3Path

@omus
Copy link
Member

omus commented Feb 17, 2022

Based upon the documentation I've read so far I think supporting access points would use the S3Path(bucket, key) constructor rather than using the URI based constructor (I've yet to see an example where "s3://$(bucket_arn)/..." is used). In fact this constructor already seems to work with ARNs:

julia> S3Path("s3://arn:aws:s3:us-west-2:123456789012:accesspoint/test", "object")
p"s3://arn:aws:s3:us-west-2:123456789012:accesspoint/test/object"

julia> S3Path("s3://arn:aws:s3:us-west-2:123456789012:accesspoint/test/object")
p"s3://arn/test/object"

I think to better support access points we should error when an ARN is provided via the URI constructor and not display an access point in the URI format as it cannot be parsed that way.

I'm also reminded of #226 here as this is another example of S3 paths being different from a file system path

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants